Add SandboxToolset for isolated agent code execution in common.ai#68847
Add SandboxToolset for isolated agent code execution in common.ai#68847zozo123 wants to merge 10 commits into
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
|
boring-cyborg
Bot
added
area:dev-tools
area:providers
backport-to-v3-3-test
|
Converted to draft to ensure, this is discussed on the mailing list first. My first main concern (without looking into much details) is how would this integrate with Common AI Provider and why should it be separate provider instead of part of |
|
Agrees with @kaxil - we also need to know that we have real person behind who is comitted to maintain it - not only drop AI generatad code, so discussion on devlist is a minimum, ideally some good discussion about why this is a good idea - and explaining (in human words) some rationale behind it, and choice of the sandbox "providers" (which cannot be named providers due to obvious conflict) - ideally you should present your idea, how it is already used and what actual real-use problems it solves at our dev call. |
|
Thanks @kaxil @potiuk — pivoted per your feedback. It's no longer a separate provider: it's now a Not single-vendor: the same per-session-microVM primitive now exists in AWS Lambda microVMs (Firecracker, shipped 2026-06-22 for "user or AI-generated code"), Daytona, E2B, Modal, and islo. The local backend needs no credentials. On maintenance: I'll own it — I'm at Incredibuild and islo (islo.dev) is our product, so there's a committed maintainer and real usage behind it, not a drive-by AI PR. The reference backend I commit to is islo. It already works end to end: a real agent's |
You will need to elaborate on that. Since you are new to the community we need to make sure there is a long term promise/ownership. Check out guidelines for adding new providers |
|
Thanks @eladkal, that's a fair bar and I appreciate you spelling it out. Two named maintainers at Incredibuild will own this long term: myself (@zozo123, yossi.eliaz@incredibuild.com) and Yuval Raz (yuval.r@incredibuild.com). support@incredibuild.com is our team alias, also committed and monitored by the islo team for issues and escalations. Incredibuild builds islo (islo.dev), a commercial product, so we have a direct reason to keep this alive. We'll land the first commit and maintain both the The backend interface is pluggable so any vendor can own its own. We plan to invite E2B, AWS (Lambda microVMs), Daytona, and Tensorlake; whether they pick it up is on them. We're not asking the community to carry single-vendor code. We'll answer issues and PRs, and meet the incubation and governance commitments in ACCEPTING_PROVIDERS. One scope note: this is a I'm new here, so who's the right committer to approach as a sponsor? I'll start the |
Should there be a need for this new provider I am happy to be the sponsor but lets take it one step at a time. The item in #68847 (comment) must be answered first. |
zozo123
changed the title
b6f8b95 to
d387a5c
Compare
|
Still conflicts:( |
Give common.ai agents a run_code tool that executes model-generated Python off the Airflow worker, in a disposable per-run microVM, closing the isolation gap in code mode (whose run_code runs in-process on the worker with full credentials). A vendor-neutral SandboxBackend contract ships two backends: Docker Sandboxes via the sbx CLI (local microVM) and islo (hosted microVM). Airflow injects none of its context, connections, or worker environment into the sandbox; the islo backend keeps outbound network off by default.
Give each agent run its own sandbox so concurrent runs cannot share or tear down each other's; stream sandbox command output with a hard cap so generated code cannot exhaust worker memory; name the tool run_python_in_sandbox so it coexists with the run_code meta-tool that Monty code_mode reserves; report a run as timed out only on GNU timeout's unambiguous exit 124; clean up an orphaned microVM and workspace when provisioning fails; cap islo output like the sbx backend and keep timeout-driven teardown best-effort; and state the islo timeout honestly - it is enforced by the backend's poll deadline, not by the islo API.
The toolsets page is not a supported provider how-to guide path, which prevents static provider validation from succeeding.
The rebased provider dependency changes must match the current workspace metadata so CI resolves the same dependency graph.
Review feedback: GNU timeout exits 137 instead of 124 when generated code ignores SIGTERM and --kill-after escalates, so a stubborn timeout was reported as timed_out=False and indistinguishable from an OOM kill. Exit 137 now counts as a timeout when the call also outlived its budget. The passthrough args validator let a tool call without a string "code" reach call_tool and fail the task with KeyError; a real schema validator now turns malformed calls into retryable validation errors so max_retries works as intended.
d387a5c to
8d54768
Compare
|
Rebased onto latest
Ready for review again. |
4 participants
Summary
Adds a
SandboxToolsetto the Common AI provider. It gives an agent a code-execution tool that runs model-generated Python in a separate sandbox with a full Python runtime and the dependencies supplied by that environment.This complements the restricted Monty code mode added by #68407; it is not an Airflow executor and does not implement the future whole-task execution tracked by #68845.
Placement in Airflow's sandboxed-execution model
These are separate execution boundaries, not one progressively stronger API:
SandboxOperator, tracked by #69862ExecuteTaskworkloadSandboxExecutor, tracked by #68845Non-goals
This PR does not:
AgentOperator, LLM calls, or the complete Airflow task off the worker;The
SandboxBackendintroduced here is a narrow agent code-execution adapter. A future executor should use Airflow's existingBaseExecutorandExecuteTaskcontracts.Relationship to Monty code mode
Monty code mode is deny-by-default: generated Monty code cannot directly access the worker filesystem, environment, or network. It can invoke only registered tools, which continue to run on the worker with the capabilities explicitly granted by the DAG author.
SandboxToolsetaddresses a different use case: generated code that needs full Python, third-party packages, a filesystem, or heavier computation in a separate execution boundary. It is an alternative code-execution surface, not evidence that Monty directly exposes worker credentials.Design and hardening
SandboxBackendcontract providescreate,run, anddestroy.timeoutexpiry and SIGKILL escalation, without misclassifying an immediate exit 137.KeyErrorfailures.Initial backends
sbxCLI, as a host-local microVM backend.The abstraction is open to independently maintained Daytona, Tensorlake, E2B, and other adapters, but this PR does not claim uniform provider behavior beyond its tested contract.
Prerequisites
sbxCLI on a supported worker host. Linux workers require KVM/nested-virtualization support.apache-airflow-providers-common-ai[sandbox-islo]and configure an Airflow connection or the supported SDK environment credentials.Verification
Live end-to-end checks against both initial backends covered basic execution, nonzero exits, stderr, file persistence across calls, timeouts, and teardown.
The final rebased branch was also validated with:
uv lock --checkandgit diff --check.Continuous live service conformance can be expanded separately without widening this PR into the future operator/executor work.
Maintenance
Maintained by Yossi Eliaz (@zozo123) and Yuval Raz at Incredibuild/islo, with the islo team providing direct ownership of the hosted adapter. Other hosted adapters should preferably have explicit vendor or third-party ownership.
Related work
SandboxExecutortracker.Was generative AI tooling used to co-author this PR?
Generated with generative AI assistance following the project guidelines.