Add SandboxToolset for isolated agent code execution in common.ai

[zozo123]

Summary

Adds a SandboxToolset to the Common AI provider. It gives an agent a code-execution tool that runs model-generated Python in a separate sandbox with a full Python runtime and the dependencies supplied by that environment.

This complements the restricted Monty code mode added by #68407; it is not an Airflow executor and does not implement the future whole-task execution tracked by #68845.

Placement in Airflow's sandboxed-execution model

These are separate execution boundaries, not one progressively stronger API:

Surface	What moves into the sandbox	Status
Restricted code orchestration	Monty-compatible glue code that invokes registered tools	#68407, merged
Sandboxed code execution as a tool	Model-generated full Python while the agent remains outside	This PR
Sandboxed command/job execution	One author-defined command or job, potentially through a future resumable/deferrable operator	Future SandboxOperator, tracked by #69862
Sandboxed TaskInstance execution	The complete Task SDK ExecuteTask workload	Future SandboxExecutor, tracked by #68845

Non-goals

This PR does not:

• move AgentOperator, LLM calls, or the complete Airflow task off the worker;
• release the Airflow worker/task slot while sandbox code is running;
• define an Airflow executor SPI;
• provide scheduler restart adoption, durable remote task state, or Task SDK workload delivery;
• claim that all sandbox vendors provide identical isolation or lifecycle guarantees.

The SandboxBackend introduced here is a narrow agent code-execution adapter. A future executor should use Airflow's existing BaseExecutor and ExecuteTask contracts.

Relationship to Monty code mode

Monty code mode is deny-by-default: generated Monty code cannot directly access the worker filesystem, environment, or network. It can invoke only registered tools, which continue to run on the worker with the capabilities explicitly granted by the DAG author.

SandboxToolset addresses a different use case: generated code that needs full Python, third-party packages, a filesystem, or heavier computation in a separate execution boundary. It is an alternative code-execution surface, not evidence that Monty directly exposes worker credentials.

Design and hardening

• A narrow SandboxBackend contract provides create, run, and destroy.
• Each agent run gets a lazily created sandbox, reused across that run's code calls and destroyed afterward; concurrent runs do not share lifecycle state.
• Generated-code output is streamed with a hard cap before it reaches worker memory.
• Provisioning rollback and best-effort teardown clean up failed or timed-out sandboxes.
• Timeout reporting covers normal GNU timeout expiry and SIGKILL escalation, without misclassifying an immediate exit 137.
• Tool arguments are schema-validated so malformed model calls become retryable validation errors instead of task-level KeyError failures.
• Airflow does not deliberately inject its context, connections, credentials, or worker environment into the sandbox.
• Hosted sandbox services remain an additional data-processing and trust boundary: generated code may contain prompt content or prior tool results, and execution output returns to the agent/LLM.
• Files persist inside the per-run sandbox between tool calls. This narrow API intentionally does not introduce an Airflow artifact-transfer contract.

Initial backends

• Docker Sandboxes, through the sbx CLI, as a host-local microVM backend.
• islo, as a hosted microVM backend with outbound internet disabled by default by this adapter.

The abstraction is open to independently maintained Daytona, Tensorlake, E2B, and other adapters, but this PR does not claim uniform provider behavior beyond its tested contract.

Prerequisites

• Docker Sandboxes backend: install and authenticate/configure the sbx CLI on a supported worker host. Linux workers require KVM/nested-virtualization support.
• islo backend: install apache-airflow-providers-common-ai[sandbox-islo] and configure an Airflow connection or the supported SDK environment credentials.

Verification

Live end-to-end checks against both initial backends covered basic execution, nonzero exits, stderr, file persistence across calls, timeouts, and teardown.

The final rebased branch was also validated with:

• the complete Common AI unit suite: 1,117 passed on Python 3.10;
• Common AI mypy: 160 source files, no issues;
• provider-wide manual mypy and all changed-file pre-commit/manual hooks;
• a clean Common AI documentation build and spellcheck;
• Common AI wheel and sdist builds, including wheel-from-sdist validation;
• source-free installation of the Airflow, Airflow Core, Task SDK, and provider wheels, followed by the provider distribution verifier: 1,209 classes imported and all distributions importable;
• uv lock --check and git diff --check.

Continuous live service conformance can be expanded separately without widening this PR into the future operator/executor work.

Maintenance

Maintained by Yossi Eliaz (@zozo123) and Yuval Raz at Incredibuild/islo, with the islo team providing direct ownership of the hosted adapter. Other hosted adapters should preferably have explicit vendor or third-party ownership.

Related work

• Add code mode (Monty sandbox) to common.ai AgentOperator #68407 — Monty-based restricted code orchestration.
• Track a future SandboxExecutor for isolated Task SDK workloads #68845 — future whole-task SandboxExecutor tracker.
• https://github.com/zozo123/airflow-provider-sandbox — experimental executor prototype, separate from this PR.

---

Was generative AI tooling used to co-author this PR?

• Yes — Claude Code (Opus 4.8) and OpenAI Codex

Generated with generative AI assistance following the project guidelines.

[boring-cyborg]

Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide
Here are some useful points:

• Pay attention to the quality of your code (ruff, mypy and type annotations). Our prek-hooks will help you with that.
• In case of a new feature add useful documentation (in docstrings or in docs/ directory). Adding a new operator? Check this short guide Consider adding an example Dag that shows how users should use it.
• Consider using Breeze environment for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
• Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
• Please follow ASF Code of Conduct for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
• Be sure to read the Airflow Coding style.
• Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
  Apache Airflow is a community-driven project and together we are making it better 🚀.
  In case of doubts contact the developers at:
  Mailing List: dev@airflow.apache.org
  Slack: https://s.apache.org/airflow-slack

[kaxil]

Converted to draft to ensure, this is discussed on the mailing list first. My first main concern (without looking into much details) is how would this integrate with Common AI Provider and why should it be separate provider instead of part of common.ai.

[potiuk]

Agrees with @kaxil - we also need to know that we have real person behind who is comitted to maintain it - not only drop AI generatad code, so discussion on devlist is a minimum, ideally some good discussion about why this is a good idea - and explaining (in human words) some rationale behind it, and choice of the sandbox "providers" (which cannot be named providers due to obvious conflict) - ideally you should present your idea, how it is already used and what actual real-use problems it solves at our dev call.

[zozo123]

Thanks @kaxil @potiuk — pivoted per your feedback.

It's no longer a separate provider: it's now a SandboxToolset inside common.ai (a normal pydantic-ai toolset, like SQLToolset/MCPToolset). It adds a run_code tool that runs the agent's Python in a real per-session microVM — off the worker — closing code_mode's gap, where the Monty run_code still executes in-process on the worker with full creds. The small create/run/poll/destroy backend contract lives in common.ai too, and is deliberately not named "providers".

Not single-vendor: the same per-session-microVM primitive now exists in AWS Lambda microVMs (Firecracker, shipped 2026-06-22 for "user or AI-generated code"), Daytona, E2B, Modal, and islo. The local backend needs no credentials.

On maintenance: I'll own it — I'm at Incredibuild and islo (islo.dev) is our product, so there's a committed maintainer and real usage behind it, not a drive-by AI PR. The reference backend I commit to is islo.

It already works end to end: a real agent's run_code executing in a live islo microVM, including through AgentOperator in airflow dags test. Happy to walk through the rationale and demo it live at a dev call — just point me at a slot.

[eladkal]

On maintenance: I'll own it — I'm at Incredibuild and islo (islo.dev) is our product, so there's a committed maintainer and real usage behind it, not a drive-by AI PR. The reference backend I commit to is islo.

You will need to elaborate on that. Since you are new to the community we need to make sure there is a long term promise/ownership. Check out guidelines for adding new providers
https://github.com/apache/airflow/blob/main/providers/ACCEPTING_PROVIDERS.rst#approval-process

[zozo123]

Thanks @eladkal, that's a fair bar and I appreciate you spelling it out.

Two named maintainers at Incredibuild will own this long term: myself (@zozo123, yossi.eliaz@incredibuild.com) and Yuval Raz (yuval.r@incredibuild.com). support@incredibuild.com is our team alias, also committed and monitored by the islo team for issues and escalations. Incredibuild builds islo (islo.dev), a commercial product, so we have a direct reason to keep this alive. We'll land the first commit and maintain both the SandboxToolset framework and the islo backend.

The backend interface is pluggable so any vendor can own its own. We plan to invite E2B, AWS (Lambda microVMs), Daytona, and Tensorlake; whether they pick it up is on them. We're not asking the community to carry single-vendor code.

We'll answer issues and PRs, and meet the incubation and governance commitments in ACCEPTING_PROVIDERS.

One scope note: this is a SandboxToolset added to common.ai, not a new top-level provider.

I'm new here, so who's the right committer to approach as a sponsor? I'll start the [DISCUSS] thread next, and I'm happy to demo a live islo run at a dev call.

[eladkal]

I'm new here, so who's the right committer to approach as a sponsor? I'll start the [DISCUSS] thread next, and I'm happy to demo a live islo run at a dev call.

Should there be a need for this new provider I am happy to be the sponsor but lets take it one step at a time.

The item in #68847 (comment) must be answered first.