Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions docs/design/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -354,6 +354,30 @@ Scheduler etiquette learned live: `--exclude` is disallowed on Torch and node
pinning (`-w`) queues behind reservations — the orchestrator requests
partitions and lets the scheduler place jobs, never nodes.

## Environments and containers (decided with Mengye, 2026-08-06)

Torch supports exactly one container runtime — Apptainer (Singularity); no
Docker daemon exists ([cluster docs](https://services.rt.nyu.edu/docs/hpc/containers/intro/)).
Containers are applied where they pay, not uniformly:

| What | Environment | Why |
| --- | --- | --- |
| Orchestrator tick | host `uv` venv, deliberately NOT containerized | it exists to drive the host's `sbatch`/`sacct`/`squeue`; binding Slurm binaries, config, and the munge socket into a container is well-known friction, bought for a four-dependency pure-Python loop that needs none of it |
| Experiments | **per-target Apptainer image, declared in the target's contract** (Mengye's call) | each researched repo owns its dependency world; a pinned image makes the contract's "deterministic and re-runnable" promise stronger (same digest at claim time and at CI re-verification), and GPU runs use the supported `apptainer exec --nv` path. Targets without an image declare none and run in their own uv-managed env |
| Agent sessions | host binary today; Apptainer `--no-home --bind <workspace>` is the designated hardening step | this is the named mechanism for the threat model's accepted residual risk: a contained session cannot read same-user absolute paths (key files, other runs), closing the gap the per-run HOME redirect only narrows |

The contract grows an optional `environment` block (phase 5):

```yaml
environment:
container: /shared/images/jepa-agent-2026-08.sif # or docker://... to pull
```

Image files live on the shared filesystem next to the state root, referenced
by absolute path (pin by content digest once images churn). The experiment
launcher wraps the benchmark command in `apptainer exec --nv <image> ...`
when the block is present, and runs it bare otherwise.

## Threat model

- **Untrusted text.** Task sources are attacker-writable once target repos are
Expand Down
2 changes: 2 additions & 0 deletions docs/roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,8 @@ the research repos' porting timelines; mistakes are free; adversarial testing
(injection via issues) is staged here, never on research repos.

- [ ] Bot opt-in on the pilot: Write grant + token scope
- [ ] Contract `environment.container` (per-target Apptainer image; see
architecture "Environments and containers") wired into experiment launch
- [ ] Task pinning: target SHA + contract hash at task start, re-validated each
poll; baseline re-run at merge-base
- [ ] Full loop on the pilot benchmarks; bounded iterations; PR with results
Expand Down
Loading