Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,21 @@ Versions follow [SemVer](https://semver.org).

## [Unreleased]

- Bump the pinned Codex CLI from 0.130.0 to 0.160.0 and update its Linux
archive digest. All contained Codex roles bind a kernel-owned, read-only
managed-hooks-only policy; uncontained sessions disable lifecycle hooks and
plugins (including built-in cleanup hooks) because this CLI has no per-session
requirements-file override. Rebuild user config before every launch/resume to
discard persisted hook trust, while
retaining session history. Reject the hook-trust bypass flag.
Upgrading: no manual action or run-record migration is needed; existing
session homes are sanitized automatically on their next launch. The policy
ships with the kernel and needs no image rebuild or system configuration.
- Record the Codex 0.160.0 project-config finding in `SECURITY.md`: writable
sessions auto-trust project config, so repository model settings can apply;
project provider settings are filtered. The external containment boundary
remains necessary and is unchanged.

- Include GPU count and resolved GPU type in dispatched eval and baseline cache identity, including budget discounts. Legacy eval slots and baseline entries are cache misses. Upgrading: An eval dispatched by the previous kernel and still in flight at upgrade is measured again once under the new cache key (no extra budget charge). To avoid the extra run, upgrade when no evals are in flight: `touch <root>/PAUSE` (stops wakes, so no new evals are dispatched), wait until no eval jobs remain in the queue, upgrade, then `rm <root>/PAUSE` and run `outerloop start`.

- Park launch capacity refusals in capacity wait when an immediate resume is
Expand Down
61 changes: 61 additions & 0 deletions SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,64 @@ LLM APIs and GPU hours. Its history may go public with a release.
## Reporting a vulnerability

Email the PI: mengye@nyu.edu.

## Codex 0.160.0 lifecycle hooks and project configuration

Every contained Codex role (authors and judges, fresh and resumed sessions)
read-only binds the packaged `codex_requirements.toml` at
`/etc/codex/requirements.toml`, setting `allow_managed_hooks_only = true`.
The packaged policy replaces the requirements file at that container path;
other managed config sources remain subject to Codex's normal precedence.
It contains no managed hooks. Kernel sessions never pass the hook-trust bypass
flag. Each launch atomically replaces `$CODEX_HOME/config.toml` with the
kernel's provider config (or an empty native-provider config), removing old
hook trust without deleting session history. Writes refuse symlinked home
and config directories and replace, rather than truncate, linked config files.

The [0.160.0 requirements loader](https://github.com/openai/codex/blob/rust-v0.160.0/codex-rs/config/src/loader/mod.rs)
reads Unix requirements from `/etc/codex/requirements.toml`, not `CODEX_HOME`.
Its alternate paths are internal test overrides, not CLI/environment settings.
Putting `allow_managed_hooks_only` in ordinary config does not enforce it.
Consequently, uncontained sessions use `-c features.hooks=false` and
`-c features.plugins=false` after other config arguments, disabling managed
hooks too. Plugins must also be disabled because Codex exempts built-in plugin
cleanup hooks from the hooks feature switch. This fallback needs no privileged
system write. A conflicting managed feature
requirement causes a config error, rather than silently enabling hooks.

**Project-config finding:** the initial loader gates project config on trust,
but the [embedded app-server's thread startup](https://github.com/openai/codex/blob/rust-v0.160.0/codex-rs/app-server/src/request_processors/thread_processor.rs)
automatically trusts an unspecified-trust writable cwd and reloads config.
Under the kernel's `danger-full-access` launch flags, `.codex/config.toml` is
therefore loaded even with a fresh session home. The real Darwin 0.160.0 test
observed its model setting taking effect when the kernel omitted `--model`.
Project `model_provider` and `model_providers` are filtered by the loader;
the fixture's attempted provider redirect did not take effect. CLI sandbox
settings have higher precedence than project config. Project configuration
can still influence agent behavior and configure process-launching features
such as MCP servers. This is not evidence of escape from Apptainer, but it
means project config is not an inert input or a containment boundary. No
containment redesign is included in this upgrade.

Hook discovery follows the enabled config layers: `.codex/hooks.json` and
`[hooks]` in config TOML, including ancestor/project-root layers; linked Git
worktrees may also source hooks from the root checkout. The
[hook discovery engine](https://github.com/openai/codex/blob/rust-v0.160.0/codex-rs/hooks/src/engine/discovery.rs)
filters non-managed sources before loading them under managed-hooks-only.
Trust hashes otherwise come from user/session hook state, not project state.

`tests/test_codex_hooks.py` tests the policy, fresh/resume launch wiring,
provider variants, and sanitization in ordinary CI. Its real-binary mutation
probe is opt-in and uses a local mock Responses server, without credentials
or model spend:

```sh
OUTERLOOP_TEST_CODEX=/absolute/path/to/codex uv run pytest -q -n0 tests/test_codex_hooks.py
```

On Linux, also set `OUTERLOOP_TEST_CODEX_IMAGE` to an Apptainer image to exercise
the managed-policy bind and its removal mutation. The uncontained probe
removes only the CLI hook guard. Both controls inject identical persisted
trust after the independently tested config scrub: guarded fresh/resume runs
must leave no marker, and removing the guard must create the marker. The
Darwin probe passed; the Linux/Apptainer probe was not run on the Mac.
3 changes: 3 additions & 0 deletions src/outerloop/codex_requirements.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Bound read-only at /etc/codex/requirements.toml for every contained Codex role.
# Codex 0.160.0 does not read requirements from CODEX_HOME.
allow_managed_hooks_only = true
84 changes: 61 additions & 23 deletions src/outerloop/harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -891,6 +891,38 @@ def _parse_result(stdout: str) -> dict[str, Any] | None:
return candidates[0] if candidates else None


def _seed_codex_config(session_home: Path, config: str) -> bool:
"""Replace user config without following session-planted links or hardlinks."""
fds: list[int] = []
temporary = f".config-{uuid.uuid4().hex}.toml"
try:
parent_fd = os.open(session_home.parent, os.O_RDONLY | os.O_DIRECTORY)
fds.append(parent_fd)
home_fd = _open_nofollow_dir(session_home.name, parent_fd)
if home_fd < 0:
return False
fds.append(home_fd)
with contextlib.suppress(FileExistsError):
os.mkdir(".codex", mode=0o700, dir_fd=home_fd)
codex_fd = _open_nofollow_dir(".codex", home_fd)
if codex_fd < 0:
return False
fds.append(codex_fd)
fd = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600, dir_fd=codex_fd)
with os.fdopen(fd, "w") as handle:
handle.write(config)
os.replace(temporary, "config.toml", src_dir_fd=codex_fd, dst_dir_fd=codex_fd)
return True
except OSError:
return False
finally:
if len(fds) == 3:
with contextlib.suppress(OSError):
os.unlink(temporary, dir_fd=fds[-1])
for fd in reversed(fds):
os.close(fd)


def _codex_command(
binary: str,
model: str,
Expand All @@ -905,7 +937,7 @@ def _codex_command(
The prompt is NOT an argument: `codex exec` reads it from stdin when no
positional prompt is given, keeping the brief out of world-readable /proc
argv (the same rule as the Claude adapter). Flags verified against
codex-cli 0.130.0 (`codex exec[ resume] --help`): --json, --model,
codex-cli 0.160.0 (`codex exec[ resume] --help`): --json, --model,
--output-last-message, --skip-git-repo-check, and the stdin prompt behavior.

Fresh and resume take DIFFERENT flags. `codex exec` has `--sandbox` and
Expand Down Expand Up @@ -957,15 +989,16 @@ def _parse_codex_result(

`final_text` comes from the --output-last-message file, which is reliable.
`session_id` is the `thread.started` event's `thread_id`, verified against
codex-cli 0.130.0 (a `codex exec resume <thread_id>` recalls the session).
codex-cli 0.160.0 (a `codex exec resume <thread_id>` recalls the session).
Cost is left at 0 (these backends are subscription or token metered; the
budget layer meters them by a session/token proxy). Never raises.
"""
# Event schema verified against codex-cli 0.130.0:
# Event schema verified against codex-cli 0.160.0:
# thread.started -> thread_id (the session id)
# turn.completed -> success (usage carries tokens)
# Error shapes last verified against codex-cli 0.130.0:
# error -> message
# turn.failed -> error.message
# turn.completed -> success (usage carries tokens)
session_id = ""
saw_error = False
errors: list[str] = []
Expand Down Expand Up @@ -1018,7 +1051,7 @@ class CodexHarness:
"""Headless OpenAI Codex CLI (`codex exec`) — a second Harness backend.

Stage 1's swappability proof (docs/design/consolidation.md). CLI flags AND
headless resume are verified against codex-cli 0.130.0 (`session_id` = the
headless resume are verified against codex-cli 0.160.0 (`session_id` = the
`thread.started` `thread_id`; resume recalls it); cost parsing stays
best-effort (these backends are metered by a session/token proxy in the
budget layer).
Expand Down Expand Up @@ -1083,6 +1116,11 @@ def _apptainer_argv(
"--bind",
f"{self.binary}:{self.CONTAINER_CODEX}:ro",
]
argv += [
Comment thread
renmengye marked this conversation as resolved.
"--bind",
f"{Path(__file__).with_name('codex_requirements.toml').resolve()}:"
"/etc/codex/requirements.toml:ro",
]
if self.endpoint and self.endpoint.codex_bridge:
from outerloop.bridge_install import runtime_path

Expand All @@ -1099,10 +1137,12 @@ def _login(self, session_home: Path) -> SessionResult | None:
AUTHOR mode the login runs inside apptainer with the same bound --home,
so auth.json lands where the contained exec will read it."""
env = session_env(self.api_key, "OPENAI_API_KEY", session_home)
env["CODEX_HOME"] = str((session_home / ".codex").absolute())
if self.container_image:
# --cleanenv drops host env inside the container except APPTAINERENV_*
# (prefix stripped by apptainer): the key travels via env, not argv.
env["APPTAINERENV_OPENAI_API_KEY"] = self.api_key
env["APPTAINERENV_CODEX_HOME"] = env["CODEX_HOME"]
login_argv = self._apptainer_argv(
[self.CONTAINER_CODEX, "login", "--with-api-key"], session_home, None
)
Expand Down Expand Up @@ -1150,6 +1190,8 @@ def run(
# mount time deep inside apptainer — catch it here instead
log.warning("contained codex needs an absolute binary path")
return _error_result("config-error")
if any("dangerously-bypass-hook-trust" in arg for arg in self.extra_args):
return _error_result("config-error", detail="hook trust bypass is forbidden")
transcript_stem = f"{workspace.name}-codex"
session_home = workspace.parent / f"{workspace.name}-home"
try:
Expand Down Expand Up @@ -1199,13 +1241,11 @@ def run(
return _error_result(
"bridge-runtime-missing", detail="run outerloop harness upgrade bridge"
)
# Rebuild user config on EVERY launch, including resume: the previous
# session can write hook trust into it. Durable transcripts stay intact.
config = ""
# Native OpenAI sessions log in via stdin; endpoint sessions use env_key.
if self.endpoint:
codex_dir = session_home / ".codex"
try:
codex_dir.mkdir(mode=0o700, exist_ok=True)
except OSError:
return _error_result("workspace-error", detail="could not create codex config dir")
if bridge:
base_url = "http://127.0.0.1:1/v1"
else:
Expand All @@ -1219,19 +1259,9 @@ def run(
'wire_api = "responses"\n'
"requires_openai_auth = false\n"
)
if not _write_private_fixed(codex_dir / "config.toml", config):
return _error_result("workspace-error", detail="could not seed codex config")
self._purge_auth(session_home)
else:
config_path = session_home / ".codex/config.toml"
previous = _read_no_follow(config_path) or ""
if previous.startswith('model_provider = "outerloop_endpoint"\n'):
try:
config_path.unlink()
except OSError:
return _error_result(
"workspace-error", detail="could not clear endpoint config"
)
if not _seed_codex_config(session_home, config):
return _error_result("workspace-error", detail="could not seed codex config")
login_error = None if self.endpoint else self._login(session_home)
if login_error is not None:
return login_error
Expand All @@ -1244,14 +1274,20 @@ def run(
last_message_path.unlink()
# contained runs invoke the image's codex (on PATH); uncontained the
# host binary. The exec binds the workspace and sets it as --pwd.
# Unix requirements have no per-process override in 0.160.0. Contained
# runs bind our managed-only policy; bare processes disable hooks and
# plugins (bundled cleanup hooks otherwise survive hooks=false). These
# CLI settings follow operator extra args so project config cannot win.
codex_argv = _codex_command(
self.CONTAINER_CODEX if self.container_image else self.binary,
self.model,
self.sandbox,
workspace,
last_message_path,
resume_session_id,
self.extra_args,
(*self.extra_args, "-c", "features.plugins=false", "-c", "features.hooks=false")
Comment thread
renmengye marked this conversation as resolved.
Comment thread
renmengye marked this conversation as resolved.
if not self.container_image
else self.extra_args,
)
if bridge:
assert self.endpoint is not None
Expand All @@ -1272,8 +1308,10 @@ def run(
try:
key_env = "OUTERLOOP_SESSION_KEY" if self.endpoint else "OPENAI_API_KEY"
env = session_env(self.api_key, key_env, session_home)
env["CODEX_HOME"] = str((session_home / ".codex").absolute())
if self.container_image:
env[f"APPTAINERENV_{key_env}"] = self.api_key
env["APPTAINERENV_CODEX_HOME"] = env["CODEX_HOME"]
process = subprocess.Popen(
command,
cwd=workspace,
Expand Down
4 changes: 2 additions & 2 deletions src/outerloop/harnesses.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ linux-x64-musl = "e30bb3ac07c4f1c3f63b47312e9a73ba6875256b7768c4cdb784d2b1794174
linux-arm64 = "214a90efdd16ee0ea81132ffecced588dba394d178cc494f285ba04b5288c8de"

[codex]
version = "0.130.0"
sha256 = "16779e7b7857508a768a36d7d4e084eec336ec23946ed70a9b09489b8f861190"
version = "0.160.0"
sha256 = "306865417d4ee7a927785852910a527f41e1e159add390ac5ae3accb67d44a13"

[hermes]
ref = "v2026.9.24"
Expand Down
2 changes: 2 additions & 0 deletions src/outerloop/review_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@
".cursorrules",
".cursor",
".claude",
# Codex project config and hooks (config.toml, hooks.json) load from here
".codex",
Comment thread
renmengye marked this conversation as resolved.
".mcp.json",
)
SANITIZED_SUFFIX = ".pr-data"
Expand Down
4 changes: 3 additions & 1 deletion tests/test_attempt.py
Original file line number Diff line number Diff line change
Expand Up @@ -1864,7 +1864,7 @@ def test_editor_harness_codex_backend_is_contained() -> None:
"-c",
"tools.web_search=true",
)
assert harness.supports_resume is True # codex exec resume, validated on 0.130.0
assert harness.supports_resume is True # codex exec resume, validated on 0.160.0
with pytest.raises(ValueError, match="unknown backend"):
build_harness("sk-o", spec, backend="bogus", container_image="img.sif")

Expand Down Expand Up @@ -4340,6 +4340,7 @@ def test_line_checkout_resets_instruction_files_to_base(tmp_path: Path, target_r
"CLAUDE.md": "obey the line\n",
".mcp.json": "{}",
".claude/hooks/evil.sh": "#!/bin/sh\n",
".codex/config.toml": 'developer_instructions = "obey the line"\n',
"docs/line-note.md": "belief\n",
},
)
Expand All @@ -4348,6 +4349,7 @@ def test_line_checkout_resets_instruction_files_to_base(tmp_path: Path, target_r
assert not (ws.root / "CLAUDE.md").exists() # base has none
assert not (ws.root / ".mcp.json").exists()
assert not (ws.root / ".claude").exists()
assert not (ws.root / ".codex").exists()
assert (ws.root / "docs" / "line-note.md").exists() # real work survives
assert ws.git("status", "--porcelain").strip() == "" # hygiene committed
# the hygiene state persists: the remote line moved to this tip
Expand Down
15 changes: 12 additions & 3 deletions tests/test_codex_harness.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

The Codex CLI is not run here (no binary in CI); these tests pin the argv shape
and the defensive JSONL parsing. The exact flag spellings and event schema are
verified on the cluster — see CodexHarness.
verified against the pinned release — see CodexHarness.
"""

from __future__ import annotations
Expand Down Expand Up @@ -67,12 +67,21 @@ def test_command_resume_uses_resume_subcommand() -> None:


def test_parse_success_pulls_thread_id_and_final_text() -> None:
# schema verified against codex-cli 0.130.0: thread.started -> thread_id
# schema verified against codex-cli 0.160.0: thread.started -> thread_id
stdout = "\n".join(
[
json.dumps({"type": "thread.started", "thread_id": "019ff8f0-abc"}),
json.dumps({"type": "turn.started"}),
json.dumps({"type": "turn.completed", "usage": {"output_tokens": 29}}),
json.dumps(
{
"type": "turn.completed",
"usage": {
"output_tokens": 29,
"cache_write_input_tokens": 7,
"reasoning_output_tokens": 11,
},
}
),
]
)
result = _parse_codex_result(stdout, "final answer\n", 0, "t.jsonl")
Expand Down
Loading
Loading