Move to the agent-tools core: kit session, verdict tool, no limbs - #74
Merged
Conversation
…ol call. codex.ts no longer runs the Codex SDK thread and parses its final message against an outputSchema. It opens a kit AppServerSession (model and effort as per-session flags, read-only sandbox, the kit's stall and turn timeouts, scrubbed env) with one tool, review_verdict, over the existing ReviewOutput schema. The kit validates the shape mid-turn and hands a bad call back to the model as the tool error; the tool's run throws for what the schema can't say (an anchor that is not a right-side line this diff touches, a convergence verdict carrying findings), so the model corrects itself instead of the runner demoting the finding after the fact. The model's text is never read. No verdict submitted is a fail, never a clean. Same two turns as before: the review prompt, then the hand-written second pass. github.ts loses the demotion path since anchors are checked at submission; the 422 body-only fallback stays. Drops @openai/codex-sdk and the failureReason regex; bumps @bevyl-ai/agent-tools 0.3.0 -> 0.13.0.
…status file. stupify is now the engine and nothing else: src/review-sweep.ts reads top to bottom as one sweep, src/sweep/* is the domain, and everything generic (codex session, exe.dev, host primitives) is @bevyl-ai/agent-tools. Gone, with their only readers: cli.ts and bin/ (provision, setup, upgrade, init, prime, status, review), prime.ts, status.ts (state/ status.json, read by `stupify status`), commit-status.ts (the GitHub App JWT minting behind one checkmark), review-one.ts (the on-demand review), the packs template, the release workflow, and the clack/picocolors deps. Rotation moves from review-pr.ts into the codex session's onTurnError hook, so the `raw` field on a limit outcome goes too. Deploy is two scripts and a runbook, earshot-style: deploy/vm-setup.sh (codex gateway config + bun, handed to `exe.dev new`) and deploy/push.sh (build, copy, first push writes config.env and installs the cron with GH_HOST on the cron line: gh reads it from the environment and a Bun process can't hand a runtime env write to a child spawn). DEPLOY.md is the runbook. README, AGENTS.md, ARCHITECTURE.md, and the landing page say the same thing. src: 2,967 lines -> 1,226.
stupify's own review of the first commit: review_verdict was live for the whole session, so a first-turn call could fill the verdict and be returned after runTurns without the ownership challenge ever running. The tool now refuses until the second-pass prompt is the running turn, with an error that tells the model when to call it.
stupify's second finding on this PR: the kit's runTurns passes no per-turn
sandbox override and its turn default is dangerFullAccess, so the
thread-level 'read-only' string owned nothing. turnSandboxPolicy
{ type: 'readOnly' } is what codex enforces per turn.
stupify's third finding: the kit's start() spawns codex and rethrows a failed handshake without stop(), and runTurns only enters its try/finally after start() returns. Under a minute cron that is an orphan per failed sweep. One stop() in our catch until the kit's start() owns it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per Noah: rip out the SDK codex runner and replace it with the earshot / agent-tools core, then kill the limbs.
Commit 1: the session and the verdict
codex.tsopens a kitAppServerSession:model/effortas per-session flags (they were read and never applied), the kit's 5-minute stall timeout instead of a 20-minute silent hang, scrubbed env.review_verdicttool overReviewOutput, earshot'sverdictshape. The kit validates the arguments mid-turn and hands zod's error back to the model; the tool throws for what a schema can't say (an anchor that isn't a right-side diff line, a convergence verdict carrying findings, a call before the second pass) so the model corrects itself. The model's text is never read; no call is afailthat retries, never a clean.@openai/codex-sdk.Commit 2: the limbs
Gone with their only readers:
cli.tsandbin/(provision, setup, upgrade, init, prime, status, review),prime.ts,status.tsandstate/status.json,commit-status.tsand the GitHub App minting,review-one.ts, the packs template, the release workflow, clack and picocolors. Rotation moves into the session'sonTurnErrorhook.src/review-sweep.tsis now one sweep, top to bottom. Deploy isdeploy/vm-setup.sh+deploy/push.sh+ DEPLOY.md, earshot-style. README, AGENTS.md, ARCHITECTURE.md, and the site match.GH_HOSTstays on the cron line rather than inconfig.env: under Bun a runtimeprocess.envwrite does not reach a child spawn unlessenvis passed, and the kit'sexecdoesn't pass it.Commits 3 to 5: stupify reviewed this PR and was right three times
The new engine went live on
stupify-octember-stupifyand reviewed each push of this branch. Each finding was real and is fixed in its own commit:review_verdictwas live for the whole session, so a first-turn call could return a verdict before the second pass ran. Now refused until the second-pass prompt is the running turn.threadSandbox: 'read-only'owned nothing: the kit's per-turn default isdangerFullAccess. NowturnSandboxPolicy: { type: 'readOnly' }, which is what codex enforces.start()rethrows a failed handshake without killing the codex it spawned, andrunTurnsonly enters its try/finally afterstart(). Onestop()in our catch until the kit owns it.On the fifth head it posted
nice, all fixed ✅and resolved all three threads.Kit follow-ups (agent-tools, not this PR)
exec()should passenv: process.envso a runtime env write reaches children.start()shouldstop()the child before rethrowing a failed handshake.DEFAULT_TURN_POLICYis full access; consumers that never setturnSandboxPolicyrun wide open.Verified
stupify-octember-stupify(backupreview-sweep.ts.bak-pre-kit-core): four real reviews of this PR, three findings, one convergence. No codex processes left behind between sweeps.deploy/push.sh <vm>. Bevyl's homeREVIEW-PROMPT.mdsays "Output pure JSON only" twice; reword to "submit withreview_verdict" when rolling out there.