feat(acp): usage, tool cards, idle wake, and monitor - #573
Closed
justrach wants to merge 16 commits into
Closed
Conversation
graff acp was already an adapter over the existing agent loop. Split the JSON-RPC table off the process command so we can fill PromptResponse.usage from the CostTally delta, emit usage_update, advertise agentInfo, accept string protocol versions, and acknowledge session/cancel — without a second engine (ADR 0012).
ACP hosts now see tool_call/tool_call_update and streamed agent_message_chunk during a turn (ADR 0013). Background jobs that finish between turns wake the next REPL popSteer or session/prompt. monitor watches a command as complete lines without stealing bash_output's unread cursor.
Eleven new tests (ACP tool cards, idle job wake, monitor) plus the existing slack gap; the ratchet was 40 behind the suite and would not notice a dropped module.
Graff's edit_file already is grok-build search_replace (unique span, replace_all, batched edits). Refuse old_string == new_string — a no-op splice used to report success. Drive grok's cases through exec.execTool. Empty old_string still does not create a file; hashline stays out.
No loop changes. Whole-file reads stay byte-exact (CRLF kept, no sparse numbers) so edit_file old_string matches. write_file does not mkdir -p. Grep, list_dir, and PDF extractors stay codedb/bash, not new names.
edit_file still splices exact bytes. On a miss, name the first line that holds the longest token from old_string (grok search_replace recovery). The webfetch fallback GET now prefers text/markdown so doc sites that serve markdown skip HTML. No new catalog names.
Foreground bash can take timeout_ms (10h cap). Root omit/0 stays unbounded so a long build is still Esc-only. Subagents still default to 120s. Background jobs ignore it. read_file contains with no hit is now is_error so the model retries instead of treating a miss as success.
tools_anthropic_sub is a substring check; 'subagents default' looked like the subagent tool in the child catalog.
The suite grew 29 tests on this branch (edit/read contracts, nearest match, bash timeout). Slack is 25; the floor was far enough behind to miss a dropped module.
First-class `vercel` provider on the coding-agent OpenAI surface. Active only when AI_GATEWAY_API_KEY (or `graff key set vercel`) is set; no key means the row stays inert. Live catalog from /coding-agent/v1/models.
The usage regex expected the old "(N cached) +" footer and dropped every token after cache writes landed. Parse both shapes, print calls/wall/ttft/in/cached/writes/out/hit% per task, and overlap sandboxes with --jobs N.
Scorecard now records Pi cacheWrite so graff-dev vs pi-codegraff hit rates are comparable. README ranks the suite-wall levers and pins a same-model compare (defaults are not fair).
Defaults, --model pin, usage columns, session/cache, and that only graff rows run schema-output (12/12 vs 11/11, not 11/12).
Grok reports ordinary input separately from cache_read_input_tokens, so hit% was cached/ordinary (261% on a warm exact-reply). Match graff: in = ordinary + read + write.
One-rep SuperGrok run, ReleaseFast graff. Shared 11 tasks: grok 36 calls / 108s / 84% cache; lean graff 40 / 142s / 39%; full 53 / 122s / 49%. pi/claude/dsh not present on this VM.
CI's embedder control used wire tool names, so folded monitor looked absent (ungated catalog is missing ['monitor']). Read the Folded native listing instead, and omit gated/lean-dropped names from that listing so --no-local-tools does not teach the model the tool exists.
Owner
Author
|
Closing as stale kitchen-sink draft vs tip. ACP mid-turn streaming, TUI-as-ACP-client, and usage already landed (ADR 0032 / 0041 / 0018). This Aug 19 adapter (usage + tool cards + monitor + Vercel + eval scorecard) does not apply to current ACP. Reopen a focused PR against 282 for any slice that is still missing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#375 asked for ACP as a thin adapter.
graff acpalready shipped in v0.0.236 — this PR is how to grow it without a second agent.How to do ACP in graff
ACP is not a new harness. The editor is the client; graff is the agent. JSON-RPC 2.0, one message per line on stdio. The load-bearing choice, same as
--json:Agentper processsession/promptappends to the sameroot.messagesrunTurnWithFallback(tools, MCP, subagents, same loop as-p)json_mode,root.out = null,g_out = null)That append-only history is why a long-lived ACP process stays warmer than
grok -p/-c(new process each turn). A second catalog, a second cache key, orchdironsession/newcwdwould look like “more ACP” and would bust the prefix or fight the workspace tool (ADR 0006).What this PR adds
Usage (ADR 0012)
v0 answered
{stopReason: "end_turn"}and nothing else. Clients could not see cache.PromptResponse.usage— per-turnCostTallydelta.inputTokensis the full prompt (ordinary + cache-write + cache-read), matching grok ACP and--jsoninput_tokens, notuncached_input_tokens.session/updateusage_update— sessionused/sizeand cumulative USD.initialize.agentInfo—{name, title, version}.protocolVersion("1","2025-11-25","0.1") accepted; we still advertise integer1.session/cancelas a request →{}. Notifications stay silent. An in-flight turn is not aborted (the read loop is blocked inside it).Tool cards, idle wake, monitor (ADR 0013)
The grok-build comparison’s next three slices, on this same adapter:
session/updateemitstool_call/tool_call_updatefrom engine tool events, and streamedagent_message_chunkfromtext_delta. The final text chunk is omitted when deltas already streamed. Root tools only.popSteeror the nextsession/promptwith a harness note. Does not interrupt a blocked readline.monitortool: same pump as backgroundbash, wakes on complete lines without stealingbash_output’s unread cursor (~40 lines/wake, ~200 lifetime then stop). Base + local; not in--lean.src/acp.zigstays the process command. Protocol isacp_protocol.zig; usage isacp_usage.zig; the ACPEngineSinkisacp_sink.zig; idle wake isjob_wake.zig.CI: folded monitor vs
--no-local-toolsscripts/test-no-local-tools.pyfailed on the control run (ungated catalog is missing ['monitor']) becausemonitoris a folded native — it is not a wire tool untilload_tool_schemas. The control catalog now includes the Folded native listing. The listing itself omits gated and--lean-dropped names so--no-local-toolsdoes not teach the model thatmonitorexists.edit_file is grok search_replace (ADR 0014)
Grok-build’s
search_replace/ OpenCodeeditis alreadyedit_file(path,old_string,new_string,replace_all, batchededits). No second catalog name.src/edit_grok_contract.zigruns grok’s cases throughexec.execTool.replace_all, miss mentionsread_file, consecutive edits, CRLF preserved as bytes, MultiEdit viaedits.old_string == new_stringis now refused (a no-op splice used to report success).old_string(src/edit_hint.zig). Splice stays byte-exact.old_stringstill does not create a file — that iswrite_file.hashline_editstays out (grok LINE:HASH anchors; numberedread_file+ exactedit_fileis the equivalent).Basic tools stay graff-shaped (ADR 0015)
No loop changes for
read_file,write_file,bash.src/basic_grok_contract.ziglocks the mapping.read_filestays byte-exact soedit_filecan copyold_string. No sparse numbers, no CR-strip, nooffset/limitaliases.write_filedoes not mkdir -p.grep/list_dir/ persistent bash cwd.Native-tool improvements (same names)
webfetchfallback GET now sendsAccept: text/markdown, …so doc sites that serve markdown skip HTML (kuri already converts; this is the harness client).bashoptionaltimeout_msis foreground-only (10h cap, same aswait_ms). Omit or0on the root stays unbounded (Esc still kills). Child bash calls still default to 120s (workflow: subagent bash commands hang indefinitely when codedb refuses cwd, causing ~48min timeout #93). Background jobs ignore it —bash_killowns them.read_filecontainsmiss is nowis_error.Vercel AI Gateway
vercelis a built-in provider (19th row). It only lights up whenAI_GATEWAY_API_KEYis set orgraff key set vercel …— no key, no seat. OpenAI chat on the coding-agent surface (https://ai-gateway.vercel.sh/coding-agent/v1/...) so traffic is marked as a coding agent. Default modelanthropic/claude-sonnet-4.6; live catalog from/coding-agent/v1/models. Not a flat-rate login (sub_loginstays false — credits / BYOK).Eval runner: scorecard,
--jobs, live grok-4.6graff-evals/run.pyprints per-task calls / wall / TTFT / in / cached / writes / out / hit%. Live SuperGrok one-rep (--jobs 2, ReleaseFast): lean graff 12/12 (43 calls, 150s, 37% cache); grok CLI 11/11 (36 calls, 108s, 84% cache); graff-full 12/12 (56 calls, 131s, 49% cache). Shared-task row is the fair compare (schema-outputis graff-only).Suite 1520 total / 1519 pass (baseline 1518).
Still out of scope (on purpose)
Client
fs/*andsession/request_permission,session/load, mid-turn cancel, interrupting a blocked readline,GRAFF_STABLE_CATALOGas a default, asearch_replacealias, hashline anchors,offset/limitaliases, built-in PDF extractors, unfoldingwebfetch, grok’s root-default 120s bash deadline, auto-background-on-timeout, a Vercel OIDC / device-login path (API key only),GRAFF_CACHE_PROJECT/ an eval daemon that reuses one process across tasks.