feat(cu): add runtime observation and action leases - #892
Merged
Conversation
hqhq1025
marked this pull request as ready for review
July 13, 2026 16:29
Contributor
Author
|
@astrohan This Computer Use stack slice is ready for review. Please follow the dependency and merge order stated at the top of the PR; the exact net diff is linked there. |
This was referenced Jul 13, 2026
hqhq1025
force-pushed
the
codex/cu-runtime-foundation
branch
from
July 13, 2026 17:41
025cb11 to
388c01d
Compare
hqhq1025
force-pushed
the
codex/cu-runtime-foundation
branch
from
July 13, 2026 18:03
388c01d to
7c99c35
Compare
Astro-Han
approved these changes
Jul 13, 2026
Astro-Han
left a comment
Contributor
There was a problem hiding this comment.
Approved. I found no P0 or P1. These P2s are worth tracking, but none should hold up this PR.
P2
clearSession()only marks an existing session-state record as stopped. A first invocation queued beforesessionState()runs can survive a direct clear and later reactivate the session (computer-use-tools.ts:857, 1245). Desktop Stop aborts the runtime signal in the same task, so this is not a P1 in the shipped path, but the public cleanup contract is incomplete.screenshot,list_apps,cursor_position, andwaittake neither an observation lease nor an action lease (computer-use-tools.ts:859-890, 898-1010). A caller that usesclearSession()without aborting the signal can still read host state afteruser_stopped.- Terminal states are not absorbing.
screenLocked(),reobserveRequired(), and similar events can overwriteuser_stoppedorblocked_url, and a later unlock can reopen observation (cua-session-state.ts:84-113). - Controlled Computer Use failures return
{ text, error }, then get normalized to ordinary text and recorded as successful tool invocations (tool-runtime.ts:568-600). This also resets the identical-failure loop gate. - Backend exceptions can persist raw UI-derived text. A targeted screenshot calls
observeApp()without a Computer Use-specific exception projection (computer-use-tools.ts:981-985), while the generic runtime stores the thrown message after secret-pattern redaction only (tool-runtime.ts:643-710, 828-832). AX labels and window titles that are not secret-shaped can enter session history. - When a backend returns
observationand a separatescreenshot,freshFullObservation()registers the observation without merging the screenshot dimensions (computer-use-tools.ts:748-773). The next bound action can fail withtarget_missingbecause the new frame has no usable coordinate bounds. cursor_positiondropsresolvedScreenPoint. The action can succeed without returning the coordinates the model asked for.permissionArgs()parses the action-specific schema before the tool-call record and runtime error boundary exist (tool-runtime.ts:280-311). A wire-valid but action-invalid call, such as a click without coordinates, can escape without a persistedtool_callortool_result, telemetry, or loop-gate accounting.
Validation: the isolated Runtime suite passed with 1,462 tests and 7 platform skips. Typecheck and git diff --check also passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upstream stack notice
This is stack PR B. It depends on #857 and must not merge before #857.
Because GitHub cannot use a branch from the contributor fork as this PR base, the Files tab currently includes PR A. After #857 merges, this branch will be rebased onto
mainand the diff will shrink to the Runtime-owned slice.Review the exact 9-file net diff now in fork-local PR hqhq1025#1.
Current rebase verification: full repository build; Runtime 1452 passed with 7 platform skips; full repository typecheck.
Stack position
PR B, stacked on
codex/cu-foundation-core(upstream draft PR #857).This fork-local PR preserves the real stacked diff because the upstream repository cannot use a fork branch as a PR base. After #857 merges, this branch will be retargeted to
maka-agent/mainand opened upstream.Summary
Adds the provider-neutral Runtime layer without adding a native executor, Desktop wiring, provider adapter, or cursor presentation.
Codex lab alignment
maxInFlight: 1is implemented per Computer Use session, not globally;Non-goals
Verification