Skip to content

feat(cu): connect typed host lifecycle events - #897

Merged
Astro-Han merged 1 commit into
apache:mainfrom
hqhq1025:codex/cu-host-events-e2e
Jul 13, 2026
Merged

feat(cu): connect typed host lifecycle events#897
Astro-Han merged 1 commit into
apache:mainfrom
hqhq1025:codex/cu-host-events-e2e

Conversation

@hqhq1025

Copy link
Copy Markdown
Contributor

Upstream stack notice

This is stack PR G1. It depends on #896 and must not merge before it.

The Files tab is cumulative until preceding fork branches are rebased after merge.

Review the exact 6-file host-event net diff now in fork-local PR hqhq1025#6.

Current rebase verification: @maka/computer-use 113/113; Runtime 1466 passed with 7 platform skips; Desktop typecheck.


Goal

Connect only Computer Use lifecycle events with a typed, attributable source, and add a scoped cross-layer deterministic contract.

This is G1, not the complete host-event or real-Electron E2E layer.

What this PR does

  • preserves typed cua-driver lifecycle errors instead of collapsing them to capture_failed
  • maps typed dispatch outcomes into Runtime state:
    • user_intervened -> re-observe
    • screen_locked -> locked
    • blocked_url -> terminal URL block
    • outcome_unknown, service unavailable, or service mismatch -> re-observe
  • propagates cua-driver child/generation loss into Runtime reobserve_required
  • invalidates every session retaining observation or keyboard ownership when a shared service generation is lost
  • keeps explicit session_cleared local to that session
  • adds a backend-factory seam for hermetic host-event wiring tests
  • adds a cross-layer deterministic contract for:
    • bound target propagation
    • presentation -> dispatch -> fresh observation ordering
    • duplicate/stale rejection
    • typed failure cancellation
    • unknown-outcome re-observation
    • explicit session cleanup
    • persisted-tool-text privacy projection
  • documents connected producers and deliberate gaps

Latest reverse-engineering constraints applied

Reviewed the current local reverse-engineering lab, including V10 and chapters 26/27.

Key consequences:

  • Codex native target lifetime is canonical app path plus the current live process instance.
  • Maka does not claim this stronger identity until cua-driver or a native host API exposes an atomic, high-precision process identity.
  • A provisional ps/formatted-start-time implementation was rejected during review rather than shipped as false protection.
  • Codex turn-ended is a signed-helper Apple Event lifecycle. Maka does not invoke or emulate it; helper exit is not proof of service cleanup.
  • AX/DOM content changes are not treated as physical user-input evidence.
  • The current driver provides no trustworthy intervention debounce deadline, so typed intervention advances directly to re-observe instead of entering an unrecoverable debounce state.

Non-goals

  • global physical-input producer
  • intervention debounce timer producer
  • screen-unlock producer
  • Codex turn-ended helper integration
  • canonical app path/current-process-instance production binding
  • real target/decoy Electron execution
  • real process-restart E2E
  • persistence-level privacy audit
  • real macOS/real-window cumulative E2E
  • provider/model-loop matrix

Verification

  • npm --workspace @maka/runtime test — 1434 passed, 2 skipped
  • focused Runtime/Frame producer tests — passed
  • npm --workspace @maka/computer-use test — 109 passed
  • cross-layer deterministic contract — 3 passed
  • npm --workspace @maka/desktop run typecheck
  • git diff --check

Follow-up stack

  1. native/driver-level canonical app path + live process-instance identity
  2. deterministic target/decoy cua-driver harness
  3. Desktop terminal lifecycle + persistence privacy verification
  4. reliable macOS physical-input/unlock producers
  5. real Electron and real macOS cumulative gates

Stacked on #5.

@hqhq1025
hqhq1025 marked this pull request as ready for review July 13, 2026 16:30
@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan This Computer Use stack slice is ready for review. Please follow the dependency and merge order stated at the top of the PR; the exact net diff is linked there.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan P0 blocker found in live concurrent use: the current compatibility driver PID-bound CGEvent path does not move the pointer or steal focus, but it can still interfere with the user physical mouse button state. Do not merge the Computer Use stack as fully background-safe yet. I am preparing a fail-closed follow-up that disables every CGEvent mouse/keyboard path and retains only AX/CDP/screenshot operations until the native event backend can prove isolation.

@hqhq1025
hqhq1025 force-pushed the codex/cu-host-events-e2e branch from 7780f44 to c9b6197 Compare July 13, 2026 18:04
@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan P0 follow-up #905 now contains the fail-closed fix for the live physical-click interference. Coordinate CGEvent click/scroll/drag/key paths are disabled by default; please include #905 before treating the stack as concurrent-user safe.

@hqhq1025
hqhq1025 force-pushed the codex/cu-host-events-e2e branch from c9b6197 to 1fd7f8e Compare July 13, 2026 18:18

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on content. I found no P0 or P1. These two state-mapping issues are P2 and do not require another fix round.

P2

  1. applyTypedOutcomeState() changes outcome_unknown to reobserve_required before the post-dispatch lease check (packages/runtime/src/computer-use-tools.ts:636-658, 1332-1336, 1445-1450). That lease failure replaces the backend's "the action may have executed; do not retry" result with a generic re-observe result. Preserve outcome_unknown in the model-facing return while still invalidating the frame.
  2. The same late outcome mapping can overwrite a terminal session state. If user_stopped, screen_locked, or blocked_url is set while an action is in flight and the backend then returns a typed failure, applyTypedOutcomeState() writes a new state such as reobserve_required (packages/runtime/src/computer-use-tools.ts:636-658, 1332, 1445). Terminal states should be absorbing for that generation. Desktop's synchronous abort chain makes this a race rather than a normal path, so it remains P2.

The current branch still needs to follow its parent stack before squash merge. The focused lifecycle and cumulative tests passed on the reviewed patch.

@Astro-Han
Astro-Han force-pushed the codex/cu-host-events-e2e branch from 1fd7f8e to e91cdbb Compare July 13, 2026 18:59

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restack-only rereview on e91cdbbf. Its patch-id exactly matches the reviewed host-lifecycle change. Typecheck, test, and e2e are green. No P0 or P1.

P2

  1. outcome_unknown is replaced by generic reobserve_required during the post-dispatch lease check, losing the model-facing "may have executed; do not retry" result.
  2. A late typed outcome can overwrite an absorbing user_stopped, screen_locked, or blocked_url state for the same generation.

Both remain non-blocking. Approved for squash merge with no additional fix round.

@Astro-Han
Astro-Han merged commit c34b8d6 into apache:main Jul 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants