test(cu): prove no-focus process restart isolation - #903
Conversation
0734f14 to
ad69fb2
Compare
|
@astrohan Updated this PR from a single restart proof to a five-round concurrent-user soak. The same backend/Runtime survived five real app restarts; all old observations failed closed, four fresh background actions succeeded, one real occlusion failed closed, service generations stayed stable, and the fixture never became frontmost. |
|
@astrohan P0 blocker found in live concurrent use: the current compatibility driver PID-bound CGEvent path does not move the pointer or steal focus, but it can still interfere with the user physical mouse button state. Do not merge the Computer Use stack as fully background-safe yet. I am preparing a fail-closed follow-up that disables every CGEvent mouse/keyboard path and retains only AX/CDP/screenshot operations until the native event backend can prove isolation. |
ad69fb2 to
5efd222
Compare
|
@astrohan Updated the PR body to remove the obsolete pixel-safety claim. Current evidence is the #905 AX-only 5-round restart soak: target_missing 5/5, fresh AX readback 5/5, zero compatibility dispatch, normal user mouse/keyboard input. The visible fixture launch remains a documented non-zero-disturbance boundary. |
Astro-Han
left a comment
There was a problem hiding this comment.
Approved on the current test-only change. I found no P0 or P1. This head must not be used to claim concurrent-user safety without #905, and it still needs to follow #898 before merge.
P2
- The focus monitor may exit silently. The launcher treats a zero exit or
SIGTERMas non-failure even when it did not initiate the stop (scripts/cu-process-restart-e2e-launcher.mjs:258-265). If the monitor dies during the five rounds, the no-focus oracle disappears and the test can still pass. - Every fresh action may return
target_occludedand the run still passes (scripts/cu-process-restart-e2e.mjs:334-350). That proves fail-closed behavior but not that the new process action path works. Require at least one successful fresh dispatch or mark an all-occluded run inconclusive. - Despite the updated PR description, this head still performs coordinate
left_clickthrough the compatibility path (scripts/cu-process-restart-e2e.mjs:319-349). That path is known to interfere with physical mouse-button state. It is an explicit opt-in real-machine test, so this is P2 rather than a production P1; #905's AX-only replacement should land before this safety claim is used. SIGINT,SIGTERM, andSIGHUPbypass the async cleanup, andcaffeinateis not tied to the parent with-w(scripts/cu-process-restart-e2e-launcher.mjs:282-284, 390-400). Interrupted runs can leave fixture, monitor, harness, or sleep-prevention processes alive.- Fixture commands and cleanup have no timeout (
scripts/cu-process-restart-e2e-launcher.mjs:19-35, 393). A stuckstop.shcan prevent focus restoration, process cleanup, and temporary-directory removal indefinitely. - The launcher and harness execute a fixture from the hard-coded, unauthenticated
/Users/haoqing/...checkout (scripts/cu-process-restart-e2e-launcher.mjs:11, 34-35;scripts/cu-process-restart-e2e.mjs:9-11). Other maintainers cannot reproduce it, and the executed scripts can drift independently of this repository.
P3
- Start-to-end pointer displacement is not evidence that a user supplied physical input during the run (
scripts/cu-process-restart-e2e-launcher.mjs:379-389). It records net position only and cannot identify the event source. - The evidence arrays omit the PID produced by the fifth restart because PIDs are recorded only at the start of each round (
scripts/cu-process-restart-e2e.mjs:211-235, 398-411). Per-case data still contains it, but the aggregate report is incomplete. - The focus monitor starts after initial stop/reset and stops before final cleanup (
scripts/cu-process-restart-e2e-launcher.mjs:287-290, 391-393). A brief fixture activation during setup or cleanup is outside the no-focus claim. - The harness derives the checkout path through URL
.pathnamerather thanfileURLToPath()(scripts/cu-process-restart-e2e.mjs:7), so paths containing spaces or escaped characters fail.
CI is green, and the new contract, syntax, Swift typecheck, and diff checks passed. These findings do not require another fix round, but the branch must be restacked and #905 must complete the safety boundary.
|
@Astro-Han Current disposition: do not merge this old cumulative restart branch as-is. #895/#896 are now on main, and #910 adds a privacy-safe native semantic dispatch trace that fixes the AX soak observability gap. The remaining useful restart proof should be rebuilt on current main with a repository-owned or hash-pinned fixture, bounded cleanup, and an inconclusive result when no fresh action succeeds. I will keep this PR open as evidence until that reduced replacement is ready. |
5efd222 to
ce207d5
Compare
Astro-Han
left a comment
There was a problem hiding this comment.
Restack-only rereview on ce207d58. Its patch-id exactly matches the reviewed process-restart harness commit. Typecheck, test, and e2e are green. No P0 or P1.
P2
- The focus monitor can exit silently and remove the no-focus oracle.
- Five rounds may all return
target_occludedwithout proving a fresh action path works. - This layer still uses the known-interfering compatibility coordinate click; #905 supplies the AX-only replacement.
- Process signals can bypass cleanup and leave child or
caffeinateprocesses alive. - Fixture commands and cleanup have no timeout.
- The fixture is an author-specific, unauthenticated external checkout.
P3
- Net pointer displacement is not reliable evidence of physical user input.
- The aggregate report omits the PID produced by the fifth restart.
- Focus monitoring does not cover the full setup and cleanup lifecycle.
- URL
.pathnamebreaks checkout paths containing spaces.
All remain non-blocking test-harness findings. Approved for squash merge; #905 must follow before concurrent-user safety is claimed.
Stack position
Follow-up after #898. This PR adds the repeated real-process restart harness. Its original pixel-dispatch safety conclusion is superseded by P0 follow-up #905, which must be included before the stack is treated as concurrent-user safe.
Goal
Match the recovered Codex process-lifetime boundary:
An observation from a terminated app process must never authorize an action after the same
.apppath launches a new process, even across repeated restarts and concurrent user activity.Current five-round scenario
One cua-driver backend and one Runtime tool instance stay alive across five real target-app restarts.
Every round performs:
.apppath without activation;target_missing, no dispatch, zero mutation;set_valueand require exact fresh AX readback;restartAttempts == 0.The current harness does not send compatibility click, scroll, drag, key events, or synthetic HID pulses.
Concurrent-user behavior
The fixture is a normal layer-0 on-screen window moved to the active Space without calling
NSApp.activate. A continuous Swift sentinel:The visible AppKit fixture can still be noticeable during launch or window ordering. This is not a zero-disturbance WindowServer claim.
Current real result
The July 14, 2026 AX-only run passed all five rounds while the user continued using the Mac:
target_missing: 5/5;set_valuewith exact fresh-observation readback: 5/5;1, capture generation stayed0, restartAttempts stayed zero;The earlier pixel soak remains historical process-restart evidence only. Live use later proved PID-bound CGEvent can interfere with physical mouse-button state, so #905 disables that compatibility input path by default.
Verification
Passed:
@maka/computer-use119/119;npm run e2e:computer-use-process-restarton macOS, AX-only, five rounds.Remaining boundary
This proves repeated ordinary process-restart isolation. It does not force macOS to reuse an old numeric PID. Deliberate PID-reuse safety still requires a native atomic process-instance identity or a deterministic driver/host fixture.