Skip to content

test(cu): prove no-focus process restart isolation - #903

Merged
Astro-Han merged 1 commit into
apache:mainfrom
hqhq1025:codex/cu-process-restart-e2e
Jul 13, 2026
Merged

test(cu): prove no-focus process restart isolation#903
Astro-Han merged 1 commit into
apache:mainfrom
hqhq1025:codex/cu-process-restart-e2e

Conversation

@hqhq1025

@hqhq1025 hqhq1025 commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Stack position

Follow-up after #898. This PR adds the repeated real-process restart harness. Its original pixel-dispatch safety conclusion is superseded by P0 follow-up #905, which must be included before the stack is treated as concurrent-user safe.

Goal

Match the recovered Codex process-lifetime boundary:

canonical application path + current live process instance

An observation from a terminated app process must never authorize an action after the same .app path launches a new process, even across repeated restarts and concurrent user activity.

Current five-round scenario

One cua-driver backend and one Runtime tool instance stay alive across five real target-app restarts.

Every round performs:

  1. observe and bind the current process/window;
  2. terminate the real synthetic app process;
  3. relaunch the same canonical .app path without activation;
  4. require a globally new host PID and WKWebView WebContent PID;
  5. run the old observation and require target_missing, no dispatch, zero mutation;
  6. clear session ownership and observe the new process;
  7. run native AX set_value and require exact fresh AX readback;
  8. require cua-driver action/capture generations unchanged and restartAttempts == 0.

The current harness does not send compatibility click, scroll, drag, key events, or synthetic HID pulses.

Concurrent-user behavior

The fixture is a normal layer-0 on-screen window moved to the active Space without calling NSApp.activate. A continuous Swift sentinel:

  • allows the user to switch among their own apps;
  • allows typing and physical pointer movement;
  • aborts on screen lock;
  • aborts if the synthetic fixture ever becomes frontmost.

The visible AppKit fixture can still be noticeable during launch or window ordering. This is not a zero-disturbance WindowServer claim.

Current real result

The July 14, 2026 AX-only run passed all five rounds while the user continued using the Mac:

  • old observations failed target_missing: 5/5;
  • stale native dispatch count: 0;
  • fresh AX set_value with exact fresh-observation readback: 5/5;
  • cua-driver action generation stayed 1, capture generation stayed 0, restartAttempts stayed zero;
  • the synthetic fixture never became frontmost;
  • the physical pointer moved 217.1 logical points;
  • the user reported normal mouse and keyboard input during the AX-only run.

The earlier pixel soak remains historical process-restart evidence only. Live use later proved PID-bound CGEvent can interfere with physical mouse-button state, so #905 disables that compatibility input path by default.

Verification

Passed:

Remaining boundary

This proves repeated ordinary process-restart isolation. It does not force macOS to reuse an old numeric PID. Deliberate PID-reuse safety still requires a native atomic process-instance identity or a deterministic driver/host fixture.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan This is the next Computer Use truth gate after #898. It specifically fixes the earlier test-harness focus steal and proves real target-process restart isolation without activating the fixture.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan Updated this PR from a single restart proof to a five-round concurrent-user soak. The same backend/Runtime survived five real app restarts; all old observations failed closed, four fresh background actions succeeded, one real occlusion failed closed, service generations stayed stable, and the fixture never became frontmost.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan P0 blocker found in live concurrent use: the current compatibility driver PID-bound CGEvent path does not move the pointer or steal focus, but it can still interfere with the user physical mouse button state. Do not merge the Computer Use stack as fully background-safe yet. I am preparing a fail-closed follow-up that disables every CGEvent mouse/keyboard path and retains only AX/CDP/screenshot operations until the native event backend can prove isolation.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan P0 follow-up #905 now contains the fail-closed fix for the live physical-click interference. Coordinate CGEvent click/scroll/drag/key paths are disabled by default; please include #905 before treating the stack as concurrent-user safe.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan Updated the PR body to remove the obsolete pixel-safety claim. Current evidence is the #905 AX-only 5-round restart soak: target_missing 5/5, fresh AX readback 5/5, zero compatibility dispatch, normal user mouse/keyboard input. The visible fixture launch remains a documented non-zero-disturbance boundary.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved on the current test-only change. I found no P0 or P1. This head must not be used to claim concurrent-user safety without #905, and it still needs to follow #898 before merge.

P2

  1. The focus monitor may exit silently. The launcher treats a zero exit or SIGTERM as non-failure even when it did not initiate the stop (scripts/cu-process-restart-e2e-launcher.mjs:258-265). If the monitor dies during the five rounds, the no-focus oracle disappears and the test can still pass.
  2. Every fresh action may return target_occluded and the run still passes (scripts/cu-process-restart-e2e.mjs:334-350). That proves fail-closed behavior but not that the new process action path works. Require at least one successful fresh dispatch or mark an all-occluded run inconclusive.
  3. Despite the updated PR description, this head still performs coordinate left_click through the compatibility path (scripts/cu-process-restart-e2e.mjs:319-349). That path is known to interfere with physical mouse-button state. It is an explicit opt-in real-machine test, so this is P2 rather than a production P1; #905's AX-only replacement should land before this safety claim is used.
  4. SIGINT, SIGTERM, and SIGHUP bypass the async cleanup, and caffeinate is not tied to the parent with -w (scripts/cu-process-restart-e2e-launcher.mjs:282-284, 390-400). Interrupted runs can leave fixture, monitor, harness, or sleep-prevention processes alive.
  5. Fixture commands and cleanup have no timeout (scripts/cu-process-restart-e2e-launcher.mjs:19-35, 393). A stuck stop.sh can prevent focus restoration, process cleanup, and temporary-directory removal indefinitely.
  6. The launcher and harness execute a fixture from the hard-coded, unauthenticated /Users/haoqing/... checkout (scripts/cu-process-restart-e2e-launcher.mjs:11, 34-35; scripts/cu-process-restart-e2e.mjs:9-11). Other maintainers cannot reproduce it, and the executed scripts can drift independently of this repository.

P3

  1. Start-to-end pointer displacement is not evidence that a user supplied physical input during the run (scripts/cu-process-restart-e2e-launcher.mjs:379-389). It records net position only and cannot identify the event source.
  2. The evidence arrays omit the PID produced by the fifth restart because PIDs are recorded only at the start of each round (scripts/cu-process-restart-e2e.mjs:211-235, 398-411). Per-case data still contains it, but the aggregate report is incomplete.
  3. The focus monitor starts after initial stop/reset and stops before final cleanup (scripts/cu-process-restart-e2e-launcher.mjs:287-290, 391-393). A brief fixture activation during setup or cleanup is outside the no-focus claim.
  4. The harness derives the checkout path through URL .pathname rather than fileURLToPath() (scripts/cu-process-restart-e2e.mjs:7), so paths containing spaces or escaped characters fail.

CI is green, and the new contract, syntax, Swift typecheck, and diff checks passed. These findings do not require another fix round, but the branch must be restacked and #905 must complete the safety boundary.

@hqhq1025

Copy link
Copy Markdown
Contributor Author

@Astro-Han Current disposition: do not merge this old cumulative restart branch as-is. #895/#896 are now on main, and #910 adds a privacy-safe native semantic dispatch trace that fixes the AX soak observability gap. The remaining useful restart proof should be rebuilt on current main with a repository-owned or hash-pinned fixture, bounded cleanup, and an inconclusive result when no fresh action succeeds. I will keep this PR open as evidence until that reduced replacement is ready.

@Astro-Han
Astro-Han force-pushed the codex/cu-process-restart-e2e branch from 5efd222 to ce207d5 Compare July 13, 2026 19:09

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restack-only rereview on ce207d58. Its patch-id exactly matches the reviewed process-restart harness commit. Typecheck, test, and e2e are green. No P0 or P1.

P2

  1. The focus monitor can exit silently and remove the no-focus oracle.
  2. Five rounds may all return target_occluded without proving a fresh action path works.
  3. This layer still uses the known-interfering compatibility coordinate click; #905 supplies the AX-only replacement.
  4. Process signals can bypass cleanup and leave child or caffeinate processes alive.
  5. Fixture commands and cleanup have no timeout.
  6. The fixture is an author-specific, unauthenticated external checkout.

P3

  1. Net pointer displacement is not reliable evidence of physical user input.
  2. The aggregate report omits the PID produced by the fifth restart.
  3. Focus monitoring does not cover the full setup and cleanup lifecycle.
  4. URL .pathname breaks checkout paths containing spaces.

All remain non-blocking test-harness findings. Approved for squash merge; #905 must follow before concurrent-user safety is claimed.

@Astro-Han
Astro-Han merged commit db966e1 into apache:main Jul 13, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants