Skip to content

feat(cu): wire verified backend into desktop - #895

Merged
Astro-Han merged 2 commits into
apache:mainfrom
hqhq1025:codex/cu-desktop-wiring
Jul 13, 2026
Merged

feat(cu): wire verified backend into desktop#895
Astro-Han merged 2 commits into
apache:mainfrom
hqhq1025:codex/cu-desktop-wiring

Conversation

@hqhq1025

Copy link
Copy Markdown
Contributor

Upstream stack notice

This is stack PR E. It depends on #894 and must not merge before it.

The Files tab is cumulative until preceding fork branches are rebased after merge.

Review the exact 9-file Desktop wiring net diff now in fork-local PR hqhq1025#4.

Current rebase verification: full repository build; Desktop 2416/2416; Desktop typecheck.


Goal

Wire the verified @maka/computer-use backend into the Desktop host without pulling presentation, provider-specific loops, or cumulative E2E into the same review unit.

What this PR does

  • resolves the development and packaged cua-driver artifact locations
  • verifies a regular, non-symlink executable against the pinned SHA-256 before backend selection
  • requires distributionReady: true in packaged mode while still allowing the verified development artifact
  • registers Computer Use tools only when backend selection succeeds
  • projects live action/capture service health into Capability Center
  • reports scoped target/action-category approval instead of per-action approval
  • clears Runtime and executor ownership on stop, archive, remove, terminal turn events, stream errors, and app shutdown
  • keeps shutdown cleanup isolated through Promise.allSettled
  • builds @maka/computer-use before Desktop tests, build, screenshots, and smoke commands

Reverse-engineering constraints applied

The implementation follows the local Codex Computer Use lab findings:

  • approval is a scoped app capability, not evidence that a particular action is valid
  • action requires a fresh observation after approval
  • user stop, intervention, stale state, and service recovery invalidate prior execution ownership
  • service availability is derived from live role state, not binary presence alone
  • packaged enablement fails closed while distribution provenance/signing/notarization are incomplete

Relevant lab chapters: wrapper policy/TOCTOU, policy error state machine, service lifecycle, and Electron presentation contract.

Non-goals

  • cursor overlay or presentation lifecycle
  • provider-specific Computer Use adapters/model loops
  • physical intervention, lock/unlock, and blocked-URL host event producers
  • deterministic cumulative Electron E2E
  • release packaging, Developer ID signing, notarization, SBOM, or artifact attestation
  • Headless support

Verification

  • npm --workspace @maka/runtime test — 1420 passed, 2 skipped
  • npm --workspace @maka/computer-use test — 97 passed
  • npm --workspace @maka/desktop run typecheck
  • npm --workspace @maka/desktop test — 2392 passed
  • git diff --check

Remaining release blocker

Hash verification still precedes path-based process spawn. Packaged release remains disabled until the artifact is distribution-ready; the release chain must additionally bind the verified file identity to the executed canonical artifact and complete signing/notarization/attestation.

Stacked on #3.

@hqhq1025
hqhq1025 marked this pull request as ready for review July 13, 2026 16:30
@hqhq1025

Copy link
Copy Markdown
Contributor Author

@astrohan This Computer Use stack slice is ready for review. Please follow the dependency and merge order stated at the top of the PR; the exact net diff is linked there.

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. I found no P0 or P1 on the current head. The physical-input guard is wired through Desktop, the host, and the backend selector, and its error path fails closed. These remaining findings are non-blocking.

P2

  1. Computer Use is registered without checking model vision support (apps/desktop/src/main/main.ts:569-589). The app already computes supportsVision later, but does not use it to filter this tool group. Since observe and screenshot return image-data, a non-vision model with function calling can load the tool and fail on the next model request. This needs an availability gate tied to the selected model.
  2. smoke:browser still builds only @maka/core before build:main (apps/desktop/package.json:31). In a clean checkout with packages/computer-use/dist absent, it fails with TS2307 for @maka/computer-use. Add the new workspace build to this smoke path.

P3

  1. If the backend was selected successfully but the action or capture service later becomes unavailable or disposed, the top-level capability reason still says no integrity-checked artifact was found (apps/desktop/src/main/capability-snapshot.ts:149-160). The runtime probe shows the actual service failure, so this is misleading diagnostics rather than a functional gate failure.

CI is green. The focused Computer Use tests passed, including 108 package tests and 9 Desktop tests. No fix round is required for these items; this PR should merge after #894.

@Astro-Han
Astro-Han force-pushed the codex/cu-desktop-wiring branch from 5c1aba3 to a9aeff4 Compare July 13, 2026 18:50

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Restack-only rereview on a9aeff4f. The aggregate patch-id exactly matches the reviewed Desktop wiring and physical-input guard commits. Typecheck, test, and e2e are green. No P0 or P1.

P2

  1. Computer Use can still be loaded for a non-vision model even though observation results contain image data.
  2. The clean smoke:browser path does not build @maka/computer-use before Desktop main and can fail with TS2307.

P3

  1. A later cua-driver service failure is described as an artifact-integrity failure in the top-level capability reason.

These findings remain non-blocking. Approved for squash merge with no additional fix round.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants