meta: Windows USB phantom-device detection and recoverable native-USB deployment

[zackees]

Goal

Make native-USB deployment fail safe and recoverable when Windows retains a stale/phantom COM devnode.

The concrete RP2040 failure is not missing CDC deployment support: fbuild can flash RP2040 UF2 firmware and can use a healthy Pico CDC endpoint. The bug is that Windows health discovered during enumeration is discarded before target selection, allowing a historical COM12 record to be returned as usable after it has become CM_PROB_PHANTOM.

This meta tracks the data-model fix, RP2040 state-machine fix, and an optional scoped UAC recovery path.

Confirmed evidence

Observed on a real RP2040:

• runtime USB: VID:PID 2E8A:000A;
• serial: 5303284720C4641C;
• historical runtime endpoint: COM12;
• BOOTSEL volume: RPI-RP2 on G:;
• fbuild successfully copied the UF2 and observed the BOOTSEL volume eject;
• the post-flash Windows devnode USB\VID_2E8A&PID_000A\5303284720C4641C was CM_PROB_PHANTOM;
• serial open timed out;
• the deployment path still returned COM12.

Related completed work already provides useful pieces but does not solve this:

• feat(rp2040): warn on host-detected failed USB devices and hub attachment #1087 warns about failed USB devices and hub attachment;
• fix(rp2040): wait for delayed Windows CDC handoff #1095/fix(rp2040): tolerate delayed Windows CDC re-enumeration after UF2 flash #1074 wait for delayed healthy CDC handoff;
• fbuild: absorb post-deploy serial wedge recovery into SerialMonitor; deprecate client pyserial #605 provides a post-deploy recovery seam;
• fbuild port scan misses all PJRC/Teensy (VID 16C0) serial ports + no vendor/product resolution for them #962 intentionally keeps unusual/phantom composite records visible diagnostically.

The missing invariant is: diagnostic visibility is not deploy eligibility.

Locked system invariants

1. Port enumeration preserves facts; it does not erase or silently filter health.
2. Phantom/problem records remain visible for diagnostics and fbuild port scan misses all PJRC/Teensy (VID 16C0) serial ports + no vendor/product resolution for them #962 compatibility.
3. A known-unhealthy endpoint is never auto-selected, touched, monitored, or returned as a deploy port.
4. A successful UF2 copy and a successful CDC recovery are separate outcomes.
5. Stable serial/instance identity is a matching hint, not proof that an old COM name is live.
6. Every returned post-flash port must come from a fresh scan and pass a bounded openability probe.
7. Non-Windows Unknown health preserves current behavior; do not fabricate Windows semantics.
8. Physical BOOTSEL/replug remains the universal fallback.
9. The normal daemon is never elevated or restarted as administrator.
10. Privileged recovery, when explicitly permitted, is a one-shot allowlisted helper operation followed by an unprivileged re-probe.
11. No fresh/factory-blank RP2040 is required to validate this work.

Child issues and required order

• fix(serial): preserve Windows PnP health through port probing #1146 ? preserve Windows PnP health through serial probing
• feat(windows): add scoped elevated USB PnP recovery helper #1148 ? add the scoped one-shot elevated USB recovery helper
• fix(rp2040): reject phantom CDC ports and give recoverable BOOTSEL guidance #1147 ? consume health in the RP2040 deployment state machine, integrate optional recovery, and validate on hardware

Recommended close order:

1. fix(serial): preserve Windows PnP health through port probing #1146 first. It establishes DetectedPort/health semantics and prevents new consumers from receiving lossy records.
2. feat(windows): add scoped elevated USB PnP recovery helper #1148 second. Build and test the generic policy/helper/control plane against fakes. Finish and merge it before beginning fix(rp2040): reject phantom CDC ports and give recoverable BOOTSEL guidance #1147 so the final integration consumes one stable contract.
3. fix(rp2040): reject phantom CDC ports and give recoverable BOOTSEL guidance #1147 last. Integrate both pieces into RP2040 selection/reacquisition and collect real hardware evidence.
4. Close this meta only after all three PRs are merged to main and the end-to-end criteria below are met.

Cross-platform scanning parity remains tracked by #1091. Link it in PRs where relevant, but do not expand this Windows recovery meta into untestable macOS/Linux implementation work.

Recovery ladder

For an RP2040 deploy, the final behavior is:

1. Enumerate all candidates with identity and health.
2. Show unhealthy devices diagnostically.
3. Select only a non-ambiguous candidate that is not known-unhealthy.
4. If healthy runtime CDC is openable, perform the existing 1200-baud BOOTSEL touch.
5. Otherwise use the existing manual BOOTSEL-volume path.
6. Copy UF2 with current identity/ambiguity safeguards.
7. Wait through the bounded fix(rp2040): wait for delayed Windows CDC handoff #1095 handoff window.
8. Rescan, match stable identity, require healthy/not-known-unhealthy state, and open the port.
9. Return a port only after step 8 succeeds.
10. If the exact target is phantom/problem:
    • default/--no-admin: print exact direct-port and BOOT+RESET recovery instructions;
    • explicit interactive --admin: run the feat(windows): add scoped elevated USB PnP recovery helper #1148 helper once, then repeat step 8;
    • if still unhealthy: stop recovery and give physical BOOTSEL/replug guidance.
11. Never rebuild/reflash merely to continue a post-flash USB recovery attempt.

Architecture map

Health source

crates/fbuild-serial/src/ports.rs owns the enriched record and Windows classification.

Health must survive through:

Windows PnP enumeration
  -> fbuild-serial DetectedPort
  -> CLI scan + daemon inventory
  -> RP2040 PicoCdcPort
  -> rp2040_target selection
  -> post-flash reacquisition
  -> DeploymentResult.port

Any conversion back to name/VID/PID-only data before selection is a blocker.

RP2040 consumer

Primary files:

• crates/fbuild-deploy/src/rp2040.rs
• crates/fbuild-deploy/src/rp2040_target.rs
• crates/fbuild-daemon/src/handlers/operations/deploy.rs

Do not accept a fix that merely remembers the unique pre-flash COM name. The known bad patch shape is:

manual BOOTSEL + one historical runtime candidate
  -> remember COM12
  -> UF2 succeeds
  -> return COM12 by serial match

That is unsafe unless the preflight record was healthy/openable and the post-flash endpoint is independently rediscovered, healthy, and openable.

Privileged recovery boundary

The normal CLI owns UAC policy. The normal daemon owns deployment state. The elevated process is the already-shipped fbuild.exe in a hidden helper mode and performs no daemon/cache/build work.

normal daemon detects exact unhealthy target
  -> typed recovery request
  -> normal CLI applies --admin/--no-admin policy
  -> one-shot elevated helper revalidates identity and repairs PnP
  -> helper exits
  -> normal daemon rescans/reopens

There is no ?administrator daemon mode.?

Agent handoff procedure

A smaller model should follow this without redesigning the project:

1. Read this meta and the selected child completely.
2. Work on one child per PR from current origin/main; do not combine all three into one large patch.
3. Read repository CLAUDE.md and crates/CLAUDE.md.
4. Use the repository's required soldr-prefixed Rust commands exactly as shown below.
5. Inventory port enumeration before editing:

   rg -n available_ports\( crates
   rg -n PicoCdcPort|catalogue_pico_cdc_ports|post_deploy_recovery crates/fbuild-deploy crates/fbuild-daemon
   

6. Preserve fbuild port scan misses all PJRC/Teensy (VID 16C0) serial ports + no vendor/product resolution for them #962 diagnostic visibility while separating selection eligibility.
7. Do not add a workspace crate for the helper.
8. Keep Windows APIs behind abstractions so CI tests use deterministic fixtures and never show UAC.
9. Run the child issue's targeted tests and clippy before opening its PR.
10. In the PR body, include:
    • Fixes #<child>;
    • exact files/state transitions changed;
    • test commands and counts;
    • any caller intentionally left on raw/lossy serial enumeration;
    • hardware evidence only when actually collected.
11. Do not close fix(rp2040): reject phantom CDC ports and give recoverable BOOTSEL guidance #1147 based only on unit tests. Its closeout requires real fbuild-only RP2040 evidence.
12. Do not claim test(usb): validate port scanning parity on macOS and Linux #1091 was validated from Windows.

Required automated validation

At minimum, across the three PRs:

soldr cargo test -p fbuild-serial
soldr cargo test -p fbuild-deploy rp2040
soldr cargo test -p fbuild-cli
soldr cargo test -p fbuild-daemon
soldr cargo clippy -p fbuild-serial -p fbuild-deploy -p fbuild-cli -p fbuild-daemon --all-targets -- -D warnings

Required deterministic matrix:

• healthy/present;
• present/problem;
• phantom/non-present;
• metadata unknown;
• explicit stale port;
• renumbered healthy port;
• delayed healthy handoff;
• open failure despite healthy metadata;
• manual BOOTSEL with zero/one/multiple runtime candidates;
• UAC disabled/default/allowed/cancelled;
• helper identity mismatch;
• helper success followed by unhealthy re-probe;
• post-flash recovery without reflash;
• concurrent normal daemon use during helper lifetime.

Required hardware closeout

Use fbuild through FastLED AutoResearch; do not substitute raw pyserial, picotool, or manual copy commands.

Use an isolated five-digit dev daemon port so another fbuild worktree cannot be stomped:

uv pip install --python .venv\Scripts\python.exe --editable C:\path\to\fbuild-worktree
$env:FBUILD_DEV_MODE = '1'
$env:FBUILD_DAEMON_PORT = '55175'
bash autoresearch rp2040 --rpc-smoke --timeout 120s --skip-lint

Capture:

• fbuild port scan before deployment;
• selected board/serial/instance and health;
• BOOTSEL volume discovery;
• UF2 transfer;
• post-flash scan and openability;
• RPC discovery/ping/drivers/error-handling success on a healthy CDC return;
• if the phantom state reproduces, proof that no port was returned plus the exact recovery diagnostic;
• for feat(windows): add scoped elevated USB PnP recovery helper #1148, one UAC accept and one cancel path, with proof that fbuild-daemon.exe stayed normal privilege and kept the same endpoint/PID.

An already-flashed board is sufficient. Do not require or consume another factory-fresh board.

Meta completion criteria

• fix(serial): preserve Windows PnP health through port probing #1146, feat(windows): add scoped elevated USB PnP recovery helper #1148, and fix(rp2040): reject phantom CDC ports and give recoverable BOOTSEL guidance #1147 are closed by merged PRs.
• Port health is preserved from Windows PnP scan through daemon/deployer selection.
• fbuild port scan identifies phantom/problem endpoints without hiding them.
• Known-unhealthy endpoints cannot be selected explicitly or automatically until recovery makes them healthy.
• Healthy RP2040 CDC deployment and RPC smoke pass on real hardware.
• UF2 success plus phantom CDC reports flash success but returns no stale port.
• Manual BOOTSEL instructions are correct and actionable.
• --admin performs at most one scoped helper operation; --no-admin never prompts.
• No elevated daemon, broad USB reset, driver mutation, hub restart, privileged cache write, or duplicate flash is introduced.
• Unit/security/concurrency tests pass on main.
• test(usb): validate port scanning parity on macOS and Linux #1091 remains the explicit macOS/Linux parity follow-up rather than an unsupported claim in this meta.

Non-goals

• Re-proving first-ever blank-board UF2 bring-up.
• Replacing existing CDC deployment.
• Hiding Windows device-history records globally.
• Treating longer timeouts as health detection.
• General-purpose administrator mode.
• Driver installation/removal or motherboard USB-controller recovery.
• macOS/Linux implementation without hosts; see test(usb): validate port scanning parity on macOS and Linux #1091.

Related history

• feat(rp2040): zero-touch deployment to an as-received Pico via ROM UF2 #1040 ? as-received RP2040 ROM UF2 deployment
• feat(rp2040): deploy parity with Arduino-Pico, PlatformIO, and official picotool #1049 ? RP2040 deployment parity
• fix(rp2040): tolerate delayed Windows CDC re-enumeration after UF2 flash #1074 / fix(rp2040): wait for delayed Windows CDC handoff #1095 ? delayed CDC handoff
• bug(rp2040): reject unrelated pre-existing CDC port after stock-ROM deploy #1080 ? reject unrelated CDC candidate
• feat(rp2040): warn on host-detected failed USB devices and hub attachment #1087 ? failed-device and hub warning
• fbuild port scan misses all PJRC/Teensy (VID 16C0) serial ports + no vendor/product resolution for them #962 ? unusual/phantom diagnostic visibility
• fbuild: absorb post-deploy serial wedge recovery into SerialMonitor; deprecate client pyserial #605 ? post-deploy recovery seam
• test(usb): validate port scanning parity on macOS and Linux #1091 ? macOS/Linux USB scanning parity

Canonical execution contract (read before acting)

Only #1146, #1148, and #1147 are children of this meta. References to #962, #1095, #1074, #605, #1087, #1080, and #1091 are history, dependencies, or follow-up context; do not close, reopen, or add them to this burn-down. #1091 remains an explicitly separate macOS/Linux parity follow-up.

Execute strictly in this order: #1146 merged and validated -> #1148 merged and validated by fakes -> #1147 merged and hardware-validated -> close this meta. Do not start a later child from an earlier child's branch, and do not merge all children in one PR. One child has one focused PR unless a documented review fix requires a follow-up PR.

For every child: branch from fresh origin/main; use Closes #<child> in its PR; run the child's focused tests plus its listed clippy command; validate again after merge against current main; then close/tick only that child. Do not tick a checkbox or close an issue because code merely exists on a local branch or CI is green.

If a source fact disagrees with this roadmap, preserve the locked safety invariants and add a concise issue comment naming the file/symbol and discrepancy. Do not substitute a simpler but lossy design, relax health/openability checks, or turn a diagnostic record into an eligible deploy target merely to keep moving.

Required handoff record per child

The final PR/issue comment must state: merged PR URL and merge SHA; exact targeted commands and result; source callers migrated or intentionally left raw with rationale; whether hardware was touched; and the precise remaining next child. Hardware claims must name the board serial and health state actually observed. No simulated or historic run counts as the final #1147 hardware proof.

[zackees]

[meta #1149] child #1146 -> CLOSED via #1150 (merged 66f73ac6); validated on current origin/main with soldr cargo test -p fbuild-serial health_tests (2 passed).

[zackees]

[meta #1149] child #1148 -> CLOSED via #1151 (merged 71096e2). Validated: serial recovery allowlist tests (5), CLI helper/policy tests (5), explicit deploy flag/parser tests, and strict four-crate clippy. No UAC, daemon restart, or hardware action was performed; #1147 is next.

[zackees]

New child for the unresolved no-button Windows recovery path: #1152. It consumes #1148's helper foundation and #1147's fail-closed RP2040 state machine to recover the attached board through scoped admin PnP re-enumeration.

[zackees]

[meta #1149] Live-state re-verified 2026-07-22 on the attached board (serial 5303284720C4641C): BOOTSEL composite healthy with RPI-RP2 on G:, PICOBOOT MI_01 at CM_PROB_FAILED_INSTALL (28), runtime CDC devnode phantom (historical COM12). Full evidence, hypotheses (stale volume mapping / ROM MSC state / driver-binding orthogonality), and the sequenced recovery strategy posted on #1152: #1152 (comment). Execution order per this meta: #1147 (open, next) then #1152, with hardware closeout on the currently-preserved live wedge — no deploy attempts until the new code is in place.

[zackees]

[meta #1149] Child #1152 CLOSED via merged PR #1156 (squash 9284e60, admin-merged per precedent). Unit/policy/concurrency validation in the PR body. Hardware closeout starting now on the preserved wedge (serial 5303284720C4641C): editable install of main into the FastLED venv, isolated FBUILD_DAEMON_PORT=55175, port-scan pre/post, default-policy fail-closed run, then the interactive --admin recovery run and autoresearch rpc-smoke. Evidence to follow on #1152/#1147 and here.