perf(packages): overlap download/extract/install in cold pioarduino-resolve (~185-237s serialized)

[zackees]

Context

#942 Phase-1 baseline, cold scenario: pioarduino-resolve costs 185–237 s of the 700–808 s cold build. Harness stderr shows repeated sequential esp32-xtensa-gcc ... downloading lines followed by a single extracting — the pipeline downloads, then extracts, then installs, largely serially. fbuild-packages has a download_all() that spawns per-URL tasks, but the ESP32 toolchain/platform/framework ensure path doesn't fan out, and extraction doesn't overlap with the next download.

Additionally fw-libs (162 s cold) only starts after resolve completes — compile of framework libs could begin as soon as its toolchain + sources are present rather than waiting for all packages.

Proposal

• Pipeline the package ensure path: concurrent downloads (bounded), extract-while-downloading (stream or start extract per completed archive), install-while-extracting.
• Start dependent build phases as soon as their specific inputs are ready instead of barriering on full resolve (e.g. fw-libs can start when toolchain + core sources exist, while libraries still download).

Acceptance criteria

• Cold pioarduino-resolve wall time (or its serialized portion) drops by ≥ 40% in the meta: profiling-driven build performance burndown (Docker Linux, NightDriverStrip, cold vs hot cache) #942 harness on the same network.
• Off-CPU profile shows downloads overlapping extraction/install instead of sequential stair-steps.
• Cold total drops correspondingly (~774 s baseline).
• Before/after harness numbers in the PR description.

Part of #942.

[zackees]

In progress on branch fix/953-cold-overlap (local worktree, not pushed): clippy + tests green, cold-build A/B profiling incomplete. Full handoff + remaining overlap candidates in #990.

[zackees]

Measured the #953 branch against the #942 Docker cold profile harness on the same Windows/Docker Desktop host and same network, two cold runs per side.

Baseline: origin/main at 7f28fc67 (after #989)

• command: python ci/docker-profile/run_profile.py -n 2 --scenarios cold --out ci/docker-profile/out/codex-baseline-main-7f28fc67-py
• cold iter 1: 907.43s, exit 0
• cold iter 2: 802.80s, exit 0
• median: 855.1s
• artifacts: .claude/worktrees/953-baseline/ci/docker-profile/out/codex-baseline-main-7f28fc67-py

Candidate: fix/953-cold-overlap at e31bcc59 (rebased onto current origin/main including #991)

• command: python ci/docker-profile/run_profile.py -n 2 --scenarios cold --skip-image --out ci/docker-profile/out/codex-candidate-953-e31bcc59
• cold iter 1: 863.97s, exit 0
• cold iter 2: 943.01s, exit 0
• median: 903.5s
• artifacts: .claude/worktrees/953/ci/docker-profile/out/codex-candidate-953-e31bcc59

Focused local checks on the rebased branch also pass:

• soldr cargo fmt --all
• soldr --no-cache cargo test -p fbuild-packages esp32_framework --message-format=short
• soldr --no-cache cargo check -p fbuild-build -p fbuild-packages --message-format=short
• git diff --check origin/main..HEAD

Conclusion: this does not meet the >=40% cold pioarduino-resolve acceptance bar and the cold total is not improved (median regressed from 855.1s to 903.5s). Closing as diminishing-return rather than shipping a concurrency change that does not have measurable benefit.