meta(audit): roll-up dashboard for open audit work in fbuild

[zackees]

Purpose

Single dashboard for every open "audit:" issue in this repo, so reviewers don't have to grep the issue list to see what audit work is in flight. Existing per-area trackers (notably #664) are kept as-is — this meta just links to them and groups them by theme. Marking an item below [x] means the underlying issue is closed.

Snapshot taken 2026-06-30. 19 open issues with audit: (or audit-flavored) in the title clustered into 3 themes.

Theme 1 — platform_packages override audit (parent #664, 16 per-platform sub-issues)

#664 already organizes this; nothing to migrate. Tracked here so the meta is complete.

• tracking: platform_packages override silently ignored across framework packages (audit, sibling of #663) #664 — tracking: platform_packages override silently ignored across framework packages
  • audit: apollo3 / framework-arduinoambiqapollo3 — does platform_packages override work? (parent #664) #665 — apollo3 / framework-arduinoambiqapollo3
  • audit: arduino_api — does platform_packages override work? (parent #664) #666 — arduino_api
  • audit: avr / framework-arduino-avr — does platform_packages override work? (parent #664) #667 — avr / framework-arduino-avr
  • audit: arduino_mbed / framework-arduino-mbed — does platform_packages override work? (parent #664) #668 — arduino_mbed / framework-arduino-mbed
  • audit: attiny — does platform_packages override work? (parent #664) #669 — attiny
  • audit: ch32v — does platform_packages override work? (parent #664) #670 — ch32v
  • audit: esp8266 / framework-arduinoespressif8266 — does platform_packages override work? (parent #664) #671 — esp8266 / framework-arduinoespressif8266
  • audit: esp32 / framework-arduinoespressif32 — does platform_packages override work? (parent #664) #672 — esp32 / framework-arduinoespressif32 + platform-espressif32
  • audit: nrf52 / framework-arduinoadafruitnrf52 — does platform_packages override work? (parent #664) #673 — nrf52 / framework-arduinoadafruitnrf52
  • audit: renesas (UNO R4) / framework-arduinorenesas — does platform_packages override work? (parent #664) #674 — renesas (UNO R4) / framework-arduinorenesas
  • audit: rp2040 / framework-arduinopico — does platform_packages override work? (parent #664) #675 — rp2040 / framework-arduinopico
  • audit: sam / framework-arduino-sam — does platform_packages override work? (parent #664) #676 — sam / framework-arduino-sam
  • audit: samd / framework-arduinosam — does platform_packages override work? (parent #664) #677 — samd / framework-arduinosam
  • audit: silabs / framework-arduinosilabs — does platform_packages override work? (parent #664) #678 — silabs / framework-arduinosilabs
  • audit: stm32 / framework-arduinoststm32 — does platform_packages override work? (parent #664) #679 — stm32 / framework-arduinoststm32
  • audit: teensy / framework-arduinoteensy — does platform_packages override work? (parent #664) #680 — teensy / framework-arduinoteensy

Summary: every framework package in crates/fbuild-packages/src/library/ pins its framework URL + commit + sha256 as const &str and the per-platform orchestrators never read platform_packages from env_config. PlatformIO honors the consumer's override; fbuild silently ignores it. 16/16 audited platforms exhibit the bug. Blocks downstream bisection workflows (e.g. FastLED/FastLED#3325). Fix surface is consistent across platforms (read env_config["platform_packages"], override the const pin), so an enabling shared helper + 16 mechanical migrations.

Theme 2 — #826 testing-architecture followups

Both arose during the #826 gotcha-sweep. They are process / design concerns, not lint-shaped.

• audit: replace mock-based tests with integration tests where the contract is a real integration boundary (#826 followup) #838 — audit: replace mock-based tests with integration tests where the contract is a real integration boundary
  • Mocks standing in for subprocess / FS / serial boundaries let behavioral drift slip through. Offenders identified by dylint: candidate code-quality lints (gotcha sweep) #826: crates/fbuild-build/tests/compile_many_two_stage.rs, crates/fbuild-daemon/tests/test_emu_exit_code.rs, plus the broader pattern in compile_many.rs, compile_many_stage2_perf.rs, fbuild-deploy/src/esp32/tests.rs. Proposes a CodeRabbit rule for new mocks + a per-file migration decision.
• process: ignored-test policy + bitrot audit (#826 followup) #839 — process: ignored-test policy + bitrot audit
  • ~50+ #[ignore] annotations; most legitimate (hardware-required), some stale. Proposes ci/audit_ignored_tests.py + a weekly cron that diffs the inventory + policy that every #[ignore] carries either a tracking-issue ref or a hardware requirement, with 90-day escalation.

Theme 3 — none currently open outside the above

All other historical audit: issues (#802, #813, #818, #519, #408, #150, #806-#812, #814-#821 etc.) are closed and listed in the appendix below for reference.

Closed audits (reference)

For context — these completed and are linked so the meta has the full audit lineage in one place:

• audit: blocking operations with no timeout (meta) #802 (closed) — blocking ops with no timeout (meta); 7 sub-issues audit: blocking operations with no timeout in fbuild-serial (sub-issue of #802) #803-audit: blocking operations with no timeout in fbuild-cli + fbuild-python (sub-issue of #802) #810, audit: blocking operations with no timeout in CI scripts + workflows (sub-issue of #802) #812 all closed
• audit: go fully async — whole-app tokio runtime sharing for tokio-console (meta) #813 (closed) — go fully async / whole-app tokio runtime (meta); 7 sub-issues audit: sync code that could be async in fbuild-serial (sub-issue of #813) #814-audit: sync code that could be async in fbuild-packages (sub-issue of #813) #821 all closed
• audit: cross-cutting async/sync architecture (sub-issue of #813) #818 (closed) — cross-cutting async/sync architecture (sub-issue of audit: go fully async — whole-app tokio runtime sharing for tokio-console (meta) #813)
• meta: burndown for ino preprocessing and PlatformIO compatibility audit #652 (closed) — burndown for ino preprocessing + PlatformIO compat
• audit: consolidate duplicated board / config resolution call sites across the workspace #519 (closed) — consolidate duplicated board/config resolution
• track(ci): warnings that are actually critical — meta-audit across ESP/Teensy/RP/nRF/STM32/AVR #408 (closed) — meta-audit of CI warnings across ESP/Teensy/RP/nRF/STM32/AVR
• build: audit zccache per-TU cache keys for cross-runner stability #150 (closed) — zccache per-TU cache keys cross-runner stability

How to use this issue

• A new audit issue lands → add a checkbox under the appropriate theme (or open a Theme 4 if needed).
• An audit issue closes → tick its box here; no other action needed.
• A theme's parent issue closes (e.g. tracking: platform_packages override silently ignored across framework packages (audit, sibling of #663) #664 if all 16 children resolve) → tick all its children + the parent and move the theme to the closed-audits section.
• This is a dashboard, not work itself. Closing this issue means there are no open audits — until then, leave it open.

[zackees]

All open audit work in the dashboard resolved.

Theme 1 — platform_packages override audit (#664): closed via #870/#871/#873/#876/#877. 16/16 per-platform sub-issues #665-#680 resolved.

Theme 2 — testing-architecture followups:

• audit: replace mock-based tests with integration tests where the contract is a real integration boundary (#826 followup) #838 (mock-vs-integration audit) closed via fix(testing): audit mock-vs-integration tests + add CodeRabbit rule + policy doc (#838) #879 — .coderabbit.yaml rule + audit table + policy line in CLAUDE.md.
• process: ignored-test policy + bitrot audit (#826 followup) #839 (ignored-test inventory) closed via fix(testing): add ignored-test audit script + weekly cron + policy (#839) #881 — ci/audit_ignored_tests.py + weekly cron workflow + policy section. Tracking issue tracking: ignored-test inventory (driven by #839) #880 is the long-lived output dashboard the workflow writes to weekly (stays open by design).

No audit: issues remain open. Future audit work can either reuse this dashboard pattern or just live in a fresh tracking issue; this meta has fulfilled its purpose.