hardening: mercury security audit — 12/16 → 13/16 blocking gates, 76% → 83% - #1
Merged
Merged
Conversation
Survey-depth pass over crates/ffai-mercury against the 41-gate hardening registry: 2 Completed / 0 Scheduled / 33 Incomplete / 6 N/A, v1.0.0 gates 1/16. Survey depth cannot pass an outcome gate, so ~8 of those Incompletes are for want of a tool run rather than a known failure. Headline findings: - the model cache is parsed as TRUSTED. tts/vits.rs reads ONNX bytes and two JSON configs, tts/lexicon.rs reads a binary lexicon, and asr/model.rs mmaps safetensors (UB if the file is mutated underneath the mapping). The hash verification the README advertises is not in this crate - no sha2 dependency; it lives upstream in ffai-models and is never asserted at this boundary. - no .github/workflows exists anywhere in the repository. - overflow-checks is absent from [profile.release] (lto = "thin" and nothing else), so release arithmetic wraps silently. - unsafe spans six files with 7 SAFETY comments against ~30 sites and no UNSAFE.md; unsafe impl Send/Sync on SendPtr is justified only by a comment. - byte-stable determinism is an advertised invariant with no property test. Nothing is Scheduled: that requires an owner and a date, which is a human decision rather than an audit output. The plan file carries a proposed order. The README block is GENERATED from the plan file. Edit docs/plans/use-protection-please.md and re-render; never edit between the HARDENING-TABLE markers. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The hardening block lives in BOTH the crate README and the standalone landing page (Remade-With-Rust/mercury, README.md on main), rendered from this one plan file rather than copied - two copies in two repos drift. Recording them here so the next audit re-renders every mirror in the same pass. A stale mirror is worse than no mirror: it reports a posture the unit no longer has. Note for the crates.io page, which renders the crate README: relative links do not resolve there, so any mirrored block needs the absolute --link form. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds an **Architect** header field to the plan file, rendered as the last line of the generated block in every README and mirror: **Architect** — [Nick Overlock](https://www.linkedin.com/in/nick-overlock-593235b9/) The field is per-unit and read from the plan file, never hardcoded in the renderer: an architect who moves on is changed in one plan file rather than in a script. Omit the field and the line is absent rather than empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds Phase 12 - 14 C-gates covering the technical controls that SOC 2, PCI DSS, GDPR/PII, HIPAA, EU CRA and NIST SSDF ask for and that the 41 H-gates do not: data inventory, data-flow map, encryption in transit/at rest, key management, retention and erasure, audit logging, log hygiene, least-privilege access, subprocessor inventory, incident response, change management, availability commitments, and SBOM/provenance. Scope triage for this unit declares gdpr-pii IN SCOPE and says why the others are not. The finding that drove it: mercury's diarizer and speaker modules derive VOICEPRINTS, which are GDPR Art 9 special-category biometric data when used to identify a person. Nothing in the crate documents this today, so C-01 (data inventory) and C-06 (erasure must reach embeddings, not just recordings) are live obligations rather than paperwork. Coverage renders as "technical controls met", never as a certification claim - a crate cannot be SOC 2 compliant, and a README saying so would be a false statement to customers. The block carries that disclaimer inline. GDPR technical controls: 0 of 8 in-scope gates evidenced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… fixed
Second pass on the ffai-mercury audit. 5% -> 40%; v1.0.0 gates 1/16 -> 5/16;
GDPR technical controls 0/8 -> 5/8. Phases 0 (threat modeling) and 1 (toolchain)
are now fully closed.
SUPPLY CHAIN - the gate found real problems on first run
deny.toml added; `cargo deny check` went from FAILING to
"advisories ok, bans ok, licenses ok, sources ok".
- h2 RUSTSEC-2026-0258 (unbounded empty DATA frames) FIXED: 0.4.15 -> 0.4.16.
It reached mercury through hf-hub -> hyper on the weight-download path.
- paste + ttf-parser (unmaintained) and pyo3 x2 (two real vulnerabilities)
carry DATED, SCOPED ignore justifications. pyo3 is ffai-py only and needs a
semver-major bump, so it gets a shorter review deadline (2026-09-30).
- Licence policy now enforces "nothing GPL" instead of merely claiming it.
IJG (rusty_jpeg) and CDLA-Permissive-2.0 (webpki roots) allowed deliberately.
DOCUMENTS THAT DID NOT EXIST
- SECURITY.md: private advisory reporting, 3/10/90-day windows, coordinated
disclosure, and an explicit GDPR Art 33 72-hour clause.
- crates/ffai-mercury/docs/threat-model.md: 3 trust boundaries, full STRIDE,
and the highest-value attack path written out - the model cache is trusted
and this crate does not verify it, so mmapped safetensors are UB the moment
anyone with write access mutates the file.
- crates/ffai-mercury/docs/data-inventory.md: 7 data classes. D3 (speaker
embeddings) named as GDPR Art 9 special-category biometric data, with the
obligations that land on the embedding application rather than on us.
- crates/ffai-mercury/UNSAFE.md: all 28 unsafe sites in 3 classes. Note that
#[allow(unsafe_code)] HIDES sites from the lint - 25 warn, 3 were invisible.
21 of 28 still lack a SAFETY comment, so H-16 stays open.
BUILD AND CI
- rust-toolchain.toml pins 1.95.0 with the components the gates need.
- .cargo/config.toml gains Linux/GNU binary hardening (full RELRO, -z now,
noexecstack, frame pointers), target-scoped so it cannot override the
existing wasm section or break MSVC.
- [profile.release]: overflow-checks = true, codegen-units = 1. PERF IMPACT IS
UNMEASURED - integer-only, and the hot paths are f32/f16 plus intrinsics,
but the benchmark gates must be re-run before merge. Tracked as R-005.
- .github/workflows/harden.yml + dependabot.yml.
- tools/render_hardening_table.py vendored so CI can verify the README blocks
are current; ~/.claude is invisible to a runner.
CI IS GREEN ON ARRIVAL, DELIBERATELY
fmt and clippy are advisory with a dated TODO: the workspace carries ~1539
pre-existing rustfmt diffs and clippy findings in other crates. Blocking on
that today would make the gate red from the first run, which teaches people to
ignore CI. Blocking jobs are test, supply-chain and hardening-tables, all three
verified locally before this commit. -A unsafe_code is permanent: that lint is
an inventory signal and UNSAFE.md is what enforces it.
STILL OPEN: 21 SAFETY comments (H-16), fuzzing (H-26/27), Miri and TSan
(H-23/24) - TSan is the one that would test the SendPtr disjointness claim that
currently exists only as a comment - SBOM (H-12), and branch protection (C-12),
which is a repository settings change rather than a code change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
43%, v1.0.0 gates 6/16. Fixes the issues the previous pass left open, except the
two that are not mine to make.
H-16 CLOSED. Every one of the 28 unsafe sites now states its invariant AND what
upholds it, instead of existing as an unexplained `unsafe`:
- Class A (23 SIMD sites): names the dominating have_avx2()/have_f16c() check.
These are #[target_feature] functions, so the ONLY thing making them sound is
that runtime detection dominates every call site - now written where a future
caller will actually read it.
- flash_attn's f16 K/V views additionally reinterpret candle's F16 storage as
u16. Sound because half::f16 is repr(transparent) over u16 and the tensors
were checked contiguous. Previously unstated anywhere.
- Class B (SendPtr across rayon): the comment now names the real arithmetic -
chunk co-ranges partition [0, c_out), block t-ranges partition [0, l_out) -
and says outright that the impl becomes unsound if that changes.
- Class C (mmapped safetensors): states plainly that the invariant is NOT
enforceable from inside this crate.
FUZZING (H-26 advanced, not closed)
Added crates/ffai-mercury/fuzz with libFuzzer harnesses for the two HAND-ROLLED
byte parsers - onnx::parse (a hand-written protobuf reader, and the highest-risk
path in the threat model) and Lexicon::load - each with a seed corpus. The fuzz
crate sits outside the workspace on purpose: cargo-fuzz brings its own nightly
and sanitizer flags and must not inherit lto/codegen-units=1.
Still uncovered: the three JSON config paths and phoneme_ids. The harnesses have
NOT been executed - cargo-fuzz needs nightly and is not installed here - so
H-26 and H-27 both stay open. Committing a target you have not run does not
close a gate.
ffai-core: fixed its one derivable_impls finding; that crate is now clippy-clean.
H-15 MEASURED rather than guessed: clippy reports 266 findings in ffai-mercury
alone (approximate-PI constants, clamp patterns, doc formatting). That is why the
CI clippy job is advisory with a dated TODO instead of blocking - and the number
now lives in the plan file rather than in an impression.
NOT DONE, deliberately:
- C-12 branch protection. Enabling required reviews on master while it sits 47
commits ahead of origin would break the current workflow. That is a repository
settings decision for a human.
- R-005 benchmark re-run for overflow-checks. Needs the model corpora and the
real gates, and it is the one open item that could invalidate a shipped claim.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The null arm failed first, and that is the headline.
NULL ARM CAUGHT A FALSE-SIGNIFICANT INSTRUMENT
Same binary on both sides read 5/22, z = -2.56 - a "significant" result from
identical code. Two causes, both now fixed in tools/diana_ab.ps1:
* the harness forced FFAI_PROFILE=1, putting the profiler inside the system
under test (it exists because Diana's profile_detect prints its median
that way; it is wrong for every other arm);
* no warm-up, so the 62 MB model load sat inside the paired samples.
Fixed, the null reads 13/22, z = +0.85, ratio 0.995. Everything below is
measured against that 0.5% floor.
RESULT (n=22 paired, ABBA, CPU-time primary, work parity enforced)
paired CPU-time 21/22 z = +4.26
paired in-process 22/22 z = +4.69
pipeline total 1.060x decoder stage 1.094x
Work parity: 45.0s of audio from both arms in every rep.
The arithmetic closes - decoder is 56.5% of the pipeline and slows 9.9%,
predicting 5.6% against 6.5% observed - and it is mechanistically right,
because the decoder carries the hand-written index arithmetic.
HARNESS (tools/diana_ab.ps1) gains, useful beyond this measurement:
* two-binary mode (-ExeB) for COMPILE-TIME arms an env knob cannot select;
-ExeB equal to -Exe is the null arm.
* work-count parity (-CountPattern). It throws on divergent counts rather
than reporting a ratio for arms that did different work. NOTE: the property
must not be called `count` - that is intrinsic on PowerShell arrays, so
$A.count returns the REP COUNT and the check passes vacuously. It did
exactly that on the first run here.
* -ProfileEnv (default off in two-binary mode) and -Warmup.
* a second z on the in-process metric, which carries no launch/load tax.
NOT MEASURED: the ASR path. No whisper weights are cached on this machine, and
the whisper.cpp comparison is the claim actually at risk. R-005 stays OPEN.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…k register Adds the method line and the numbers so the figure is auditable later, rather than living in a chat log: pipeline total 1.060x, decoder stage 1.094x, n=22 paired z=+4.26, null-arm floor 0.995, work parity 45.0s audio in both arms every rep. The null arm failed the FIRST time - same binary, 5/22, z=-2.56 - because the harness forced FFAI_PROFILE=1 and had no warm-up. Both fixed in the previous commit; the failure is recorded here because a floor that reads "significant" on identical code is the most useful thing a measurement session produces. Also refreshes two stale risks: R-003 said overflow-checks was OFF (it is on now) and R-004 said there was no CI (there is). Neither had been revisited after the remediation pass, which is exactly how a risk register rots. R-005 stays OPEN: the ASR path could not be measured (no cached whisper weights) and that is where the whisper.cpp claim lives. Keep / revert / split is a product decision, recorded with the numbers behind each option. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…5 closed) Decision taken with the measurement in hand: this product does not accept hostile model files, so the flag defends a threat we do not have. WHAT CHANGED [profile.release] drops `overflow-checks` (keeps lto="thin", codegen-units=1). The manifest records WHY, so the next reader does not "fix" it back: measured 1.060x on the TTS pipeline and 1.094x on the decoder stage, against a threat retired by product decision. WHY IT IS A DECISION AND NOT AN OVERSIGHT H-05 is v1.0.0-blocking, so it is now WAIVED rather than green - a waiver records that the residual risk was accepted with the numbers in front of the person accepting it, which is what STANDARD.md 14 requires. Time-bounded to 2026-11-15, and it expires IMMEDIATELY if model files become untrusted: user-supplied voice packs, a shared cache, or downloads without hash verification. R-001 (the model cache is parsed as trusted) moves from an open gap to an ACCEPTED assumption, with the same expiry condition. It is now load-bearing: the memory-mapped safetensors in asr/model.rs are UB if the file changes while mapped, and nothing in this crate can detect that. R-005 closed. NOTED WHERE USERS WILL SEE IT The crate README gains "Model files are trusted input": we do not validate model files, that is deliberate, it is why release builds carry no overflow-checks, and what it asks of the deployer - restrict write access to the model cache, install voices only from trusted sources, and rebuild with the flag on if your deployment cannot guarantee that. RENDERER FIX (found by this change) The waivers table cites a gate, so it has a row starting "| H-05 |" - which the parser read as a second checklist row and rejected as a duplicate. Its duplicate detection was right; the scoping was wrong. The parser now only reads rows inside the "## Checklist" section, so waivers and the risk register can reference gate IDs freely. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…audio
52%, v1.0.0 gates 8/16. Six gates closed, and one of them paid for itself
immediately.
H-28 FOUND A REAL DEFECT ON ITS FIRST RUN
MelSpectrogram::compute(&[]) panicked - "index out of bounds: the len is 0
but the index is 0" in reflect_pad. The n == 0 guard in reflect_index
returned index 0, which is not a valid index into an EMPTY slice: it
protected the arithmetic and not the indexing. A second latent defect sat on
the same path, `n - 1 - i.min(n - 1)` underflowing for n == 0.
AudioBuffer::samples is caller-supplied, so an empty buffer was a denial of
service in whatever process embedded us. reflect_pad is now total for n == 0
and the input is seeded as a permanent fuzz regression.
tests/properties.rs: 7 properties over the model-free untrusted surface -
length and prefix contracts for pad_or_trim_to, totality and shape contract
for compute, the frame contract for resized, and BIT-determinism of compute
and normalize. Determinism is asserted via to_bits(), not ==, because NaN is
never equal to itself and the claim being defended is BYTE stability, which
is the reason this crate's version went to 0.7.0.
FUZZING RETARGETED AT THE REAL BOUNDARY (H-26)
Accepting R-001 - model files are trusted - moved the trust boundary, and the
existing targets were aimed at the old one. Added mel_compute (audio front
end, carrying the crasher above as a seed) and normalize_text (text front
end, asserting determinism). onnx_parse and lexicon_parse stay as
defence-in-depth over trusted-but-parsed data.
ALSO CLOSED, on evidence already gathered
H-39 SECURITY.md - was left Incomplete by oversight; the file has existed
since the first remediation pass.
H-20 secrets - no key material exists to zeroize, and the C-08 audit showed
nothing content-bearing is logged by default. The asset here is personal
data, not secrets.
H-21 concurrency - the manual Send/Sync impls now carry the disjointness
argument that justifies them. Proving it with TSan is H-24 (R-002).
H-19 stays OPEN: channels == 0 and other AudioBuffer combinations are still
unaudited end to end, and that needs cached weights.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…er showed
H-22 closed, and it immediately found a hole in this audit's own work.
tools/lint_invariants.py enforces four FFai-specific invariants that no general
linter knows about, as a CI job:
R1 every unsafe site carries a // SAFETY: comment (H-16)
R2 every #[allow(unsafe_code)] file is named in UNSAFE.md (H-16)
R3 mem::forget / Box::leak / transmute are banned, with an
explicit // LINT-ALLOW: override (STANDARD 4.2)
R4 no println!/dbg! in LIBRARY code - stdout is the caller's (C-08)
Chosen over Semgrep on purpose: its Rust support is experimental, while these
rules are deterministic, stdlib-only, and enforce things specific to this
codebase.
WHAT IT FOUND
Three unsafe sites with no SAFETY comment, all
#[cfg(not(target_arch = "x86_64"))] fallback stubs. They were invisible to
the previous pass because that inventory was built from COMPILER WARNINGS on
an x86 machine, and the compiler has no opinion about code it is not
compiling. UNSAFE.md said 28 sites; the real number is 31.
That is the SECOND blind spot in a warning-derived inventory - the first was
#[allow(unsafe_code)], which silences the lint and therefore the inventory.
The general lesson is now recorded in UNSAFE.md: an inventory built from
compiler output describes ONE CONFIGURATION, not the crate. R1 and R2 read
the source instead, so neither can reopen.
R4 also fired 30 times on ffai-cli and ffai-demo - wrongly. A binary's stdout
IS its output channel. The rule now applies only to crates with a lib.rs, and
skips main.rs and src/bin. A lint that cries wolf on correct code gets
disabled, so this mattered more than the count suggested.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ker named 60%, v1.0.0 gates 8/16. Installed the missing tools and ran them, rather than recording that they exist. H-11 GEIGER BASELINE ARCHIVED (docs/geiger-baseline.txt) ffai-mercury's own row: 14/27 unsafe functions, 2/2 unsafe impls, 1216/2274 unsafe expressions. That INDEPENDENTLY CONFIRMS UNSAFE.md - 14 unsafe fn and 2 unsafe impl - from a tool that reads the source instead of the compiler's warnings about one target, which is exactly the blind spot that made the first inventory say 28 sites instead of 31. --all-features is unusable here: it turns on candle's `metal`, which pulls the macOS-only objc2 and cannot compile on Windows. The flag was the fault, not the tool. H-25 CARGO-CAREFUL GREEN 128 passed, 0 failed under extra debug assertions, -Zextra-const-ub-checks and -Zstrict-init-checks. Needs an explicit +nightly: rust-toolchain.toml pins stable, where careful cannot build its sysroot. H-23 MIRI - BLOCKER MEASURED, AND IT IS NOT WHAT WE ASSUMED Aborts on asr::adaptive::tests::an_explicit_override_wins. The cause is WIN32 FFI, not SIMD: the memory-instrumentation path calls SetProcessWorkingSetSizeEx, GetCurrentProcess, K32GetProcessMemoryInfo and CreateJobObjectW through `unsafe extern "system"`, and Miri cannot execute foreign functions. An earlier note in the plan guessed the AVX2 intrinsics would block first; they are reached later, if at all. The gate stays open with a concrete path: run Miri on a LINUX runner, where that code is not compiled. Gate stays Incomplete. A tool that could not run is not a pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
H-15 advanced hard, and the old evidence was wrong twice.
THE EARLIER "266 FINDINGS" NUMBER WAS MISLEADING
199 of them were a SINGLE lint - unsafe_op_in_unsafe_fn, the edition-2024
migration that wants each unsafe OPERATION marked inside an unsafe fn rather
than the whole function being implicitly unsafe. That is hygiene work, not
code smell, and quoting it as "266 clippy findings" overstated the mess.
Separately, the --all-targets run reported only 4 because clippy aborted on
ffai-media under -D warnings before it ever reached mercury. An instrument
reporting FEWER problems for a WIDER scope should have been chased sooner.
HOW THE 267 WENT TO 0
204 cargo fix (the unsafe_op_in_unsafe_fn migration)
~30 cargo clippy --fix (collapsible ifs, redundant bindings, ...)
6 per-site #[allow(dead_code)] with a reason each - test-verified
relative-attention helpers, an 80 MB oracle field allocated only when
scoring, a protobuf wire variant that documents the format. Per-site,
not crate-wide, so FUTURE dead code still surfaces.
7 crate lint policy for classes that are design decisions, each justified
in the manifest: manual_clamp (clamp differs from max/min on NaN and
these paths carry audio samples), approx_constant + excessive_precision
(constants transcribed from the reference; "nicer" literals change the
bits and this crate advertises byte-stable output), too_many_arguments
and type_complexity (kernel arity), needless_range_loop (the iterator
form changes what the autovectoriser emits on the measured hot path).
3 by hand, including a STRANDED DOC COMMENT: "Load from the HF safetensors
layout" sat above fallback_proj, describing a function that is not
there. A doc comment that lies about the item beneath it is worse than
none.
VERIFIED BEHAVIOUR-PRESERVING, not assumed
128 lib tests + 7 property tests pass; cargo-careful green; and cargo-geiger
reports an IDENTICAL unsafe surface afterwards - 14/27 functions, 2/2 impls,
1216/2274 expressions - despite the migration touching every SIMD kernel.
The invariant linter then caught a side effect of my own fix: cargo fix wrote
6 whole-body `) { unsafe {` wrappers with no SAFETY comment near them. They
are reformatted with a note saying the contract lives on the fn signature.
A lint that catches the fixer is worth more than one that only catches you.
CI: clippy is now BLOCKING for ffai-mercury and advisory for the rest - a crate
becomes blocking the moment it is clean, rather than waiting for the workspace.
Gate stays Incomplete: the criterion asks for pedantic + nursery workspace-wide,
and 11 crates are not yet clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Consequence of the previous commit, and worth being explicit about: running `cargo fmt -p ffai-mercury` to tidy the files I edited reformatted the WHOLE crate - 116 files, 76 of them examples I never touched. That was wider than intended. The upside is real and worth banking: ffai-mercury is now rustfmt-clean, so the fmt job blocks on it while staying advisory for the ~1500 pre-existing diffs in the other 11 crates. Same incremental-adoption pattern as clippy - a crate becomes blocking the moment it is clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
64%, v1.0.0 gates 9/16.
THE BUG, because it is the important part
normalize("99999999999999999999") PANICKED: `n = n * 10 + d` overflows u64
past 19 digits. That is reachable from synthesize(text) with a phone number,
an ID, or anything an attacker types, and it has TWO faces:
* debug/test -> panic, i.e. denial of service in the embedding process;
* release -> SILENT WRAPAROUND, because this workspace deliberately
carries no overflow-checks (H-05 waived, R-005). The
speaker reads out a different number and nothing errors.
The same loop had a second defect: the digit-by-digit branch recovered digits
with n.to_string(), dropping leading zeros, so "0071234567" lost its zeros.
Both fixed by collecting the run as CHARACTERS and parsing only when the run
is short enough to need a number. Both inputs seeded as fuzz regressions.
Found by tests/miri_safe.rs on its first run. The proptest `.*` strategy in
tests/properties.rs never generated a 20-digit run - which is the argument for
having hand-picked adversarial cases ALONGSIDE generative ones, not instead.
H-23 MIRI CLOSED, scoped and verified
tests/miri_safe.rs is 4/4 green under -Zmiri-strict-provenance and BLOCKING in
CI. The scope is documented: Miri cannot execute x86 SIMD intrinsics (rustfft,
via mel::compute) or foreign functions (Win32 memory instrumentation, via
--lib). Both blockers were MEASURED - the earlier guess that SIMD would block
first was only half right, and the Win32 path is what actually stopped --lib.
H-19 CLOSED: both front ends are total, property-tested, fuzz-targeted, and two
real input-validation defects were found and fixed rather than assumed absent.
RELEASE PIPELINE (H-12, C-14, H-38 machinery)
.github/workflows/release.yml: a signed-tag gate that fails the release when
git cat-file shows no PGP/SSH signature, a cargo-auditable build so binaries
carry their own dependency manifest, CycloneDX 1.5 SBOMs per crate, SLSA build
provenance signed via GitHub OIDC, and checksec on the Linux binary.
CHANGELOG.md defines the Security-subsection convention, including the rule
that "no security-relevant changes" must be STATED - silence cannot
distinguish "nothing changed" from "nobody looked".
Those three gates stay INCOMPLETE on purpose: the repo has no tags and no
releases, so nothing has been signed, attested, or published. The machinery is
necessary and not sufficient. They flip on the first verified release.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lass
67%, v1.0.0 gates 9/16. H-17 closed by fixing what the fuzzing found rather
than by reading the code and declaring it fine.
STRUCTURE-AWARE FUZZING, because random bytes prove nothing here
Pure random input never survives the first length check, so it exercises the
framing and nothing behind it. tests/properties.rs now generates WELL-FRAMED
protobuf - arbitrary field numbers, wire types, nesting, and every truncation
of a valid message - which is what reaches the code doing arithmetic on
attacker-chosen values. The Reader itself held up (checked_add, guarded get,
a varint shift limit). Its CALLERS did not.
THREE BUGS, one class: unchecked arithmetic on values from the file
1. parse_tensor computed the element count as dims.iter().product() over
`i64 as usize`. A negative dim - ONNX's own -1 "dynamic" marker - became
colossal, and [1<<32, 1<<32] WRAPPED TO EXACTLY 0, matching an empty
payload. Debug panicked (denial of service on model load); release, which
carries no overflow-checks by our own decision, ACCEPTED the tensor. Every
downstream index into `data` then goes through `dims`.
Now usize::try_from + checked_mul.
2. The protobuf length was `u64 as usize`, which TRUNCATES on 32-bit. This
workspace ships ffai-wasm, so wasm32 is a real target: a declared length of
2^32 + 5 truncates to 5, passes the bounds check, and returns a slice that
is not the field. Now usize::try_from.
3. Attribute ints (kernel_shape, strides, pads, dilations, group) were cast
with `as usize`, turning a negative value into a geometry dimension. Now
falls back to the default.
All three are the SAME defect the normalize overflow was, and the same one the
reflect_pad panic was: arithmetic on a caller-supplied number with no bound.
That is now the thing to look for in the remaining crates.
6 regression seeds in fuzz/corpus/onnx_parse and normalize_text.
VERIFIED: 128 lib + 11 property + 4 miri_safe tests pass, miri_safe green under
-Zmiri-strict-provenance, cargo-careful green, clippy 0, invariant lint 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…e problem
69%, v1.0.0 gates 10/16.
THE "126 UNWRAP/EXPECT" FIGURE WAS MISLEADING
It counted test code and trusted-model paths. On the surface actually
reachable from transcribe(audio) or synthesize(text), non-test library code
contains ZERO bare .unwrap() and 18 .expect("...") - every one already
carrying a message stating its invariant. The crate was in far better shape
than the audit's own evidence claimed.
AUDITED, not assumed: docs/panic-audit.md records all 17 reachable sites with
the invariant that makes each unreachable, and how it is established:
* vad.rs `bridged` is seeded vec![regions[0]] behind an is_empty guard and
only pushed to;
* diarize.rs clusters start as (0..n).map(|i| vec![i]) - one member each -
and merging is remove(j) + extend into i, which never empties a survivor;
for n == 0 the map never runs;
* onnx.rs sites follow take(4)/take(8)/chunks_exact(4), which return exactly
that length or error;
* speaker.rs and vits.rs read model ARCHITECTURE constants, which R-001 puts
outside the threat model.
REINFORCED so the property cannot erode: tools/lint_invariants.py gains R5,
failing CI on any bare .unwrap() in library code. It locks in a discipline the
crate already followed rather than imposing a new one - the cost today is zero
and the cost of losing it silently is a denial of service in someone's process.
WORTH RECORDING: the real panics this audit found were ARITHMETIC, not unwraps -
reflect_pad on empty audio, normalize on a 20-digit run, and the ONNX dims
product. Auditing unwraps found nothing; auditing arithmetic found five bugs.
The next crate should start with the arithmetic.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…losed) 71%, v1.0.0 gates 11/16. H-41 CLOSED - all five residual risks now carry an owner, a dated acceptance, and a review date. Acceptance is not closure: R-002 (SendPtr disjointness proven only by argument) keeps TSan as its outstanding proof, R-003 keeps the arithmetic bug class as the thing to hunt first in the next crate, and the single waiver (H-05) names the condition that expires it early - model files ceasing to be trusted. STANDARD.md 14 asks for acceptance made WITH the measurements in front of the accepter, and R-005 carries the benchmark numbers that justify it. H-10 cargo-vet: ENFORCED, not finished, and the distinction matters cargo vet check now runs per-PR. Seven audit sources imported (mozilla, google, bytecode-alliance, embark-studios, zcash, isrg, fermyon), covering 98 fully + 3 partially audited of 494 third-party crates. The other 474 are EXEMPTIONS. An exemption is not a certification - it is a written acknowledgement that a crate is trusted only because it was already here. The gate asks for every dependency to be covered by an audit, so it stays OPEN. Reporting 474 unread crates as "vetted" would be precisely the checklist-that-records-intentions this whole process exists to prevent. What it buys today is the RATCHET: a new dependency cannot enter without an imported audit or a reviewed exemption. The gap is frozen while it is burned down. supply-chain/README.md carries the procedure, prioritises by blast radius (untrusted-input parsers, unsafe-heavy crates, proc macros) rather than alphabetically, and states the rule that matters - never certify a crate you have not read, because a false certification stops anyone else looking. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
74%, v1.0.0 gates 11/16.
THE FINDING
Diarizer holds an embedding cache - up to 512 SPEAKER EMBEDDINGS, keyed by a
hash of the raw audio window. Those vectors are voiceprints: GDPR Art 9
special-category biometric data by this crate's own data inventory.
The only function that claimed to purge them did not.
`clear_embed_cache_counters()` was documented "Drop every cached embedding"
and its body reset two atomic counters. The HashMap was never touched.
Nothing calls it yet, so this was a trap rather than a live bug - but it had
two faces, and both are the kind this codebase cares about:
* a harness calling it to time a COLD arm would have measured a WARM cache,
which is precisely the failure the hit/miss counters beside it exist to
prevent;
* anyone calling it to honour an erasure request would have believed a
deletion that never happened.
Fixed: Diarizer::clear_embed_cache() actually clears (and recovers a poisoned
lock rather than refusing - declining would strand the data we were asked to
erase), embed_cache_len() lets an auditor see what is retained, and the
misleading doc now says what the body does. Both halves regression-tested: the
counter test runs everywhere, the cache test skips where ECAPA weights are
absent.
RETENTION DOCUMENTED (C-06)
docs/data-inventory.md 3a now states what outlives a call: nothing reaches
disk, and exactly one structure retains personal data, with its bound (512,
~0.4 MB), its lifetime (the Diarizer instance) and its erasure path. Plus the
Art 17 note integrators need - erasing a recording does not erase a voiceprint
derived from it.
The wholesale-clear eviction policy was examined and deliberately LEFT ALONE:
the access pattern is a sliding window, so old entries are genuinely dead and
an LRU's bookkeeping would cost more than the misses it saves. That is
documented at the site and is a reasoned choice, not an oversight.
Verified: 130 lib + 11 property tests, cargo-careful green, clippy 0, lint 0.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…kspace
Went looking for a mutation score (H-29) and found a build break instead. The
score is still outstanding; this is the more useful half.
WHAT WAS BROKEN
ffai-bench's BenchConfig gained a `skip_references` field. Two examples were
never updated and have not compiled since:
crates/ffai-mercury/examples/bench_tts.rs
crates/ffai-carmenta/examples/bench_ocr.rs
Nothing noticed because `cargo test --lib` NEVER COMPILES AN EXAMPLE, and the
CI test job I wrote is `--workspace --lib`. The blocking clippy job is
`-p ffai-mercury --lib` for the same reason. An example that does not compile
is dead documentation - it is the first thing a new user runs.
Both set `skip_references: false`, which is the correct value rather than the
convenient one: bench_tts exists to run the frozen third-party `tts-judge`
that scores our audio, and skipping references there would turn the quality
gate into self-grading. bench_ocr baselines our engine against its references.
NOTE: this is a one-line compile fix in ffai-carmenta, NOT the deferred
arithmetic sweep of that crate. That stays deferred as agreed.
THE GATE THAT WOULD HAVE CAUGHT IT
`cargo check --workspace --all-targets` now runs in CI, blocking. The whole
workspace compiles under it today.
H-29 mutation testing: 229 mutants enumerated (normalize 12, mel 131, onnx 86),
no score yet. The run aborts at baseline on a cargo-mutants/cargo interaction -
in its COPIED tree aws_lc_sys, xet and safe_transmute do not resolve as rlibs.
The in-place mode surfaced the real build break above instead. Next step is
--in-place on Linux, or a pre-warmed target dir with --baseline skip.
Also worth recording: the first diagnosis was wrong. The errors initially looked
like E0133 unsafe_op_in_unsafe_fn and E0462/E0463 metadata failures; both were
artifacts of the in-place run polluting target/. `cargo clean -p ffai-mercury`
reduced 4 error classes to exactly one real one. Clean the tree before believing
a build error you did not expect.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…er crates
Asked whether aws-lc-sys could be swapped for a pure-Rust crypto library.
IT CANNOT, SAFELY, AND THE PREMISE NEEDED CORRECTING
`ring` is not the Rust version - it is C plus assembly, so that swap trades one
C crypto library for a smaller one. The only genuinely pure-Rust rustls
provider, rustls-rustcrypto, is explicitly not production-recommended, and
STANDARD.md 11 says prefer a well-audited library over a novel one. Moving the
TLS path that fetches model weights onto an unaudited provider is a downgrade
wearing the word "pure".
The feature graph forbids it regardless: reqwest's `rustls` feature hardcodes
__rustls-aws-lc-rs, and hf-hub only forwards that one. Adding rustls/ring
alongside does not REMOVE aws-lc - feature unification compiles both.
SO THE CRYPTO STAYED AND THE NEED FOR IT WENT
aws-lc-sys arrives via hf-hub -> reqwest -> rustls -> aws-lc-rs, i.e. through
the weight DOWNLOADER, which ffai-mercury does not use: it calls cache_dir,
load_dir and ModelManifest and performs no network I/O at all, exactly as
docs/data-inventory.md already claimed.
hf-hub was ALREADY optional in ffai-models behind `fetch`, and ffai-diana had
ALREADY opted out. Mercury simply inherited the default.
build deps aws-lc-sys
default (fetch) 320 present
--no-default-features 154 ABSENT
166 crates and the entire TLS stack, removable by a consumer who ships their
own weights.
DEFAULT DELIBERATELY UNCHANGED
The README documents `cargo add ffai-mercury` + WhisperCandle::new() fetching
on first use, so `default = ["fetch"]` keeps that promise. The default build
resolves the same 320 dependencies as before - same graph, same code, no
performance delta available to happen. Nothing on the inference path is
feature-gated; the removed subtree is the downloader, not compute.
MAINTENANCE TRAP THIS CREATES, recorded rather than left to be discovered
Cargo forbids `default-features = false` on an INHERITED workspace dependency,
so ffai-models is now declared by path in ffai-mercury, matching ffai-diana.
The version requirement therefore lives in two places and will not follow a
workspace bump automatically.
Verified: 130 lib + 11 property tests, workspace --all-targets clean, invariant
lint clean, cargo deny still "advisories ok, bans ok, licenses ok, sources ok",
and BOTH feature configurations compile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…cking Worked H-10, H-27, H-30 and H-15. None closes; each moved, and two real problems surfaced - including one I had introduced myself. CI WAS RED AND I HAD NOT NOTICED `cargo test --workspace --lib` - the job I added - fails on any checkout without corpora, because ffai-media's png oracle ASSERTS `dirs_seen > 0` instead of skipping. I only ever ran mercury's tests, so I never saw it. It now skips with a notice, matching how the tokenizer oracle already handles missing weights. Whole workspace suite: 10 test binaries, zero failures. I was careful about red-on-arrival for fmt and clippy and still shipped it for tests. Running the job you wrote is not optional. H-10 - imports are EXHAUSTED, and the real lever is measured All nine registry sources imported (added actix, ariel-os). They changed the counts by ZERO, and `cargo vet regenerate exemptions` - which recomputes the minimal set against everything visible - moved 474 to 473. The registry covers general infrastructure; this tree is ML/audio (candle, tokenizers, rustfft, hf-hub, xet) and nobody has audited it. Learned: an exemption SHADOWS an import, so importing alone can never move the number; `regenerate` is what re-asks. The lever that does work, measured: 135 of 430 exemptions - 31% - exist only because of the optional weight downloader. Removing a dependency audits it perfectly in zero time; supply-chain/README.md now ranks that above hand-auditing. H-27 - machinery landed, clock not started fuzz.yml runs all four targets nightly on Linux and CACHES THE CORPUS between runs, so coverage accumulates instead of restarting from the seeds every night - the difference between fuzzing and the appearance of it. Linux-only for a measured reason: cargo-fuzz cannot build these targets on Windows/MSVC because onig_sys - the C regex library `tokenizers` pulls for its `onig` feature - fails under the sanitizer flags. `tokenizers` offers a pure-Rust `fancy-regex` alternative that would remove that C dependency and unblock local fuzzing. NOT swapped: the tokenizer oracle SKIPS without cached whisper weights, so it would be unverified on the ASR correctness path. H-30 - three bounded Kani harnesses, honestly labelled src/proofs.rs (cfg(kani) only; normal build verified unaffected) pins the invariants that produced real defects: pad_or_trim_to's length contract, n_frames never claiming more samples than exist, and the dims fold never wrapping. Deliberately NOT aimed at the SIMD kernels - Kani cannot execute x86 intrinsics, and a proof that stops at the intrinsic boundary proves nothing. CI job is ADVISORY: Kani does not run on Windows, so these have never been executed, and claiming a proof nobody ran is worse than no proof. H-15 - one blocking crate became five core, models and argus were ALREADY at zero and cost nothing to enforce. ffai-media's "4 findings" were 3 in one file (bench merely inherits them): a manual % instead of is_multiple_of, a `let vidx = vidx;` self-rebind, and from_rff_frame marked dead-code-with-reason rather than deleted, since it carries the BT.601 colour contract the video path needs. Remaining: bench 8, diana 30, carmenta 34, cli 64. fmt now blocks for mercury AND media. Scope note: `cargo fmt -p ffai-media` reformatted that whole crate, not just the file I fixed - the same over-reach as the earlier mercury run. Banked rather than reverted, since it is what lets media join the fmt gate. Also: cfg(kani) is DECLARED in the manifest rather than silencing unexpected_cfgs, so the lint keeps catching genuine typos; and fuzz/target + fuzz/artifacts are gitignored - their .fingerprint paths exceed Windows' limit and made `git add -A` fail outright. Verified: 5 crates clippy-0, workspace tests 10/10, --all-targets clean, vet succeeds, deny fully green, invariant lint clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second pass at H-10/H-15/H-27/H-30. The big one is H-30.
H-30 - R-002 is no longer argued, it is proved
`unsafe impl Send`/`Sync` for SendPtr(*mut f32) is sound ONLY IF distinct
rayon tasks write disjoint regions. Since the audit began that has been a
COMMENT - the last claim in the crate resting on prose, accepted as R-002
with "TSan is the outstanding proof".
The index arithmetic is now extracted into decoder_kernels::task_region,
which the kernel itself calls, and two Kani harnesses prove over it:
* distinct tasks are separated on at least one axis (rows partition
[0,c_out), blocks partition [0,l_out), so they cannot overlap);
* every region is in bounds, and `bt = t1 - t0` never underflows.
Kani cannot execute the AVX2 kernel and does not need to: soundness depends
on WHERE a task may write, not on what the intrinsics compute. Five harnesses
now, covering the unsafe modules rather than only helpers.
Extraction verified behaviour-preserving: 130 lib tests including the kernel
oracles that compare against candle.
Still ADVISORY in CI - Kani does not run on Windows, so none has been
executed. A proof nobody has run is not evidence.
H-10 - 473 exemptions -> 331, fully audited 98 -> 242
`cargo vet suggest` recommends `cargo vet trust`, which I had overlooked
entirely. 128 trusted entries across 11 publishers - dtolnay 23, Manishearth
21, kennykerr 21, epage 18, seanmonstar 14, BurntSushi 12, cuviper 8,
Amanieu 5, sunfishcode 4, sfackler, alexcrichton - each trusted BECAUSE
other imported organisations already trust them, and each recorded with the
publisher's user-id and a validity window. Iterated to convergence; round 3
produced no new candidates.
Scale for the remaining 331: cargo vet puts the hand-audit backlog at
10,964,491 lines. Trust and dependency removal are the only real levers.
H-15 - one blocking crate became SIX
core, models, media, argus, mercury, bench. ffai-bench's last four fixed by
hand: a never-used `duration` kept with a reason, and two
`child.as_raw_handle() as *mut c_void` casts that were redundant because
as_raw_handle already returns that type.
Remaining: diana 30, carmenta 34, cli 64 (binary-only).
H-27 - 20,000-case soak, zero failures
All 11 property tests at PROPTEST_CASES=20000 in release: the structure-aware
protobuf generator, every truncation, and both front ends. Real evidence of
no shallow crashers. It is not coverage-guided fuzzing and does not start the
30-day clock, which still needs the nightly Linux job to run.
Verified: 6 crates clippy-0, workspace tests 10/10, --all-targets clean, vet
succeeds, deny fully green, invariant lint clean, properties 11/11, miri_safe
4/4.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
H-15 - THREE crates now under pedantic + nursery, not just clippy::all
ffai-core, ffai-argus, ffai-models, enforced by their own manifests.
ffai-core went 94 -> 0. `--fix` cleared 63 mechanical ones; the rest were
triaged rather than blanket-allowed, because unchecked arithmetic on outside
values is the bug class this audit found five of:
* eight numeric casts, allowed AT THE SITE with the range argument written
down - 4 PB of audio, a 16.7-million-pixel image, and `.max(0.0)` which
absorbs NaN before a cast that saturates rather than wrapping;
* one clippy::suspicious_operation_groupings that is a FALSE POSITIVE, and
following it would introduce a bug: a caption word belongs to the cue
containing its START, so `w.start >= seg.start && w.start < seg.end` is
the design. Clippy's suggested `w.end < seg.end` would drop or duplicate
words straddling a boundary. Allowed at the site, with the reason.
ffai-argus needed NO allows - it is clean under both groups outright.
MEASUREMENT CORRECTION, because the first numbers were wrong: passing
`-W clippy::pedantic` on the command line lints DEPENDENCIES too and
overrides their manifest allows, so "argus 23, models 29" were ffai-core's
findings counted three times. Filtering by path gives the real debt:
argus 0, models 6, media 65, bench 253, mercury 757.
H-30 - both Class B sites proved, and the two are not equally provable
decoder_kernels: `task_region` extracted, called by the kernel, proved
disjoint and in-bounds with no underflow. (Landed previously.)
flash_attn: added, and the difference matters. Its per-head slices OVERLAP
by construction - head 0 receives a slice covering the whole buffer - so
disjointness cannot come from the slice bounds. It comes from the addressing
scheme, and two harnesses prove `h*HD + row*width + c` is injective across
heads and stays inside `seq*width`.
SCOPE STATED: that assumes flash_head_strided writes only at those offsets.
Kani does not prove it; the existing matches_three_op_path oracle exercises
it element-for-element. Class A (target_feature) is not attempted - Kani
cannot execute x86 intrinsics, and a proof stopping at that boundary proves
nothing.
Seven harnesses total, still ADVISORY: none has been executed, because Kani
does not run on Windows.
Verified: 6 crates clippy-0 under their policies, workspace tests 10/10,
--all-targets clean, invariant lint clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
H-15: four crates now under pedantic + nursery (core, argus, models, media), plus bench and mercury at clippy::all. All six blocking. ffai-media went 159 -> 0, and the two findings came from REVIEWING its 21 numeric casts rather than allowing them wholesale: 1. load_wav shifted by `bits_per_sample - 1` taken straight from the WAV fmt chunk. 0 underflows the u16 subtraction; >= 65 overflows the shift. Debug panics; RELEASE - carrying no overflow-checks by our own decision - masks the shift and produces a silently wrong `scale`, quietly rescaling every sample in the file. hound validates the formats it supports, but resting our arithmetic on a dependency's validation is the same mistake as the model-cache assumption in R-001. Now range-checked to 1..=32. 2. THREE `vec![0u8; w * h * 3]` allocations sized by unchecked multiplication on decoded frame dimensions. On 64-bit that is merely an absurd allocation; on 32-bit - and ffai-wasm makes wasm32 a real target - it WRAPS to a small buffer and the row/column indexing that follows runs past it. Same defect class as the ONNX dims product. Now checked_mul with a clean error. That is defects seven and eight of this class. The pattern is now unambiguous: reviewing arithmetic finds bugs; reviewing style does not. It is also why the casts were annotated AT THEIR FUNCTIONS with the guard written down - clamp before every `as u8`, `.max(1.0)` absorbing NaN before a saturating cast - and never blanket-allowed at crate level. Also fixed: two annotations that had landed on the wrong functions (the pts cast lives in `impl Iterator for VideoStream::next`, not `stream_frames`), caught because the lint did not go quiet. Verified: 6 crates clippy-0 under their own policies, workspace tests 10/10, --all-targets clean, invariant lint clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Defect nine of the unchecked-arithmetic class, found by reviewing mercury's
cast lints under clippy::pedantic rather than allowing them.
THE BUG
`let field = (tag >> 3) as u32;` in the protobuf reader. Field numbers are
u64 varints, so a large one NARROWS instead of being rejected - and it
narrows onto a field this parser handles. A tag whose field number is
2^32 + 1 becomes 1, which parse_tensor reads as `dims`. Crafted input could
therefore reach a handler it had no right to reach.
Protobuf caps field numbers at 2^29 - 1, so anything above that is malformed
by definition. Now rejected with an error rather than reinterpreted, and the
one remaining narrowing cast is bounded by that check and annotated as such.
Regression test pins the exact aliasing pair (2^32, 2^32+1, u64::MAX>>3,
2^29) and confirms the largest LEGAL field number still parses. Fuzz seed
added. 12 property tests now.
NOT bugs, checked and left alone:
* `push_ints` casting u64 -> i64 is CORRECT protobuf: negative int64 is
encoded as a two's-complement 10-byte varint, so `as i64` is the specified
decoding, not a wrap.
* normalize's `n as usize` is bounded by its own `match n { 0..=19 => .. }`.
H-15 progress: mercury 757 -> 395 pedantic findings after --fix (130 tests
still green); bench 253 categorised - it is dominated by benchmark statistics
casts (counts -> f64), which are inherent to a measurement crate and lower
value than mercury's parser casts.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Five crates now under the strictest tier: core, argus, models, media, mercury.
bench remains at clippy::all. All six blocking.
THE CASTS WERE SPLIT BY TRUST BOUNDARY, NOT BLANKET-ALLOWED
169 cast lints. Allowing them wholesale would have been one line and would
have thrown away the gate that has produced most of this audit's findings, so
instead:
* cast_possible_truncation / cast_sign_loss / cast_possible_wrap are
module-allowed ONLY in the model-internal DSP modules - each carrying a
note explaining that the values are dimensions and indices bounded by
already-validated weights;
* they stay DENIED in mel, fbank, onnx, normalize, lexicon, chunk,
phonemize, phoneme_ids - the untrusted surface, and where every
arithmetic defect in this audit actually lived;
* cast_precision_loss is allowed crate-wide: it ROUNDS. It cannot alias one
value onto another, which is the property that makes truncation and wrap
dangerous.
Only 23 of the 169 were on the untrusted surface, and each was read
individually: filterbank bin indices (exact at those magnitudes), a
binary_search index into a 39-entry const table, a percentile double-guarded
by an is_empty return AND a .min() clamp, and protobuf u64->i64 which is the
SPECIFIED decoding rather than a wrap.
GENUINE FINDINGS, triaged rather than silenced
* FIXED: `.ends_with(".onnx")` was case-sensitive, so a voice pack shipping
`model.ONNX` was invisible to the loader. Now an extension comparison that
ignores ASCII case.
* NOT a bug: cast_ptr_alignment on _mm_loadu_si128 / _mm256_loadu_si256 - the
`u` is the whole point, those intrinsics are the UNALIGNED loads. clippy
cannot see the consumer.
* NOT a bug: `while point < total_secs - 1e-6` - the epsilon IS the guard
against float accumulation; a bare `<` is what would be wrong.
* NOT a bug: phoneme match arms sharing a body - different graphemes map to
the same phoneme, and merging them would hide which letters are covered.
* NOT a bug: a speaker-table lock held across clustering - the table must not
change under it, which is the point of persisting speakers between calls.
Verified after the refactors: 130 lib + 12 property + 4 miri_safe tests, Miri
green under -Zmiri-strict-provenance, cargo-careful green, workspace tests
10/10, --all-targets clean, invariant lint clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
core 94, argus 96, models 108, media 159, bench 253, mercury 757 - all to zero,
each enforced by its own manifest and blocking in CI.
TWO MORE DEFECTS, from reading the casts rather than allowing them
* `resample` divided by an unvalidated sample rate. src_rate == 0 makes ratio
0, len/0 infinity, and `floor() as usize` SATURATES to usize::MAX - which
became Vec::with_capacity(usize::MAX) and aborted the process. Rates come
from a file header, so zero is reachable. Guarded, with a regression test.
* `j.assign(&child)` dropped its return value. That job object is the only
reason the footprint measurement is trustworthy: without it the harness
measured the Python launcher rather than the process doing the work and
reported 5 MiB for a reference that loads a 77.7 MB model - a 127x ratio,
impossible and plausible-looking in a table. A silent failure reverts to
exactly that. Now loud.
THE CAST POLICY IS WHY THESE WERE FOUND
truncation / sign-loss / wrap stay DENIED on the untrusted surface (mel,
fbank, onnx, normalize, lexicon, chunk, phonemize, phoneme_ids) and are
module-allowed only in model-internal DSP, each with a written reason.
cast_precision_loss is allowed where it appears because it ROUNDS - it cannot
alias one value onto another, which is the property that makes the other
three dangerous. Four defects came out of that policy across this pass and
the last.
TOOLING NOTE worth keeping: `cargo clippy --fix` applied NOTHING to ffai-bench,
because one of clippy's own suggestions produced `error[E0603]: module 'imp' is
private` and a single bad suggestion rolls back the entire crate. Applying
lint-by-lint cleared 103. A fixer that silently does nothing looks exactly like
a crate that was already clean.
Verified: 6 crates at zero under pedantic+nursery, workspace tests 10/10,
--all-targets clean, invariant lint clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
core 94, argus 96, models 108, media 159, bench 253, mercury 757, cli 38, demo 22, py 10, wasm 5 - all to zero, each enforced by its own manifest and blocking in CI. The remaining two, ffai-carmenta and ffai-diana, are DELIBERATELY DEFERRED at the user's instruction. They are not unaddressed; they are out of scope by decision, and the gate is built so that stays true without reddening CI. THE CAST POLICY IS THE POINT Truncation, sign loss and wrap stay DENIED on the untrusted surface, and are allowed only where a guard is named in writing at the site: `.clamp()` immediately before the cast, an `is_finite()` test, a bounded `match` arm, or an index into a const table. Precision loss is allowed because it ROUNDS - it cannot alias one value onto another, which is the property that makes the other three dangerous. That policy produced five findings across this pass and the previous two: ONNX field-number aliasing, the WAV bits_per_sample shift, three unchecked frame allocations, a zero-sample-rate divide that saturated into Vec::with_capacity(usize::MAX), and a dropped job-assignment result that silently invalidated footprint measurements. CI CORRECTNESS The clippy job now uses --no-deps. Without it, `-D warnings` promotes warnings in carmenta and diana - crates the job is not checking - and the gate goes red on deferred code. I found that by running the exact command CI would run, which is the habit this audit had to learn twice. Also recorded in the workflow: do NOT measure a single crate with `-W clippy::pedantic` on the command line. It re-lints dependencies and overrides their manifest allows, so a clean crate reports its dependencies' findings as its own. That artifact wasted a measurement earlier in this pass. Verified with the exact CI commands: both clippy steps 0, workspace tests 10/10, --all-targets clean, invariant lint clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…veat stated
carmenta 594 -> 0 and diana 494 -> 0 under pedantic + nursery, both now
blocking in CI. That is the headline and it is only half true, so the
caveat is written into both Cargo.toml files rather than left implicit:
These two allow cast_possible_truncation / cast_sign_loss /
cast_possible_wrap WHOLESALE - 149 and 38 sites, unreviewed. The other
ten crates cleared those casts one at a time against the trust
boundary, and that review found four real defects. carmenta and diana
parse images and documents, i.e. MORE untrusted input than mercury.
So the gate is green on the stylistic tiers, not on the parse surface.
Logged as R-006 (open, not mitigated) so a green tick cannot be read as
a clean bill of health. This is R-003's bug class, unswept, deferred by
decision - not overlooked.
Seven real findings fixed on the way through:
- two `&mut Vec<T>` params that only index and sort -> `&mut [T]`
- a doc line starting with `+`, which Markdown renders as a list
- a dead constant, an unread struct field, a dead accumulator
- a bench `if` whose two branches were byte-identical
Two read and deliberately KEPT, now with a comment saying why, so the
next reader does not "fix" them:
- diana ab.rs: the null arm calls run(false) on BOTH sides because a
null arm measures a configuration against itself. The parity swap
has nothing to swap; collapsed the expression, kept the semantics.
- diana geom_ab.rs: the two branches share a first line while running
the arms in OPPOSITE order. That is the ABBA interleave working.
One self-inflicted bug caught before it landed: dropping the
`PixelFormat` import that clippy called unused broke the test build,
because the test module reaches it through `use super::*` - which
unused_imports does not model. Imported at the point of use instead.
CI checks these two with --all-targets, not --lib: their examples are
where the never-loop and uninit-vec findings actually were.
Tests unchanged: carmenta 25 passed, diana 74 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Extended Kani coverage from two unsafe modules to four (asr::vocab_int8,
asr::f16_gemv). Writing the vocab_int8 harness surfaced a real defect.
`dot_i8_blocked` steps its inner loop with
while o < blk { ...loadu_si256(base + o)...; o += 32 }
The test is `o < blk` but each step READS 32 BYTES. Any blk that is not a
multiple of 32 reads past the block, and on the final block past the end
of the allocation - inside `unsafe`, via `_mm256_loadu_si256` and
`bscale.get_unchecked`.
`Int8Vocab::new` guarded `d % blk == 0` and NOT `blk % 32 == 0`, with a
comment directly above it saying "the kernel steps 32 lanes at a time".
Not currently reachable: the FFAI_VOCAB_BLK env path validates
`n >= 32 && n % 32 == 0`. But `Num::set` stores without validating, and
`get_usize` maps a negative override to 0, which turns `d % blk` into a
divide-by-zero panic. A memory-safety invariant was resting on a
validator in a different module. It is now checked where it is
load-bearing, falling back to the safe non-SIMD path.
Also fixed the reason this file could rot unnoticed: `mod proofs` was
`cfg(kani)`, so on any machine without Kani - every box here - it never
even type-checked. Now `cfg(any(kani, test))`, with harness bodies still
Kani-only and the pure index helpers compiling under `cargo test`.
And since Kani needs Linux, the proofs now carry evidence that runs
today: `proofs::exhaustive` enumerates the SAME domain each harness
assumes. At these bounds exhaustion and symbolic proof state the same
thing. Six tests, all green, including a NEGATIVE one asserting an
unaligned blk really does over-read - so the new guard cannot be removed
as redundant without a test failing.
R-002 (flash-attn band disjointness, "argued in a comment, not proven")
gets its first executed check as part of this.
mercury: 136 tests pass, clippy clean, invariants clean.
Kani itself stays advisory pending a Linux runner.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tion tests
Two findings, the second much larger than the first.
1. The 17 regression seeds were replayed ONLY by the nightly Linux
`cargo fuzz` job. A change reintroducing a fixed crash could sit green
in PR CI for up to a day, and could not be reproduced locally at all:
`cargo fuzz build` fails on Windows because onig_sys will not compile
under the sanitizer flags.
tests/fuzz_corpus_replay.rs feeds every corpus file to the SAME entry
point its fuzz target uses, with the same assertions - mel's shape
contract, normalize's determinism. ~1 ms, no nightly, every platform.
It explores nothing new; it is a regression net, not fuzzing.
2. Adding it revealed the test job never ran integration tests AT ALL.
# Lib + integration tests. <- the comment
cargo test --workspace --lib <- the command
`--lib` runs neither integration nor doc tests. properties.rs,
miri_safe.rs, every oracle suite, and the new replay had never run in
CI once.
The cause was two diana oracle tests asserting `ran > 0`, which is red
on any machine without model fixtures. Narrowing CI to `--lib` silenced
those two and took every other integration test with them.
Both now separate "nothing to verify" from "verification failed": skip
cleanly when fixtures are absent, fail loudly under
FFAI_REQUIRE_ORACLES=1. CI sets that variable whenever checkpoints are
present, so a vacuous pass is still caught - the original intent is
kept, just aimed at the right condition.
before cargo test --workspace --lib lib subset only
after cargo test --workspace --lib --tests
368 tests across 22 binaries, all green
Also fixed a redundant `return` and a `pub(crate)` that clippy flags once
proofs.rs compiles under cfg(test).
The 30-day continuous-fuzzing clock still has to elapse on the nightly
job. That part is wall-time, not work.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
R-002 (flash-attn band disjointness) was "argued in a comment, not proven". It now has an exhaustive disjointness check that runs on every platform - bounded, not general, and the row says so. TSan at full width is still the outstanding proof. R-004 narrows a second time: clippy blocks for 12/12 crates and the test job actually runs integration tests now. Remaining exposure is fmt coverage and branch protection. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"Needs a Linux runner" was an assumption I never checked. WSL2 Ubuntu has
been on this machine the whole time - 24 cores, 15 GB. Kani 0.67.0
installed there verified every harness in about 0.1 s each:
Complete - 13 successfully verified harnesses, 0 failures, 13 total.
The harnesses stop being prose. CI's kani job loses continue-on-error and
is now BLOCKING.
Two things the run cost, both written into the workflow so nobody pays
them twice:
--ignore-global-asm is REQUIRED. The `pulp` SIMD crate (via gemm) has
global asm that Kani will not translate, and it aborts rather than skip.
pulp is a dependency, not code under proof - every harness reasons about
pure integer index arithmetic - but Kani's "verification results may be
impacted" caveat is recorded, not hidden.
The MSRV is load-bearing, not a formality. Kani bundles rustc
1.93-nightly; `asr::vad` calls f32::mul_add inside a `const fn`, which
stabilised in 1.95. Hence `#![cfg_attr(kani, feature(const_mul_add))]`
in lib.rs - an attribute that does not exist on a normal build. My first
guess, that rust-version was "just a manifest declaration", was wrong.
Scope is unchanged and still stated plainly: Class A (target_feature) is
not attempted, because Kani cannot execute x86 intrinsics. What is proved
is the index and length arithmetic around them - which is where every
defect this audit has found actually lived, including the vocab_int8
over-read fixed two commits ago.
Mercury: 76% -> 79%, 33 of 42 gates Completed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…o quote `cargo vet suggest` reports ~5.6M lines and I twice repeated that as though it described uniform work. It does not. 5 largest crates 8,414,399 lines 77% of the backlog 85 crates <=2000 ln 78,099 lines 0.7% median crate 4,110 lines The backlog is five boulders and a long tail of pebbles. That makes LINE COUNT the wrong planning metric, because H-10 measures exemption COVERAGE: auditing the 85 small crates clears 27% of the 313 exemptions for 0.7% of the reading. Best ratio available, blocked on nothing. Four of the five boulders - aws-lc-sys (2.11M), ring, and both cudarc versions - are optional-feature paths no shipped binary compiles. Auditing them spends effort on code the product does not run. Removing the need, which the `fetch` change already did once, beats auditing them. Also flagged, deliberately NOT acted on: ffai-argus, ffai-models, ffai-wasm and the sibling rff-*/rusty_* crates are in the backlog only because `audit-as-crates-io = true` treats first-party code as third-party. Setting it false would delete them at a stroke, but it may be deliberate - it catches a published version drifting from local source. That is the maintainer's call, and flipping a security setting to improve a number is the wrong trade. cargo vet prune: run, removed nothing. The exemption set is minimal. What is left is human attestation. `cargo vet certify` records a named person having read the code, so it is not bulk-applied here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The gate reads "every unsafe module has at least one harness", so the Completed status has to say which ones and why the rest are out. Four modules proved: decoder_kernels and flash_attn (Class B, raw-pointer sharing across rayon tasks), vocab_int8 and f16_gemv (Class A - the index arithmetic around the intrinsics, which is the provable part). Two excluded: asr::model and asr::aligner are one `VarBuilder::from_mmaped_safetensors` call each. The obligation there is environmental - no other process may mutate the cache blob while it is mapped - not arithmetic. There is no safe abstraction for Kani to check and a harness would be theatre. Already carried as R-001, accepted by product decision, with UNSAFE.md Class C stating the rule. 4/6 proved, 2/6 excluded with a written reason and an owned risk. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same mistake as H-30, same cause. `cargo fuzz` was never blocked on CI: it builds and runs fine on the WSL2 Ubuntu that has been on this machine the whole time. It is WINDOWS that cannot build it - onig_sys fails under the sanitizer flags - and I had generalised that into "needs a Linux runner". Every one of these four targets had shipped unexecuted. All four ran concurrently on 24 cores for 30 minutes: ZERO crashes, zero artifacts. onnx_parse 17.9M executions normalize_text 6.2M lexicon_parse 3.0M mel_compute 748k Those are the last DIRECTLY OBSERVED counts, not the finals - a WSL restart took the logs. I would rather report the number I watched than a larger one I inferred. Corpus 17 -> 1,168 files, after `cargo fuzz cmin` reduced 1,520 raw to a coverage-preserving set (4.7 MB, largest input 3,843 B). Committed as a durable baseline: the nightly job's cache is evictable, and a cold start from 17 seeds burns hours of the 30-day budget rediscovering shapes we already have. The replay test added earlier now exercises all 1,168 in 0.18 s, so everything fuzzing found interesting is a permanent regression case on every platform - including the Windows boxes that cannot fuzz at all. STILL INCOMPLETE, and correctly so: the criterion is ">=30 days of coverage-guided fuzzing (or equivalent)". 30 minutes is not 30 days. What changed is that the clock now runs on targets known to execute, from a corpus worth starting from. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The 1168 corpus files committed a moment ago are BYTES. With core.autocrlf=true - the default on this Windows box - git treats an input containing LFs and no NUL as text and rewrites line endings ON CHECKOUT. That corruption is quiet and nasty. libFuzzer names each input after the SHA-1 of its content, so a rewritten file stops matching its own name; the regression replay no longer tests what it was seeded with; and a coverage-guided run restarts from inputs that no longer reach the paths they were minimised to reach. Verified the stored blobs are currently intact - the committed blob's sha1sum still equals its filename. This keeps them that way through every future clone. Also covers .bin/.npy/.safetensors oracle fixtures for the same reason. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Third gate this session written off as "needs a Linux runner" that ran locally in minutes once I checked. The row literally read "never run". TSan 136 tests, 0 WARNING: ThreadSanitizer ASan 136 tests, 0 ERROR: AddressSanitizer TSan is the one that mattered. It exercises BOTH Class B unsafe sites - decoder_kernels::kernels_match_candle_on_every_decoder_shape and transpose_kernel_matches_candle_on_the_three_upsamplers drive the rayon SendPtr path, flash_attn::matches_three_op_path drives the strided heads. That is precisely the claim R-002 records as "argued in a comment, not proven". R-002's two named outstanding items are now both done: the Kani harnesses were executed (not merely written), and TSan has watched the real threads. Not closed, and the row says why - TSan only sees the interleavings that occurred, and Kani proves the addressing scheme rather than that flash_head_strided writes only at those offsets. The three-op oracle checks that element-for-element without proving it. MSan deliberately NOT run, and the row says so rather than implying full sanitizer coverage: it needs every dependency instrumented including C/C++, and this graph carries onig_sys, aws-lc-sys and BLAS-family code that is not. Uninstrumented MSan produces false positives, not findings. Wired into the nightly workflow, not the PR gate: -Zbuild-std rebuilds the standard library under instrumentation, which would dominate a PR for a check that has yet to find a regression. Mercury: 79% -> 81%, 34 of 42 gates Completed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cargo fuzz cmin` minimises for COVERAGE. It keeps a smallest set of inputs
reaching the same edges and deletes the rest. That is the right objective
for fuzzing throughput and exactly the wrong one for regressions, because a
regression seed's value is not the coverage it adds - it is that this exact
input once crashed us.
I ran cmin and committed the result as if it were a superset. It was not.
Twelve named seeds went with it:
mel_compute/empty_REGRESSION_oob_panic
normalize_text/REGRESSION_20_digit_overflow, REGRESSION_leading_zeros
onnx_parse/REGRESSION_dims_overflow, REGRESSION_dims_product_wraps,
REGRESSION_field_number_aliasing
...and six descriptive seeds (nan_inf, one_sample, latin1_and_variants, ...)
Those six REGRESSION files are the audit's own findings - the reflect_pad
empty-input panic, the u64 digit overflow, the ONNX dims wrap and field
aliasing. After cmin the corpus contained NO empty input at all, so the
defect that motivated the mel fuzz target had quietly stopped being tested.
Restored all twelve. The corpus is the UNION of the minimised set and the
named seeds, never the minimised set alone.
Added `named_regression_seeds_survive_minimisation`, which fails loudly if
any of the six REGRESSION seeds goes missing and separately asserts that
the empty-input seed is still zero bytes - the filename is not the
property, the emptiness is.
This is the same class of silent degradation this audit keeps finding in
other people's work. Worth writing down that I introduced one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…he goal The old note blamed "a copied tree cannot resolve aws_lc_sys/xet/ safe_transmute as rlibs". That was a guess and it was wrong. Real cause: cargo-mutants copies the tree AND the target dir into $TMPDIR, and /tmp here is a 7.8 GB tmpfs. It fills, and the LINKER dies with `ld terminated with signal 7 [Bus error]` - which surfaces as a compile failure and reads like a dependency problem. With TMPDIR on real disk the baseline passes and it enumerates 94 mutants in tts/onnx.rs, the highest-value untrusted parser. Stopping there, deliberately. --in-place (needed to avoid the copy) forces serial execution and each mutant rebuilds mercury plus its dependents: one baseline in 32 minutes. The WSL VM also wedges under sustained load and needs a shutdown to recover, which took three attempts to characterise. H-29 is not a v1.0.0-blocking gate and is not in the active objective (H-10, H-15, H-27, H-30). Recording the diagnosis is worth more than burning further budget on it, and the row now carries the shortest path for whoever picks it up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
H-27 asks for ">=30 days of coverage-guided fuzzing (or equivalent)", and
this workflow is what defines the number, so the arithmetic belongs next
to it:
10 minutes x 4 targets x 30 nights = 1,200 target-minutes = 20 core-hours
That makes "(or equivalent)" measurable instead of a matter of opinion: a
box with spare cores can deliver the same libFuzzer compute in an
afternoon.
The comment also states what equivalence does NOT buy, because the
distinction is easy to lose: 30 nights fuzz 30 successive revisions of the
code, and concentrated compute cannot reproduce that. Equivalent compute
is a floor on effort, not a substitute for continuous coverage. The
schedule stays on for exactly that reason.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…budget
The blocker was my own reading. The criterion is ">=30 days of
coverage-guided fuzzing (OR EQUIVALENT)", and I kept treating "30 days" as
wall-clock I could not manufacture while skipping the clause that makes it
tractable.
Made the equivalence arithmetic instead of opinion. The nightly schedule
this project actually runs is:
10 min x 4 targets x 30 nights = 1,200 target-minutes = 20 core-hours
Delivered 12 concurrent workers x 100 minutes = 1,200 target-minutes, plus
the earlier session's 120 = 1,320, i.e. 110% of the 30-night budget, in one
afternoon on 24 cores.
602,793,111 executions. ZERO crashes. All four targets rc=0.
onnx_parse 322,498,053
normalize_text 142,896,239
lexicon_parse 118,560,521
mel_compute 18,838,298
Corpus 17 -> 1,407 inputs (3.3 MB), replayed in 3.4 s on every platform and
every PR. Copied back as a UNION this time - no cmin, after it silently ate
all twelve named regression seeds last round. All six REGRESSION seeds and
the empty-input seed verified present before and after.
What this is NOT, and the row says so plainly: 30 nights also fuzz 30
SUCCESSIVE REVISIONS of the code, which concentrated compute cannot
reproduce. Equivalent compute is a floor on effort, not a substitute for
continuous coverage - which is why the nightly schedule stays enabled
rather than being treated as discharged.
The other two limbs stand on their own: zero open crashers, and every past
crasher really is a regression test - true only after the cmin bug was
caught and guarded.
Mercury: 81% -> 83%, 35/42 gates, 13/16 v1.0.0-blocking.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… there
The pass criterion is "every dependency covered by an imported or LOCAL
AUDIT CERTIFICATION". An exemption is by definition not one, so closing
this gate means certifying the remainder.
`cargo vet certify` records a NAMED PERSON having read the code, and those
records get published for other organisations to import. Signing 291 of
them on the maintainer's behalf would manufacture precisely the false
assurance this audit has spent the session removing - the same failure as
a green clippy tick over unreviewed casts, or a Kani harness nobody ran.
So this gate is not closable by an agent, and that is not a tooling limit.
Every MECHANICAL lever is exhausted and verified, not assumed:
- all 9 cargo-vet registry peers imported (the registry has exactly 9)
- publisher trust iterated to convergence: 128 -> 167, next round empty
- cargo vet prune: run, removed nothing
New: supply-chain/WORKLIST.md sizes what is left, cheapest first.
79 are version DIFFS, not full reads - an audit already exists for a
nearby version and only the delta needs reading. The cheapest ten
total 192 changed lines across 25 files. Several are two-line bumps.
232 are full reads, but the smallest 40 are a few thousand lines; the
cost is five giants at 77% of the backlog, four of them
optional-feature paths nothing ships.
26 are the ORG'S OWN CODE - 10 FFai workspace members and 16 sibling
rff-*/rusty_* crates - in the list only because
`audit-as-crates-io = true` treats first-party code as third-party.
A policy decision worth a minute, not an audit.
Two honest routes remain: work the worklist, or WAIVE under H-41 with an
owner and an expiry - the framework's own mechanism for a gate whose cost
is not currently worth its risk. Leaving it Incomplete and unexplained is
the only wrong answer, and it no longer is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Chased the largest single item in the H-10 backlog to a definite answer
rather than leaving it as "we could maybe swap it".
aws-lc-sys is 2,108,351 lines - 39% of the entire audit backlog in ONE
crate, C and assembly. Removing it beats reviewing it, which is the same
move that took the graph 320 -> 154 when `fetch` was gated.
It cannot be removed from this repo, and the reason is precise:
rustls 0.23 pluggable provider, aws-lc-rs OR ring fine
reqwest 0.13.4 HAS `rustls-no-provider` hook exists
hf-hub 1.0.0 only `rustls-tls = ["reqwest/rustls"]`,
and reqwest/rustls = __rustls-aws-lc-rs THE BLOCKER
Cargo features are additive. Once hf-hub enables reqwest/rustls, nothing
downstream can un-enable the provider it drags in; adding rustls with the
`ring` feature here ADDS ring alongside aws-lc-rs rather than replacing it.
The fix is one line upstream in hf-hub:
rustls-tls-no-provider = ["reqwest/rustls-no-provider"]
With that, this repo switches over and 39% of the backlog disappears
without a line being read - and ring is already in the all-features graph
via ureq -> ocipkg -> intel-mkl-src, so it costs no new dependency.
Recorded in WORKLIST.md so the search is not repeated. aws-lc-sys is
fetch-only in the meantime: it inflates the audit number far more than the
shipped risk.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… of the risk
Project lead's call, and a defensible one. Recorded properly rather than
left as an unexplained exemption.
aws-lc-sys is 2,108,351 lines of C and assembly - 39% of the entire H-10
audit backlog in one crate - reached seven levels down:
ffai-models -> hf-hub -> reqwest -> hyper-rustls -> rustls
-> aws-lc-rs -> aws-lc-sys
Three reasons it is the right one to pass on, in order of weight:
- `fetch`-only. It is absent from a default build, so it is not in the
shipped surface at all; it inflates the audit number far more than the
risk.
- It is AWS-LC: professionally maintained, FIPS-validated, with its own
external audit regime. A hand read here would not improve on that.
- Removal is blocked UPSTREAM, not by effort. hf-hub forwards only
`reqwest/rustls`, which forces the aws-lc-rs provider, and Cargo
features are additive so nothing downstream can undo it.
Written in three places so it cannot be mistaken for an oversight: R-007 in
the audit plan with an owner and a review date, a waiver row scoped
explicitly to this crate, and a comment at the exemption itself in
config.toml where someone would actually meet it.
SCOPE: this crate only. The other 312 exemptions are NOT waived and stay on
the worklist. Reopens on any RUSTSEC advisory (cargo audit runs in CI), if
`fetch` becomes a default feature, or when the one-line hf-hub PR lands.
Effect: reading backlog ~5.6M -> ~3.5M lines. The exemption COUNT is
unchanged at 313, because the gate counts crates and not lines - so this
does not close H-10 and is not recorded as if it did.
cargo vet check still passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ated
Decision taken by the project lead after the mechanical work was exhausted
and re-verified:
all 9 cargo-vet registry peers imported (the registry holds exactly 9)
publisher trust iterated to convergence: 128 -> 167, next round empty
cargo vet prune removed nothing
cargo vet regenerate imports fetched nothing new
What remained was human attestation. `cargo vet certify` publishes "a named
person read this code" for other organisations to import, so it is not
something to automate, and the lead has accepted the residual instead of
spending the remaining budget there.
STATUS STAYS `Incomplete`, deliberately. This document's own rule is that a
waiver records an accepted risk and does not make a gate pass - exactly how
H-05 is treated. Marking it Completed would be the same fabrication this
audit spent the session removing, and it is not what waiving means.
What actually changed: the gap is now owned, dated, quantified, and paired
with NAMED compensating controls rather than sitting open and unexplained.
Recorded as R-008, with the controls stated so nobody reads the waiver as
"covered":
- cargo audit vs RUSTSEC on every PR - the control that actually catches
a malicious or vulnerable dependency, which hand-reading 313 crates
once would not
- deny.toml bans unknown registries and git sources
- Cargo.lock committed, so versions cannot drift underneath the review
- 260 dependencies ARE fully audited via imported peer audits
- cargo vet check passes, so the exempt set is explicit and reviewable
aws-lc-sys keeps its own narrower waiver and reasoning as R-007.
Expires 2026-11-15, or immediately on any RUSTSEC advisory affecting an
exempted crate. The cheapest-first path back is in supply-chain/WORKLIST.md.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The plan said Scheduled requires an owner and a date and that "neither is the auditor's to assign", which is why that table held only proposals. The project lead assigned both on 2026-08-18 by waiving H-10 in their own name with a 2026-11-15 expiry. That is precisely what turns a proposal into a schedule, so the row moves. Still NOT Completed, and that is not a technicality. The gate wants every dependency covered by a certification; 313 are exempted instead. A waiver records an accepted risk, it does not make a gate pass - the same treatment H-05 already has. The percentage is unchanged at 83% because the renderer computes it from Completed alone, which is the correct behaviour: an owned plan is not a passed gate. The README's "Next up" line now reads H-10 with the owner and date, which is the one place a reader looks to answer "what now". Also rewrote the Scheduled work section, which had gone stale and was actively misleading: all six of the original cheapest-first proposals are done (SECURITY.md, the overflow-checks decision, deny.toml + cargo audit, harden.yml, pedantic+nursery, UNSAFE.md). Listing finished work as pending is the same class of rot this audit keeps finding. Replaced with the one real schedule item, a short list of single actions waiting on the project lead rather than on work, and a record of what was delivered. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…udited I was wrong to say no honest certification path existed for an agent, and the correction is worth more than the seven crates. `cargo vet certify` takes `--who` (names the actual auditor) and `--accept-all` (a documented non-interactive path). Together they mean a review genuinely performed can be recorded ACCURATELY rather than signed in someone else's name. My objection was never to automation - it was to misattribution, and --who removes it. Seven crates certified, each with its COMPLETE diff read, each touching NO source file at all: futures-core, futures-macro, futures-sink 0.3.32 -> 0.3.33 futures-io 0.3.31 -> 0.3.33 quinn-udp 0.5.14 -> 0.5.15 zerofrom 0.1.7 -> 0.1.8 crypto-common 0.1.6 -> 0.1.7 Every entry records `who` as an AI agent, explicitly "not independently human-verified", with notes stating exactly what was inspected. Anyone importing these audits can weigh them accordingly - which is the whole point of the field. The ceiling is measured, not asserted: all 75 diff candidates were swept against the same rule and 73 TOUCH SOURCE. Seven is what unambiguous review yields, not where I got bored. Where the line sits: stable_deref_trait 1.2.0 -> 1.2.1 adds `unsafe impl StableDeref for Cow<'a, T>`. The reasoning is not hard - Cow's deref targets are heap or borrowed data whose addresses survive moving the enum, exactly as the existing String/Vec impls do - but an `unsafe impl` on a pointer-stability trait is a person's signature, not an agent's. All 73 are itemised smallest-first in supply-chain/WORKLIST.md Part 5. The first dozen touch a single source file each, so the next hour of human time is now a concrete list rather than a number. cargo vet check passes. The waiver stays as the backstop for what remains. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
rand_core 0.10.0 -> 0.10.1: complete diff read. The only source change is TWO RUSTDOC LINKS retargeted from rand::Rng to rand::RngExt. Everything else is a CHANGELOG entry and a version bump. No executable code, no unsafe, no new imports. That widens the rule slightly and honestly: the criteria require reasoning about unsafe blocks and powerful imports in the delta, not a full logic review. A doc-only source change clears that bar as cleanly as a manifest-only one. Recorded two crates I read and deliberately did NOT certify, so the line is visible rather than implied: find-msvc-tools 0.1.8 -> 0.1.9 adds a `find_windows_sdk` API that enumerates SDK directories. Ambient filesystem capability is precisely what the criteria single out for careful reasoning. stable_deref_trait 1.2.0 -> 1.2.1 adds `unsafe impl StableDeref for Cow<'a, T>`. The argument is straightforward - Cow derefs to heap or borrowed data whose address survives moving the enum, as the existing String/Vec impls do - but an `unsafe impl` on a pointer-stability trait is a person's signature, not an agent's. 268 dependencies now fully audited, 305 exempted. cargo vet check passes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s measured version_check 0.9.4 -> 0.9.5: every changed line in lib.rs and channel.rs is a rustdoc comment (/// or //!) adding a "Feature Detection" section and cross-links. No executable code, no unsafe, no new imports. Read line by line, not merely pattern-matched. Formalised the rule, which is defensible precisely because it means NO EXECUTABLE CODE CHANGED: certify a delta only when every changed line in every .rs file is a comment or blank, or no .rs file is touched at all. Then machine-scanned all 73 source-touching diffs against it. TWO qualified: rand_core and version_check. Both were re-read by eye rather than trusted to the scanner, because a pattern match can be fooled by a line inside a block comment or a string containing //. Two of seventy-three is the MEASURED ceiling for what can be certified without human judgement - not where the effort ran out. The rest change real code, and the largest are web-sys (229k lines of generated FFI), ring (262k of crypto in C and asm), cudarc, winapi, tokio. exemptions 313 -> 304 fully audited 260 -> 269 cargo vet check passes. The waiver backstops the remainder, and WORKLIST.md Part 5 itemises all 73 smallest-first for whoever picks it up. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Opening the first PR ran harden.yml for the first time and three jobs
failed immediately. The reason none of this surfaced earlier is itself the
finding: the workflow triggers on `push` to master/main and on
`pull_request`, so every topic-branch push during this audit ran NO CI at
all. The green I had been reporting was local-only.
Format crates/ffai-mercury/tests/fuzz_corpus_replay.rs was not
rustfmt-clean. My file, added this session. Formatted; the diff
touches nothing else.
Miri "the 'miri' component ... is not available for the '1.95.0'
toolchain". The job installs nightly with miri, but
rust-toolchain.toml pins 1.95.0 and rustup honours that file over
the installed default, so `cargo miri` resolved to 1.95.0, which
has no miri. Now `cargo +nightly miri`.
Kani "rustc 1.93.0-nightly is not supported by the following
packages". Kani 0.67 bundles 1.93-nightly; the workspace declares
MSRV 1.95, and cargo refuses on the DECLARED VERSION ALONE before
compiling anything. The step now relaxes rust-version for its own
duration and restores it, preserving the exit status. The code
still builds because lib.rs gates the single 1.95 feature behind
`#![cfg_attr(kani, feature(const_mul_add))]`.
I hit the Kani MSRV problem locally hours ago and worked around it by
editing a throwaway clone, which meant CI never inherited the fix. Worth
recording: a workaround applied only to a scratch copy is not a fix.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three more that only the first real CI run could have found.
Clippy ffai-core::release_load_arena tripped
clippy::missing_const_for_fn - but ONLY on Linux. Every branch
in its body is #[cfg(windows)], so off-Windows the body is empty
and nursery says "make it const"; on Windows it calls a Win32
entry point and cannot be. Obeying the lint would compile on
Linux and break on Windows, so it is allowed with that reason
written down. It never fired on the author's box, which is the
whole argument for a Linux gate.
Deny `error[unlicensed]: intel-mkl-src = 0.8.1 is unlicensed` failed
the licence check outright. It publishes no SPDX expression: it
is a build-time downloader for Intel MKL, which ships under
Intel's own Simplified Software License. Added as a per-crate
EXCEPTION with the reason recorded, not by loosening the allow
list - and it reaches the graph only through the optional `mkl`
feature, so no default build touches it.
Audit rustsec/audit-check@v2 runs inside a musl container where
rust-toolchain.toml pins 1.95.0, and rustup dies with "override
toolchain '1.95.0-x86_64-unknown-linux-musl' is not installed".
Now installs and runs cargo-audit directly. The advisory scan is
the point; the container is not.
Local: cargo deny check licenses ok, ffai-core clippy clean, fmt clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Round three of "things only a Linux CI run could tell me".
Test ffai-media's video_ingest test failed to LINK with a wall of
"undefined symbol: DeblockChromaEq4H_ssse3",
"WelsIDctFourT4Rec_sse2" and friends. Those are OpenH264 (Wels)
SIMD kernels reached through rusty_h264. The runner has no
assembler, so they were never built. A missing BUILD TOOL, not
broken code - Windows developers never hit it because the
toolchain there ships one. Installs nasm.
Deny cargo-deny-action@v2 runs in a musl container that cannot resolve
this repo's pinned toolchain, and then failed licences on
intel-mkl-src - which deny.toml ALREADY excludes via
`[graph] all-features = false`, precisely because it is an
optional mkl backend with no SPDX expression. The action was not
honouring the repo's own config. Now installed and run directly.
Reverted the exceptions entry added in the previous commit: it was
the wrong mechanism (an exception permits licences a crate
DECLARES; this crate declares none) and the config was already
correct.
Clippy ffai-bench's non-Windows `Job` stub tripped
clippy::must_use_candidate on every method, because each is a
constant-valued placeholder. The real Windows impl returns handles
and never fires it. Allowed at the impl with the reason, rather
than annotating a stub to satisfy a lint its counterpart does not
raise.
Kani and Miri both PASS in CI as of the previous commit - H-30's "proofs
run in CI" is now literally true rather than locally true.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous commit message said it reverted this. It did not - the [[licenses.exceptions]] block was still in deny.toml. Removing it now so the file matches what the history claims. The entry was the wrong mechanism anyway: an exception permits licences a crate DECLARES, and intel-mkl-src declares none, which is why cargo-deny called it `unlicensed` rather than disallowed. The real fix was already in the file - `[graph] all-features = false` keeps that optional mkl backend out of the graph entirely - and the failure came from cargo-deny-action running in a container that ignored this repo's config. cargo deny check licenses: ok. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The supply-chain gate ran properly for the first time and found REAL
VULNERABILITIES, not configuration noise:
RUSTSEC-2025-0020 pyo3 Risk of buffer overflow in
`PyString::from_object` needs >= 0.24.1
RUSTSEC-2026-0177 pyo3 Missing `Sync` bound on
`PyCFunction::new_closure` needs >= 0.29.0
Both sat in ffai-py's pyo3 0.23.5 and were invisible because the
cargo-audit step had never executed - rustsec/audit-check@v2 runs in a
musl container that could not resolve this repo's pinned toolchain, so the
job died before scanning anything. A security gate that cannot start is
worse than no gate, because the dashboard still shows a job.
Upgrade turned out cheap. pyo3 0.29.2 + numpy 0.29.0 resolve, and exactly
one API break: `Python::allow_threads` is `Python::detach` since 0.28.
Same semantics, GIL still released around the pure-Rust detect() call, and
the comment explaining WHY the GIL is released is preserved.
cargo audit: 0 vulnerabilities (2 allowed unmaintained warnings remain -
paste and ttf-parser, both already accepted)
Also fixed the last Linux-only clippy failure: ffai-diana's cputime test
module holds only `#[cfg(windows)]` tests - it exercises a Win32 counter -
so off-Windows its `use super::*` served nothing and `-D warnings` failed
on unused_imports. Gated the import with the tests.
Local: 369 tests pass, ffai-py and ffai-diana clippy clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`cargo vet` failed after the pyo3 upgrade, which is the ratchet doing its job: pyo3, pyo3-ffi and pyo3-macros-backend arrived at 0.29.2 with no audit or exemption covering them. Regenerated exemptions - recorded as EXEMPTIONS, not audits, because nobody has read those 37k changed lines. Net 304 -> 303: the new versions are covered and the superseded 0.23.5 entries drop out. Also wrote up what H-09 actually did, because it is the sharpest finding of the day: that gate had never executed. rustsec/audit-check@v2 runs in a musl container that could not resolve this repo's pinned toolchain, so the step died before scanning - while still showing up as a job on the dashboard. A security gate that cannot start is worse than no gate, because the dashboard lies on its behalf. Once it ran it immediately found two real advisories in pyo3 0.23.5, one a buffer-overflow risk. Both now fixed. cargo vet check passes. cargo audit: 0 vulnerabilities. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ttimmahlax
pushed a commit
that referenced
this pull request
Aug 20, 2026
…th kit #1: chars per detected line — crops median 5.0, full pages median 28.0, a 5.6x difference against GT regions of median 183 chars. DBNet forms proper lines on pages and shatters on tight margin-free crops; the defect is crop-condition specific and the shipped pipeline is unaffected (0.1073 end-to-end English text is not what a fragmenting detector produces). 0.4133 is a valid measurement of a condition we do not ship — walked away from. Retires §23 s next-campaign framing: no plain-text ordering defect, no raster fix, and the 2279 repro cases reproduce an artificial condition. Three sections of hypothesis closed by two scripts and zero engine changes. #2: docs/textocr-claim.md — craft-crnn reads 0.1051 (CI [0.1013,0.1091], 7019 regions / 755 English pages) against published EasyOCR 0.26 on the benchmark s own Text OCR task. craft-crnn IS the EasyOCR model stack in pure Rust, placing it between Tesseract 0.096 and OpenOCR 0.070. Four disclosures travel with it, including that our document default scores WORSE on this task and why. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full
use-protection-pleasehardening pass onffai-mercury, plus the workspace-widework it pulled in. 53 commits.
Gates closed
Defects found and fixed
dot_i8_blocked(asr::vocab_int8). The inner loop testso < blkbut READS 32 BYTES per step, so ablkthat is not a multiple of 32 reads pastthe block and, on the final block, past the allocation — inside
unsafe. The guardchecked
d % blk == 0and notblk % 32 == 0, directly under a comment saying the kernelsteps 32 lanes at a time. Now enforced where it is load-bearing.
cargo test --workspace --libunder a comment saying "Lib + integration tests".--libruns neither. Two diana oracle tests asserted
ran > 0and went red without modelfixtures, so the job had been narrowed to
--lib, silencing them and every otherintegration test with them. Now
--lib --tests: 369 tests across 22 binaries.cargo fuzz cminsilently deleted all 12 named regression seeds, including theempty-input seed for the
reflect_padpanic — after which the corpus contained no emptyinput at all. Restored, and guarded by a test that fails if any goes missing.
&mut Vec<T>params that only index and sort,a doc line Markdown read as a list, three vestigial items, a bench
ifwith identicalbranches.
Supply chain (H-10)
Exemptions 473 → 304, fully audited 98 → 269, trusted publishers 128 → 167.
Imports are exhausted as verified fact — the cargo-vet registry holds exactly nine peers
and all nine are imported. Nine crates certified after reading their complete diffs, each
recorded with
--whoas an AI-agent review and explicitly not independentlyhuman-verified.
H-10 is NOT closed. 304 dependencies remain exempted rather than certified. It is
waived under H-41 with an owner and a 2026-11-15 expiry (R-008), with compensating controls
named:
cargo auditvs RUSTSEC on every PR,deny.tomlbanning unknown registries,Cargo.lockcommitted, andcargo vet checkpassing so the exempt set is explicit.supply-chain/WORKLIST.mditemises the remainder smallest-first.Known-open, deliberately
parsers. Deferred by the project lead; the identical review found four real defects in
mercury. Both crates'
Cargo.tomlsay so at the allow block.overflow-checkswaived after measurement (1.060x pipeline / 1.094x decoder).$TMPDIR, not rlibs) andstopped deliberately; not a blocking gate.
Verification
369 tests pass, clippy clean across all 12 crates, invariants linter clean,
cargo vet checkpasses, README status block and theRemade-With-Rust/mercurymirrorboth current.
🤖 Generated with Claude Code