docs(board): record #924 — the 0-of-11 spec audit and the ACTIVE plan - #925
Merged
Merged
Conversation
…the ACTIVE plan #924 merged with neither board file updated. It was MIXED — hygiene for #923 plus the audit fold, the §3 v2 rewrite, §8, the DRAFT->ACTIVE flip, §12.18 and two board entries — so the merged-PR obligations apply to the non-hygiene half. The entry records what #924 settled: - The meta-finding: pre-registration review by independent adversarial readers is load-bearing, not ceremony. The audit split sharply by KIND of statement — 22 of 24 KNOWN claims CONFIRMED with file:line, but 0 of 11 EV test specs SOUND. The author was reliable about what IS and near-uniformly unreliable about what would FALSIFY, one document after being corrected for exactly those modes and citing the falsifiability rule inside the specs themselves. - Two [G] ledger corrections: the bitboard primitives are at ndarray/src/bitwise.rs, not src/hpc/; and "whole-field shift" overstated — legA rigid-translates a 24x24 SPRITE within a 256x256 field, toroidally, so §12.16's "2 bytes moves the whole field" is corrected by §12.18. Whole-field advection is one (dx,dy) PER TILE, which is what EV-1 actually tests. - The eleven spec defects catalogued by class, including the ones with teeth: a silence half that was an identity comparison; another that was an arithmetic tautology; a fixture that 404s at the pinned timestep feeding a pipeline whose anomaly step destroys the bound-mass under test; an "exact to 1e-4" gate over 3-sig-fig prose AND Monte-Carlo statistics; a "winter" run that could pass on stale summer bytes; and a missing cos(lat) wrong by up to 2.6x exactly where the probe would fire. Honest limit recorded in the Confidence line: the audit's verify half is [G] (source quotes), but its attack half is a DESIGN REVIEW, not a measurement — a v2 spec is audited, not validated. Validation is the probe run, and zero EVs have run. STATUS_BOARD gains a note that the rows' descriptions summarize v1 shapes while the specs are now v2. EV-9 (Wave 0) needs no data and is the unblocked next move; it closes the only two [H] rows. This commit is hygiene-only, so per the termination clause it generates no further board obligations of its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (18)
📝 WalkthroughWalkthroughThe PR activates the audited evaluation plan and adds EV7B, EV9, EV3, EV4, EV8, and EV10 probes. It also adds weather fixture metadata, generated analysis results, and ignore rules for locally fetched arrays. ChangesEvaluation probes and audit state
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Poem
✨ Finishing Touches📝 Generate docstrings
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_fedc6fce-a425-4534-adb4-234dc2f8d253) |
…est as provenance The pre-baked ERA5 fixture landed untracked and was NOT covered by any .gitignore rule — 36 MB of .npy slabs one `git add -A` away from a commit, against the evaluation plan's own "fetched, never committed" discipline and the corpus-vs-scratch rule (committed fixtures are gates; fetched fixtures are measurements). Ignores the .npy under probes/weather-p1/fixture/ (both the flat legacy paths and the per-timestep dirs) while keeping manifest.json TRACKABLE, because the manifest is the provenance a fresh clone needs: per-array time_index, shape, nonfinite count, and min/max. EV-10's stale-fixture guard reads exactly this file to assert the timestep it declares is the timestep it loaded — a run whose fixture cannot prove its own time_index reports NO-VERDICT. Recorded here because it is measured, not assumed: re-fetching from gs://gcp-public-data-arco-era5 and re-running p1_ci_vs_floor.py reproduces every committed number BIT-EXACTLY (sat_frac 0.008476845430728927 / 0.008202342425643397, all three interior-CI values, all four floors). The fetch is deterministic and the committed JSON was honestly produced from these bytes. Also measured during the pre-wave check, exhaustively (3 chunk-key forms x 3 timesteps): 10m_v_component_of_wind, total_cloud_cover, sea_ice_cover and surface_pressure are declared in .zmetadata with .zarray identical to the working variables, but their chunks are NOT materialized — every key 404s. That data-blocks EV-1, EV-2 and EV-5. The board entry for the wave will carry this; noting it in the commit that touches the fixture so the two facts travel together. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
…bility, season PRE-AUDIT. These four slices of the autoattended wave (run wf_daae6e63-d39) are complete and ran; the PP-13 / PP-15 savant pass has NOT yet reviewed them, and the two Rust slices (EV-9, EV-7b) are still in flight. Committing now because the artifacts are real work at risk of a container reset, not because they are cleared. Any defect the savants find lands as a follow-up. Orchestrator checks done before this commit: - Rule 1 (unique-file write discipline) HELD: every path is new; no slice modified an existing file. - EV-3 re-run is byte-identical (deterministic). - EV-4's _summary reports overall_check1_pass only — it does NOT claim an overall pass, correctly leaving check2 unasserted. EV-4 — the knob-reached-the-code proof PASSES on all three variables. The CI ratio equals the (hi-lo) ratio to abs_diff 7.3e-15 / 6.4e-15 / 1.3e-15. This is the v2 assertion the audit demanded: a dead knob yields a constant curve, so this equality is what proves the window parameter reaches the code. check2_fisher_z_monotone is FALSE on all three, and neither increasing nor decreasing. The v2 spec PRE-REGISTERED that as a finding to investigate, not an apparatus failure (both the scale factor and (zlo,zhi) move with the window). Under the v1 wording this would have been auto-assigned to a bug hunt. EV-3 — a structural asymmetry the closed form surfaced and a grid search could not have: the 1%-exceedance flip-point is carried by 6/16/25 linear buckets at ~1.0% mass, versus a SINGLE fisher_z bucket at 14.22% / 11.31% / 4.46% mass. Fisher-Z's flip-point is set by one bucket holding an eighth of the field. Cross-validation: EV-3's summer linear flip-point 0.094119530779043 matches the committed p1_ci_vs_floor interior_ci_max EXACTLY, from an independently written implementation. EV-10 — both arms agree across seasons well inside the pre-registered factor-of-2 bar: linear 1.058x (0.0941 -> 0.0996), fisher_z 1.098x (0.4290 -> 0.4709). The stale-fixture guard fired as designed: max_abs_diff 72.55 K between summer and winter arrays, with time_index 547476 vs 543852 asserted against fixture/manifest.json. v1 could have passed by re-measuring summer bytes; this cannot. EV-8 — 12 series over K=25 independent seed-resamples, sd 5.69e-06 to 6.11e-04, CI-95 widths 2.2e-5 to 2.4e-3. The widest are the cross-unit T x u and Td x u per-variable arms, the expected place for spread. n is stated as 200,000 sampled PAIRS, not the 1,038,240 gridpoints (v1's error). No p-value computed — the deliverable is CI width and cross-seed stability, because rho(d_shared, truth) is a codec-consistency statistic, not inference. Not in this wave, data-blocked and measured as such: EV-1, EV-2 (need 10m_v_component_of_wind) and EV-5 (need total_cloud_cover / sea_ice_cover) — all 404 at every timestep probed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
…tests
Closes the plan's only two [H] rows: K-12 (latitude-band angular error) and
K-13 (palette-circular azimuth), whose measuring scratch tests were deleted
earlier in this arc, leaving numbers cited in three places with nothing
re-running them.
Written edit-only by a wave worker (run wf_daae6e63-d39, iron rule 7: no
cargo); compiled, run and disable-verified centrally by the orchestrator.
5 tests, all green in release.
The v2 spec DROPPED two v1 assertions as near-tautologies and this file
refuses them explicitly in its header: the bare 0.99-1.02x equal-budget ratio
(both arms have ~equal step size BY CONSTRUCTION, 1/127 vs 2/255) and the bare
0.352-degree mean (= step/4 arithmetically). What is asserted instead:
(i) anti-vacuity — house 7-bit+sign vs naive 8-bit-over-[-1,1] must
produce DIFFERENT bytes on >=5% of N=65536 samples, proving two code
paths exist at all
(ii) the equal-budget ratio with tolerance derived from the analytic step
sizes (ANALYTIC_RATIO = 255/254), not an eyeballed band
(iii) THE SPREAD, not two drifting points: pole-band mean / equator-band
mean >= 20x (K-12 measured ~29.75x)
(iv) comparative: palette-circular azimuth mean < nearest-(n,sign) mean on
the SAME sample
(v) the sampling scheme pinned in code — azimuth coverage >=200/256
coarse buckets per band, reusing signed360_claims.rs's K-3 technique
ORCHESTRATOR DISABLE-RUNS — all five verified RED under an injected defect:
D1 make the "naive" arm an alias of the house arm -> RED
D2 ASYMMETRIC: perturb ONLY the house arm's decode -> RED
D3 swap the spread's numerator/denominator -> RED
D4 palette arm ignores its azimuth input -> RED
D5 constant azimuth before bucketing (1/256 buckets hit) -> RED
Honest note on D5: my FIRST injection collapsed AZ_SAMPLES to 1 and the test
stayed green, which I nearly recorded as a vacuous assertion. It was my
injection that was wrong — AZ_SAMPLES feeds only test (iv); test (v) iterates
0..N and buckets the REAL encoded azimuth field. Running the disable-run the
test itself prescribes (constant azimuth) turns it red with "only 1/256
coarse buckets hit". A null result is a claim about the apparatus until the
apparatus is checked.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_0bff2bef-1b26-41eb-83f4-5462c352dec8) |
…limit on what they prove Written edit-only by a wave worker (run wf_daae6e63-d39); compiled, formatted, run and canary-probed centrally by the orchestrator. OVERALL PASS as shipped, plus 11 inline tests green: regular (stride 1 / mod 16, commensurate) ratio 2.916582e4 >= R_HI 150 comma (stride 4 / mod 17) ratio 6.806976e-1 <= R_LO 15 coprime2 (stride 5 / mod 17) ratio 6.806976e-1 <= R_LO 15 comma phase_total_energy 6.510592e6 (nonzero) Two things the orchestrator gate established that the run alone does not. (1) THE IDENTICAL COMMA/COPRIME2 RATIOS ARE CORRECT, NOT A DEAD PARAMETER. Bit-identical statistics from two different strides is the exact signature of a parameter that never reaches the code, so it was canaried rather than assumed. Changing COPRIME2_STRIDE 5 -> 3 leaves the ratio bit-identical (6.806976e-1), but forcing the COMMA arm to a constant walk (stride 17 over 17) drives phase_total_energy to 0.000000e0 and BAR4 FAILS -> OVERALL FAIL. So the parameter does reach the code, and the invariance across coprime strides is itself the finding: every stride coprime to 17 is a permutation of Z_17, so the occupied harmonic set is the same and U0 is not in it. That invariance IS the "coprime is what matters, 17 is not magic" claim. The v2 silence-side fixture guard (BAR4) works exactly as the audit demanded. (2) MEASURED P1 — THE INSTRUMENT CHECKS ONE BIN AND CAN BE EVADED. U0 = FIELD_SIZE / TILE_PITCH = 17 is the tile-pitch FUNDAMENTAL only. A commensurate walk whose period is a PROPER DIVISOR of the tile pitch aliases at a HARMONIC and passes the silence bar. Measured, not derived: stride 4 over modulus 16 (gcd 4, period 4, nonzero energy so BAR4 cannot catch it) scores 6.808293e-1 at bin 17 — comfortably under R_LO=15, a PASS — while at bin 68 (the 4th harmonic) the same walk scores 3.529307e4. It aliases hugely; the probe is not looking there. Consequence, stated precisely: this does NOT weaken the comma result. The comma arm is silent at BOTH bins (0.681 at 17, 0.376 at 68), so it genuinely does not alias. What is narrower than it appears is the probe's DISCRIMINATING POWER — it proves "the comma does not alias at the fundamental", not "the comma does not alias". Follow-up: take the statistic over the tile-pitch fundamental AND its harmonics (max or sum), not U0 alone. Also noted: R_LO = 15 against a measured noise-plus-comb floor of ~0.68 is a 22x margin, so the silence bar is loose enough that essentially anything but the regular arm clears it. The harmonic fix above is what would give that bar real discriminating work to do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
…ing" was an artifact
The PP-13 / PP-15 savant pass returned 32 findings (3 P0, 14 P1, 15 P2), both
verdicts FIX-FIRST. This commit closes the two P0s. I verified each myself
rather than accepting the report.
P0-A — EV-7b's 11 pre-registered-bar tests DO NOT RUN under `cargo test`.
Cargo's default for an [[example]] target is `test = false`, and helix declares
only `[[example]] name = "prove_residue"`, so ev7b was auto-discovered, BUILT,
and its `#[cfg(test)] mod tests` silently skipped. Measured before the fix:
`cargo test --release | grep -c '^test tests::'` = 0. They passed in my gate
only because I happened to use the named `--example` selector. Fix: an explicit
`[[example]] name = "ev7b_comma_antimoire" / test = true` stanza in the shared
crates/helix/Cargo.toml — a file NEITHER worker was scoped to, so this is the
orchestrator's consolidation job (pattern Rule 3). After: 11 tests run under a
plain `cargo test`, whole crate green.
P0-B — I PUBLISHED A WRONG FINDING. I reported the non-monotone Fisher-Z
interior-CI curve as a substantive result with a mechanism ("the scale's
controlling tail flips"). PP-13 called it a numerical artifact of
np.percentile interpolation against the eps arctanh clip pole. Measured, by
perturbing ONLY apparatus knobs and leaving the data untouched
(ev4_apparatus_sensitivity.py, new):
eps=1e-6 -> [0.006406, 0.012076, 0.018147, 0.042380] INCREASING
eps=1e-9 -> [0.005607, 0.002447, 0.001722, 0.015087] non-monotone
eps=1e-12 -> [0.004906, 0.000476, 0.000153, 0.005146] non-monotone
eps=1e-15 -> [0.004291, 0.000090, 0.000013, 0.001717] non-monotone
percentile method spread (max/min over the same data): 64.1x
'lower'/'higher'/'nearest' land an order of magnitude below 'linear'
The SHAPE changes with eps (monotone at 1e-6, non-monotone below) and the
MAGNITUDE moves 64x with the interpolation method. A shape that survives
neither knob is not a property of the field. The curve must NOT feed D-2.
What survives, and is independently real: `scale = max(|lo_q|,|hi_q|)` is set
by the HI tail at the three narrower windows and by the LO tail at 0.02-99.98
(lo_q=-31.999 vs hi_q=+30.527). That tail flip is a genuine structural fact
about the parameterization. My error was attributing the non-monotonicity TO
it — the flip is real, the curve it was invoked to explain is not.
Meta: the v2 spec instructed "report it as a FINDING to investigate, NOT an
apparatus failure". That instruction was right; investigating is what showed
it to be apparatus. My mistake was elevating an un-investigated observation to
a headline in the same breath as reporting it. "Finding to investigate" is not
"finding".
Remaining: 14 P1 + 15 P2, several substantive (EV-4's ratio clause is itself a
tautology; EV-3/EV-4/EV-8 read the legacy flat fixture path with no timestep
guard; EV-8's pre-registered indictment condition FIRED and was resolved
post-hoc; EV-9 measures decoded round-trip error, which needs an explicit plan
§0 scope carve-out). Tracked for the consolidation pass; not silently dropped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
AdaWorldAPI
marked this pull request as ready for review
August 11, 2026 15:50
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_0a2af276-3840-497d-b5c0-642e1a96e699) |
AdaWorldAPI
pushed a commit
that referenced
this pull request
Aug 12, 2026
…h, fixed in all three homes Codex P2 on #930: "identical R-bar with mu shifted 100.3 deg" fused two different comparisons. Identical R-bar belongs to steering<->rotated (0.343 = 0.343, mu shifted exactly 90 -- rotation preserves concentration by construction); the 100.3 deg separation belongs to surface<->rotated, where R-bar is NOT identical (0.516 vs 0.343). The composite was false in all three homes: LATEST_STATE, the #929 arc entry, report SS5.13. Stated correctly the finding is STRONGER -- the control separates from surface in BOTH channels. Unmerged board entries composed in place (append-only rule's unmerged-PR allowance); the merged report gets a dated correction note. Lesson banked: the 13/13 figure verification checked every NUMBER and still missed this -- a relation between two individually correct numbers can be false. Verify comparative claims AS claims: every "identical/same/larger" must name both operands, and the check must evaluate the relation. Also codex P2 #2: the LATEST_STATE shipped-PR table had stalled at #780. Added #926-#929 rows plus an explicit gap-note row for #781-#925 (carried by PR_ARC_INVENTORY) -- honest gap, not silent reconstruction of ~150 rows. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
AdaWorldAPI
pushed a commit
that referenced
this pull request
Aug 12, 2026
…ng, and the append-only audit could not fail Operator: "930 has comments / check also previous 5 / if you want go back another 5." All 68 review comments on #920-#930 enumerated and checked against the TREE, not the merge. Depth is non-uniform and said so: #922-#930 finding-by-finding; #920 (27) and #921 (5) spot-checked on P1/governance only (both clean), ~30 older findings there left explicitly UNVERIFIED. Clean: #922/#924/#925/#929 zero comments; all 27 of #926's findings fixed in the tree (equal-budget grid_pts + regenerated E2 JSON, seam-wrapping subgrid_min, find_center->None, F7d 35->40, CT_F12 NO-VERDICT, persisted storm metadata, np.roll longitude, __file__-relative write, net-decay E6, the 93-97%->90.9-94.3% headline); #923's plan-status P2 resolved. Three open, all frozen in append-only ledgers, all corrected in NEW entries: 1. "+92.76 Pa moves R2 in the 5th decimal" is refuted by the report's own carve table 15 lines above it: carve A's +92.76 Pa moved R2 0.9212->0.9129 (0.0083, THIRD decimal); carve D's +1.59 Pa moved it 2.4e-06 (SIXTH). The var() BUG was blind at every magnitude; the STATISTIC is near-blind only in the single-digit-Pa regime -- which is exactly where "lossless" was claimed. 2. "10 probe scripts with committed JSON" undercounts: 15 .py added, 11 with a committed .json. 3. "+13/-0, +10/-0, +0/-0": measured +13/-0, +17/-0, and EPIPHANIES absent from the net diff -- it was +1/-1 at 0f9e6bc (the in-place edit), zeroed by the revert. The +0/-0 was a revert artefact, not evidence of purity. And the audit METHOD is replaced: zero-deletions proves ADDITIVE, not PREPEND. The suffix check (new.endswith(old)) proves it. Both halves measured before banking: a constructed end-append scores zero-del=True/suffix=False; a true prepend True/True. It also fires on THIS PR (LATEST_STATE suffix=False at +15/-0) -- correctly, since #930 inserts table rows mid-file and composes an unmerged entry in place. A False is a demand for justification, not a verdict. Same defect shape as #930's relation error, twice in three days: two individually-true numbers asserted of one pair when each belongs to a different one. A figure-by-figure check cannot catch it; the relation must be evaluated as a claim. Already standing in report SS10.1 and the worker preamble of weather-w-probes-v1 (both verified present, not assumed). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Post-merge board hygiene for #924, which merged with zero references in either board file. It was mixed — hygiene for #923 plus the audit fold, the §3 v2 rewrite, §8, the DRAFT→ACTIVE flip and §12.18 — so the merged-PR obligations apply to the non-hygiene half.
What the entry records
The meta-finding. Pre-registration review by independent adversarial readers is load-bearing, not ceremony. The audit split sharply by kind of statement: 22 of 24 KNOWN claims CONFIRMED with
file:line, but 0 of 11 EV specs SOUND. Reliable about what is; near-uniformly unreliable about what would falsify — one document after being corrected for exactly those modes, citing the falsifiability rule inside the specs themselves.Two
[G]ledger corrections. Bitboard primitives are atndarray/src/bitwise.rs, notsrc/hpc/. And a sprite moves for 2 bytes, not the field — legA rigid-translates a 24×24 sprite within a 256×256 field, toroidally; §12.18 corrects §12.16, and whole-field advection is one(dx,dy)per tile, which is what EV-1 actually tests.The eleven defects catalogued by class, including the ones with teeth: an identity-comparison silence half; an arithmetic-tautology silence half; a fixture that 404s at the pinned timestep feeding a pipeline whose anomaly step destroys the bound-mass under test; an "exact to 1e-4" gate over 3-sig-fig prose and Monte-Carlo statistics; a "winter" run that could pass on stale summer bytes; a missing
cos(lat)wrong by up to 2.6× exactly where the probe would fire.The honest limit, in the Confidence line
The audit's verify half is
[G](source quotes). Its attack half is a design review, not a measurement — a v2 spec is audited, not validated. Validation is the probe run, and zero EVs have run.STATUS_BOARDgains a note that its row descriptions summarize v1 shapes while the specs are now v2.EV-9 (Wave 0) needs no data and closes the only two
[H]rows — the unblocked next move.Scope
Three board files, 30 insertions, 0 deletions. No code, no types, no plans. Per the termination clause, this hygiene-only PR generates no further board obligations of its own.
🤖 Generated with Claude Code
https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi
Summary by CodeRabbit
New Features
Documentation
Chores