From 8977a11ebb6c241fecd3c8a5abb58d4b3f31129b Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 13 Aug 2026 10:55:15 +0000 Subject: [PATCH 1/2] =?UTF-8?q?CORRECTION:=20D-WXS-7=20was=20never=20block?= =?UTF-8?q?ed=20on=20the=20mint=20=E2=80=94=20I=20said=20it=20was,=20four?= =?UTF-8?q?=20times?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The session repeatedly reported the gate as sitting behind D-WXS-4 (the bake) and therefore behind D-WXS-0 (the classid mint). Checked against bar B6 rather than against my own previous sentence, that is wrong. B6 needs real ERA5 values at grid scale, the SHIPPED floor.rs quantiser, a shuffled decode table as the control, a 16/64/256 resolution ladder, and a Spearman. Every one of those exists. It needs no Lance dataset, no NodeRow, and no classid -- because it measures the CODEC, and the codec shipped with D-WXS-3. The mis-read, named: the deliverable line says "representation fidelity on the substrate ... over pairs sampled from the whole grid". "On the substrate" and "at grid scale" describe the SCALE and the SOURCE -- the whole real field through the shipped codec, rather than four hand-picked boxes through a numpy re-derivation. They do not say "persisted to Lance". I read the phrase as implying the bake and then repeated the conclusion without returning to the bar. This is the session's own recurring defect, committed by me, about my own plan: a claim that was plausible, load-bearing, never verified against the artifact, and propagated by repetition. Same shape as the stale saturation figure, the three vacuous disable probes, and the "no external review" statement -- all of which this session also caught. The countermeasure is already on the board: an audit must terminate at an artifact, and a BLOCKER is an artifact-checkable claim like any other. I had been treating blockers as background facts rather than as claims. What IS blocked, precisely: D-WXS-4 and D-WXS-5 write rows and a row needs a routable classid; D-WXS-6 needs datasets to read; D-WXS-9/10 need the neighbour read the bake provides. D-WXS-7 and D-WXS-8 need none of it. Consequence: the gate everything else hangs on has been runnable since D-WXS-3 landed. Re-scoped from Blocked to READY on the board, with the reason recorded in plan section W3 so the next session does not inherit the wrong blocker. Section 4's "W0 -> W1 -> W2 -> W3" is a dependency ordering for the BAKE deliverables, not a licence to treat W3 as unreachable. Doc-only; no bar changed, no verdict claimed, nothing run yet. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .claude/board/STATUS_BOARD.md | 4 +-- .claude/plans/weather-soa-bake-v1.md | 45 ++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+), 2 deletions(-) diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index c59c8372..7be16beb 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -18,8 +18,8 @@ with a control that can lose and a stay-silent twin. | D-WXS-4 | the bake: one timestep → 1,038,240 NodeRows → ONE Lance version | 1 | Queued — **blocked behind D-WXS-0** (must refuse to write without a minted classid) | bar B3; the missing path | | D-WXS-5 | statics bake — separate classid, separate dataset, exactly ONE version | 1 | Queued | bar B4; avoids ~1.3 PB of rewritten constants | | D-WXS-6 | version-range read (`QueryReference::at(v,rung)` + `deinterlace`) + version-count scaling measurement | 2 | Queued | bar B5; KILL if growth is superlinear at 92,044 versions | -| D-WXS-7 | **D-WXA-5 re-homed and RE-SPECIFIED** — ρ(code_dist, field_dist) via `jc::reliability::spearman` over whole-grid pairs. (a) ρ ≥ 0.9996 (the bar a real pair FAILED at 0.999556); (b) shuffled-codebook control < 0.98 (measured losable at 0.003–0.159); (c) 16/64/256-level ladder must be MONOTONE before any verdict | 3 | Queued | ⚠ poc-v2's ρ ≥ 0.98 is at risk of being vacuous — D-CZ-1 §6.4 measured real-arm ρ spread 3e-6…4.7e-5 | -| D-WXS-8 | cross-variable comparability at grid scale (≥4 variables, ≥2 units, ≥3 seasons); per-variable-floor control must LOSE | 3 | Queued | P2 (0.9997 vs 0.857–0.875) promoted from [H] or refuted | +| D-WXS-7 | **D-WXA-5 re-homed and RE-SPECIFIED** — ρ(code_dist, field_dist) via `jc::reliability::spearman` over whole-grid pairs. (a) ρ ≥ 0.9996 (the bar a real pair FAILED at 0.999556); (b) shuffled-codebook control < 0.98 (measured losable at 0.003–0.159); (c) 16/64/256-level ladder must be MONOTONE before any verdict | 3 | **READY — NOT blocked** (corrected 2026-08-13; the session reported this as gated behind the bake/mint four times without checking). Bar B6/B7 need real ERA5 values, the SHIPPED `floor.rs` quantiser, a shuffled control and a Spearman — **no Lance dataset, no NodeRow, no classid**. "On the substrate" describes scale and source, not persistence. Runnable since D-WXS-3 landed | ⚠ poc-v2's ρ ≥ 0.98 is at risk of being vacuous — D-CZ-1 §6.4 measured real-arm ρ spread 3e-6…4.7e-5 | +| D-WXS-8 | cross-variable comparability at grid scale (≥4 variables, ≥2 units, ≥3 seasons); per-variable-floor control must LOSE | 3 | **READY — NOT blocked** (corrected 2026-08-13; the session reported this as gated behind the bake/mint four times without checking). Bar B6/B7 need real ERA5 values, the SHIPPED `floor.rs` quantiser, a shuffled control and a Spearman — **no Lance dataset, no NodeRow, no classid**. "On the substrate" describes scale and source, not persistence. Runnable since D-WXS-3 landed | P2 (0.9997 vs 0.857–0.875) promoted from [H] or refuted | | D-WXS-9 | ζ = ∂v/∂x − ∂u/∂y as a substrate read (neighbour-key stencil) + the differencing-amplifies-quantisation falsifier, reported per ζ-magnitude decile | 4 | Queued | bar B8; KILL ⇒ ζ becomes its own baked lane | | D-WXS-10 | **D-CZ-8 re-homed** — ζ-percentile regime bands over the WHOLE grid + coverage-matched donors + range-normalised `L`; shuffled-ζ control; same-band stay-silent twin | 4 | Queued | the grid buys CONTROL of the §7.9 confound, NOT its dissolution; KILL if `L̄`~range ρ≈1.0 survives matching | | D-WXS-11 | MEASURE a full-grid bake wall time; state the ~8.1 s serial extrapolation as the prior and confirm/correct it in the same artifact | 5 | Queued | replaces an extrapolation from ONE cycle (514 ms / 65,536 owners); D-KIA-A2 still unbuilt | diff --git a/.claude/plans/weather-soa-bake-v1.md b/.claude/plans/weather-soa-bake-v1.md index b1c3a256..5a1bf36b 100644 --- a/.claude/plans/weather-soa-bake-v1.md +++ b/.claude/plans/weather-soa-bake-v1.md @@ -655,6 +655,51 @@ floor loses to per-variable floors on any cross-unit pair at grid scale, §4 pol refuted and the substrate is a per-variable store — which would retract the arc's product claim, and is the reason this runs early. +#### ⚠ CORRECTION 2026-08-13 — `D-WXS-7` is NOT blocked on the bake, and I said it was, four times + +The session repeatedly reported `D-WXS-7` as gated behind `D-WXS-4` (the bake) +and therefore behind `D-WXS-0` (the classid mint). **That is wrong, and it was +never checked before being repeated.** + +Read bar B6 above against what it actually needs: + +| bar B6 needs | where it lives | blocked? | +|---|---|---| +| real ERA5 field values, whole-grid scale | the store, over HTTP | **no** — `era5_variable_census.py` proved the fetch path works | +| a 256-level quantiser + its `[lo,hi]` window | `floor.rs`, **shipped** (`D-WXS-3`) | **no** | +| a shuffled decode table (the control) | a permutation of that codebook | **no** | +| a 16/64/256 resolution ladder | three `calibrate` calls | **no** | +| Spearman ρ | `jc::reliability::spearman`, shipped | **no** | +| a Lance dataset, a `NodeRow`, a classid | — | **not required by any part of B6** | + +**The mis-read, named.** The deliverable line says *"representation fidelity on +the substrate … over pairs sampled from the whole grid"*. "On the substrate" +and "at grid scale" describe the **scale and the source** — the whole real +field, through the shipped codec, rather than four hand-picked boxes through a +numpy re-derivation. They do **not** say "persisted to Lance". I read the phrase +as implying the bake and then repeated the conclusion without going back to the +bar. + +**This is the session's own recurring defect, committed by me, about my own +plan** — a claim that was plausible, load-bearing, never verified against the +artifact, and propagated by repetition. It is the same shape as the stale +saturation figure, the vacuous disable probes, and the "no external review" +statement, all of which this session also caught. The countermeasure is the one +already on the board: **an audit must terminate at an artifact**, and a blocker +is an artifact-checkable claim like any other. + +**What IS blocked, precisely.** `D-WXS-4` (the bake) and `D-WXS-5` (statics) need +the mint, because they write rows and a row needs a routable classid. +`D-WXS-6` (the version-range read) needs datasets to read. `D-WXS-9`/`D-WXS-10` +(ζ) need the neighbour read the bake provides. **`D-WXS-7` and `D-WXS-8` need +none of that** — they measure the *codec*, and the codec is shipped. + +**Consequence:** the gate everything else hangs on has been runnable since +`D-WXS-3` landed. It is re-scoped here from *blocked* to *ready*, and the +sequencing note in §4 ("`W0 → W1 → W2 → W3`") is a dependency ordering for the +*bake* deliverables, not a licence to treat W3 as unreachable while W1 is +incomplete. + ### W4 — re-homing `D-CZ-8` (ζ + a range-normalised transfer metric) | D-id | deliverable | From a40076094670e2db8646e3a41f816924d3a12a32 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 13 Aug 2026 11:36:10 +0000 Subject: [PATCH 2/2] D-WXS-7/D-WXS-8 RUN: the real gate ran -- 12/12 PASS, and a real mixed result on the other The gate this arc had reported as blocked behind the classid mint four times turned out to need none of it (previous commit). Once corrected, it ran the same day, against REAL live-fetched ARCO-ERA5 grid data, 3 real calendar seasons, computed with jc::reliability::spearman as bar B6 literally requires -- not a Python re-implementation. Method. fidelity_probe_fetch.py extends the proven fetch.py pattern (real HTTPS, real Zarr v2 decode, 404=all-NaN=valid-missing-chunk semantics) to grid scale. The 3 seasons were NOT assumed -- a first attempt at 4 fixed 2021 calendar anchors found only the SAME 3 variables present at every one of them (2m_temperature, 2m_dewpoint_temperature, 10m_u_component_of_wind); 10m_v_component_of_wind / mean_sea_level_pressure / surface_pressure / total_column_water_vapour / total_cloud_cover / sea_surface_temperature were absent at all 4, confirming the README's "sparse by design" finding is not a one-timestep artifact. 10m_wind_speed does not exist in this store at all, confirming the pre-existing weather-normalized-substrate.md finding. A follow-up live HEAD-sweep of 24 candidate timesteps across the whole 1959-2023 archive found winter/spring/summer, each independently meeting bar B7's >=4-variable />=2-unit floor with genuinely present, fully-finite fields. sea_surface_temperature came back PHYSICALLY MASKED (686,364 of 1,038,240 finite -- land is NaN by definition) and was correctly excluded as not-fully-usable rather than force-included. fidelity_probe_prep.py quantises via the exact floor.rs formula (re-expressed in Python for the same reason floor.rs itself is zero-dep -- this stage fetches over HTTP, which the crate deliberately cannot do) and writes 200,000-pair raw (truth, code) f64 arrays per comparison, matching p2_probe.py's own N. crates/weather-poc/examples/fidelity_probe.rs reads them and computes every rho with jc::reliability::spearman, added as a DEV-DEPENDENCY ONLY (Cargo.toml explains why that is safe here and was not safe for helix: dev-only, and jc is itself zero-external-dep by default). Sanity-checked before trusting a number: the shuffled-decode arm is a genuinely different array from the unshuffled one (range [0,4.92] vs [0,202], not a copy), and its rho collapses to 0.02-0.024 -- the pipeline demonstrably discriminates. D-WXS-7 (bar B6): 12/12 PASS, all 3 seasons. rho256 = 0.999909 / 0.999895 / 0.999684. This DOES NOT REPLICATE the plan's own cited near-miss (K x K = 0.999556, below bar, from p2_probe.py's smaller fixture) -- the earlier number is not retracted, it was real on a real smaller fixture, but it no longer licenses "the substrate's fidelity is marginal" at grid scale. D-WXS-8 (bar B7): mixed, reported in full, nothing filtered or softened. Control (per-variable floor must LOSE, the KILL-gated clause) 16/16 PASS -- rho_pervar 0.245-0.939 vs rho_shared 0.9987-0.9999 on identical pairs. The KILL does not fire; the shared-floor design is not refuted. Primary (rho_shared >= 0.9996 exactly) 10/16 PASS, 6 close misses (0.9987-0.9996, skewed toward wind/pressure pairs). Stay-silent twin: the diff<=0.0001 half passes at spring/summer, fails at winter (0.000174); the ZERO EMPTY BUCKETS half fails at ALL THREE seasons (38/39/45 of 256) -- the literal "zero" from the 1-timestep/3-variable fixture this claim was carried from does not hold once the shared floor pools a wider real multi-unit window at grid scale. The reusable shape, recorded on the board: a KILL clause and a primary/stay-silent bar are not the same severity, and collapsing them into one verdict sentence loses the finding. This run had three distinct tiers on ONE deliverable -- a KILL that did not fire (nothing retracted), a strict numeric bar that partly missed (a real, reported gap), and an earlier measurement that did not replicate at the new scale (a correction, not a failure). Every pre-registered verdict applied literally; every one printed, including the FAILs. Data discipline: the 138 MB of raw pair-distance f64 arrays and the store's 48 KB .zmetadata snapshot are gitignored, matching the repo's existing rule for probes/weather-p1/fixture/*.npy (same rationale: re-fetch deterministic from a committed script, do not carry regeneratable bytes). Committed: both Python stages, the Rust example, the small manifest/meta.txt sidecars, and the two measured-results JSONs (the actual receipts). A stale fixture/season_autumn/ directory, leftover from the first (unsuccessful) 4-fixed-season fetch attempt, was deleted rather than left orphaned. Gates: cargo build --example fidelity_probe clean; cargo test 35/35; fmt clean; jc's own pre-existing warnings (not touched) are the sibling crate's, not this crate's. Python syntax-checked. Co-Authored-By: Claude Claude-Session: https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi --- .claude/board/EPIPHANIES.md | 64 ++++ .claude/board/STATUS_BOARD.md | 4 +- .claude/plans/weather-soa-bake-v1.md | 116 +++++++ .gitignore | 12 + crates/weather-poc/Cargo.lock | 7 + crates/weather-poc/Cargo.toml | 17 + crates/weather-poc/examples/fidelity_probe.rs | 308 ++++++++++++++++++ probes/weather-p1/fidelity_probe_fetch.py | 101 ++++++ probes/weather-p1/fidelity_probe_prep.py | 214 ++++++++++++ .../fixture/fidelity_pairs/manifest.json | 246 ++++++++++++++ .../fixture/fidelity_pairs/spring_meta.txt | 11 + .../fixture/fidelity_pairs/summer_meta.txt | 11 + .../fixture/fidelity_pairs/winter_meta.txt | 15 + .../fixture/fidelity_probe_availability.json | 125 +++++++ .../fixture/fidelity_probe_results.json | 60 ++++ 15 files changed, 1309 insertions(+), 2 deletions(-) create mode 100644 crates/weather-poc/examples/fidelity_probe.rs create mode 100644 probes/weather-p1/fidelity_probe_fetch.py create mode 100644 probes/weather-p1/fidelity_probe_prep.py create mode 100644 probes/weather-p1/fixture/fidelity_pairs/manifest.json create mode 100644 probes/weather-p1/fixture/fidelity_pairs/spring_meta.txt create mode 100644 probes/weather-p1/fixture/fidelity_pairs/summer_meta.txt create mode 100644 probes/weather-p1/fixture/fidelity_pairs/winter_meta.txt create mode 100644 probes/weather-p1/fixture/fidelity_probe_availability.json create mode 100644 probes/weather-p1/fixture/fidelity_probe_results.json diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 4ea125c1..b9bff00a 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,67 @@ +## 2026-08-13 — E-THE-REAL-GATE-RAN-AND-QUALIFIED-NOT-RETRACTED-THE-CLAIM-1 + +**Status:** FINDING `[G]` — measured, real live-fetched grid data, 3 seasons, +`jc::reliability::spearman`, disable-sanity-checked. **Confidence:** High. + +**What ran.** `D-WXS-7`/`D-WXS-8` — the gate this arc had been treating as +blocked behind the classid mint for four consecutive statements +(`E-...` — see the same-day correction entry above this one) turned out to +need none of it. Once corrected, it ran the same day: real ARCO-ERA5 fetch +(public HTTPS, 404=NaN=valid-missing-chunk semantics), 3 real calendar +seasons found by a live availability sweep (not assumed — a first attempt at +4 fixed 2021 anchors found only 3 variables present at ALL of them), +200,000-pair sampling at the exact `floor.rs` quantisation, scored with +`jc::reliability::spearman` (added to `weather-poc` as a **dev-dependency +only** — the default `[lib]` build stays zero-dep). + +**Two results, and they cut in different directions on the SAME underlying +claim — recorded together because reading either alone would mislead.** + +1. **`D-WXS-7` (bar B6): the K×K pair does NOT replicate an earlier + near-miss.** The plan's own §0.3 cites a smaller-scale measurement — + K×K = 0.999556, below the 0.9996 bar — as the reason the bar "can fire" at + all. At real grid scale, 3 real seasons, computed with `jc` not `scipy`: + 0.999909 / 0.999895 / 0.999684. Clears the bar with margin, every time. + The earlier number is not wrong or retracted — it was a real measurement + on a real, smaller fixture — but it no longer licenses "the substrate's + fidelity is marginal" as a grid-scale conclusion. + +2. **`D-WXS-8` (bar B7): the core claim holds; two literal thresholds do + not.** The KILL-gated control — per-variable floors must LOSE on every + cross-unit pair — passed **16/16**, by a wide margin (0.245–0.939 vs + 0.9987–0.9999). The KILL did not fire; the shared-floor design is not + refuted. But the PRIMARY bar (ρ≥0.9996 exactly) missed on 6/16 pairs + (close misses, 0.9987–0.9996, skewed toward wind/pressure), and the + stay-silent twin's **"zero empty buckets"** half — carried verbatim from + a 1-timestep/3-variable fixture — failed at **all three** real seasons + (38–45 of 256 empty). Read plainly: a percentile-trimmed window pooled + across more variables at real grid scale necessarily leaves some slack + for any one variable's narrower distribution. Unsurprising in direction; + simply never re-verified at scale before this run. + +**The reusable shape: a KILL clause and a PRIMARY/twin bar are not the same +severity, and a report that collapses them loses the finding.** This run had +three tiers of outcome on ONE deliverable — a KILL that did not fire (nothing +retracted), a strict numeric bar that partly missed (a real gap, reported), +and an EARLIER measurement that did not replicate at the new scale (a +correction, not a failure). Flattening any two of those into one sentence — +"D-WXS-8 failed" or "D-WXS-8 passed" — would have been wrong either way. The +discipline that produced a correct report here: apply every pre-registered +verdict literally, print every one including the FAILs, and only THEN decide +what the KILL clause specifically licenses. + +**Sanity check before trusting any ρ:** the shuffled-decode arm was verified +to be a genuinely different array from the unshuffled one (range `[0, 4.92]` +vs `[0, 202]`, not a copy or a stub) before its collapsed ρ (0.02–0.024) was +read as a real result rather than a pipeline artifact. + +**Cross-ref:** the same-day correction above (`D-WXS-7` was never blocked); +`.claude/plans/weather-soa-bake-v1.md` sec 4 W3 (bars B6/B7, verbatim); +`probes/weather-p1/p2_probe.py` (the fixture-scale predecessor); `E-A-DISABLE- +PROBE-CAN-ITSELF-BE-VACUOUS-1` (the same session's sibling method finding). + +--- + ## 2026-08-13 — E-A-TOTAL-FUNCTION-THAT-CANNOT-REFUSE-IS-A-CORRUPTION-PATH-1 **Status:** FINDING `[G]` — two measured instances, one crate, one hour. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 7be16beb..e2e245e7 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -18,8 +18,8 @@ with a control that can lose and a stay-silent twin. | D-WXS-4 | the bake: one timestep → 1,038,240 NodeRows → ONE Lance version | 1 | Queued — **blocked behind D-WXS-0** (must refuse to write without a minted classid) | bar B3; the missing path | | D-WXS-5 | statics bake — separate classid, separate dataset, exactly ONE version | 1 | Queued | bar B4; avoids ~1.3 PB of rewritten constants | | D-WXS-6 | version-range read (`QueryReference::at(v,rung)` + `deinterlace`) + version-count scaling measurement | 2 | Queued | bar B5; KILL if growth is superlinear at 92,044 versions | -| D-WXS-7 | **D-WXA-5 re-homed and RE-SPECIFIED** — ρ(code_dist, field_dist) via `jc::reliability::spearman` over whole-grid pairs. (a) ρ ≥ 0.9996 (the bar a real pair FAILED at 0.999556); (b) shuffled-codebook control < 0.98 (measured losable at 0.003–0.159); (c) 16/64/256-level ladder must be MONOTONE before any verdict | 3 | **READY — NOT blocked** (corrected 2026-08-13; the session reported this as gated behind the bake/mint four times without checking). Bar B6/B7 need real ERA5 values, the SHIPPED `floor.rs` quantiser, a shuffled control and a Spearman — **no Lance dataset, no NodeRow, no classid**. "On the substrate" describes scale and source, not persistence. Runnable since D-WXS-3 landed | ⚠ poc-v2's ρ ≥ 0.98 is at risk of being vacuous — D-CZ-1 §6.4 measured real-arm ρ spread 3e-6…4.7e-5 | -| D-WXS-8 | cross-variable comparability at grid scale (≥4 variables, ≥2 units, ≥3 seasons); per-variable-floor control must LOSE | 3 | **READY — NOT blocked** (corrected 2026-08-13; the session reported this as gated behind the bake/mint four times without checking). Bar B6/B7 need real ERA5 values, the SHIPPED `floor.rs` quantiser, a shuffled control and a Spearman — **no Lance dataset, no NodeRow, no classid**. "On the substrate" describes scale and source, not persistence. Runnable since D-WXS-3 landed | P2 (0.9997 vs 0.857–0.875) promoted from [H] or refuted | +| D-WXS-7 | **D-WXA-5 re-homed and RE-SPECIFIED** — ρ(code_dist, field_dist) via `jc::reliability::spearman` over whole-grid pairs. (a) ρ ≥ 0.9996; (b) shuffled-codebook control < 0.98; (c) 16/64/256-level ladder must be MONOTONE before any verdict | 3 | **RUN 2026-08-13 — 12/12 PASS, all 3 real seasons.** `crates/weather-poc/examples/fidelity_probe.rs` (jc as dev-dep) against real live-fetched ARCO-ERA5 grid data (200k pairs/comparison). K×K ρ256 = 0.999909/0.999895/0.999684 — **does not replicate** the earlier smaller-scale near-miss (0.999556); shuffle ρ collapses to 0.02–0.024; ladder strictly monotone all 3 | poc-v2's ρ≥0.98 concern did not fire at grid scale — the earlier near-miss was fixture-scale, not grid-scale | +| D-WXS-8 | cross-variable comparability at grid scale (≥4 variables, ≥2 units, ≥3 seasons); per-variable-floor control must LOSE | 3 | **RUN 2026-08-13 — mixed, reported in full.** Control (per-var must lose) **16/16 PASS** — KILL does not fire, ρ_pervar 0.245–0.939 vs ρ_shared 0.9987–0.9999 on identical pairs. Primary (ρ_shared≥0.9996) **10/16 PASS**, 6 close misses (0.9987–0.9996), skewed toward wind/pressure pairs. Stay-silent twin: `\|diff\|≤0.0001` PASS at spring/summer, FAIL at winter (0.000174); **zero-empty-buckets FAILS all 3 seasons** (38/39/45 of 256) — the literal "zero" from the small-fixture claim does not hold at grid scale | P2 (0.9997 vs 0.857–0.875) **CONFIRMED at grid scale, 3 seasons, 16 pairs** — the directional claim holds; the exact thresholds do not hold universally | | D-WXS-9 | ζ = ∂v/∂x − ∂u/∂y as a substrate read (neighbour-key stencil) + the differencing-amplifies-quantisation falsifier, reported per ζ-magnitude decile | 4 | Queued | bar B8; KILL ⇒ ζ becomes its own baked lane | | D-WXS-10 | **D-CZ-8 re-homed** — ζ-percentile regime bands over the WHOLE grid + coverage-matched donors + range-normalised `L`; shuffled-ζ control; same-band stay-silent twin | 4 | Queued | the grid buys CONTROL of the §7.9 confound, NOT its dissolution; KILL if `L̄`~range ρ≈1.0 survives matching | | D-WXS-11 | MEASURE a full-grid bake wall time; state the ~8.1 s serial extrapolation as the prior and confirm/correct it in the same artifact | 5 | Queued | replaces an extrapolation from ONE cycle (514 ms / 65,536 owners); D-KIA-A2 still unbuilt | diff --git a/.claude/plans/weather-soa-bake-v1.md b/.claude/plans/weather-soa-bake-v1.md index 5a1bf36b..8775e533 100644 --- a/.claude/plans/weather-soa-bake-v1.md +++ b/.claude/plans/weather-soa-bake-v1.md @@ -700,6 +700,122 @@ sequencing note in §4 ("`W0 → W1 → W2 → W3`") is a dependency ordering fo *bake* deliverables, not a licence to treat W3 as unreachable while W1 is incomplete. +#### `D-WXS-7` / `D-WXS-8` — RUN 2026-08-13. Mixed: `D-WXS-7` PASSES cleanly at grid +scale (a prior smaller-scale near-miss does **not** replicate); `D-WXS-8`'s +control (the thesis-level claim) PASSES at every pair; its strict primary and +its stay-silent twin each have real, honest FAILs that qualify but do not +retract the claim. + +**Method, real not simulated.** `probes/weather-p1/fidelity_probe_fetch.py` +live-fetched real ARCO-ERA5 grid chunks (public HTTPS, the `fetch.py`-proven +404=all-NaN=valid-missing-chunk path) at **3 real calendar seasons** found by a +live HEAD-sweep of 24 candidate timesteps across the whole 1959–2023 archive — +**not assumed**: a first attempt at 4 fixed 2021 calendar-season anchors found +only the SAME 3 variables (`2m_temperature`, `2m_dewpoint_temperature`, +`10m_u_component_of_wind`) present at **every one** of them; +`10m_v_component_of_wind` / `mean_sea_level_pressure` / `surface_pressure` / +`total_column_water_vapour` / `total_cloud_cover` / `sea_surface_temperature` +were absent at all 4 — confirming `probes/weather-p1/README.md` §1's +"sparse by design" finding is not a one-timestep artifact. `10m_wind_speed` +does not exist in this store at all, confirming the pre-existing +`weather-normalized-substrate.md` §2 finding. A follow-up HEAD sweep found +**winter** (Dec 1969, 5 vars, 4 units: K/m·s⁻¹/Pa/¹), **spring** (Mar 1970, +4 vars, 3 units: K/m·s⁻¹/kg·m⁻²) and **summer** (Aug 1970, 4 vars, 3 units: +K/m·s⁻¹/¹), each independently meeting bar B7's ≥4-variable/≥2-unit floor with +genuinely present, fully-finite fields (`sea_surface_temperature` came back +**physically masked** — 686,364/1,038,240 finite, land is NaN by definition, +not a missing-chunk case — and was correctly excluded as not-fully-usable +rather than force-included). + +`fidelity_probe_prep.py` quantises via the exact `floor.rs` formula +(re-expressed in Python for the same reason `floor.rs` itself is zero-dep — +this stage fetches over HTTP, which the Rust crate deliberately cannot do) and +writes raw `(truth_distance, code_distance)` f64 pair arrays — **200,000 pairs +per comparison**, matching `p2_probe.py`'s own `N`. `crates/weather-poc/ +examples/fidelity_probe.rs` (jc added as a **dev-dependency only** — see +`Cargo.toml`'s comment on why that is safe, unlike `helix`) reads them and +computes every ρ with **`jc::reliability::spearman`**, per bar B6's own +wording, and applies every verdict exactly as pre-registered above — no +softened bar, no filtered output. Sanity-checked before trusting any number: +the shuffled-decode arm is a genuinely different array from the unshuffled one +(range `[0, 4.92]` vs `[0, 202]`, not a copy or a stub), and its ρ collapses to +`0.02–0.024` — the pipeline demonstrably discriminates. + +**`D-WXS-7` (bar B6) — 12/12 PASS, all three seasons.** The K×K pair (winter, +spring) or its within-variable degenerate twin (summer, where only one K +variable was available): + +| season | ρ(L16) | ρ(L64) | ρ(L256) | ρ(shuffled) | (a) primary ≥0.9996 | (b) shuffle <0.98 | (c) monotone | +|---|---|---|---|---|---|---|---| +| winter | 0.986011 | 0.998926 | **0.999909** | 0.023855 | PASS | PASS | PASS | +| spring | 0.984137 | 0.998792 | **0.999895** | 0.020747 | PASS | PASS | PASS | +| summer | 0.969079 | 0.997344 | **0.999684** | 0.020245 | PASS | PASS | PASS | + +**This directly updates §0.3's own cited evidence, honestly, not silently.** +§0.3 (and this section's own bar text above) cites `p2_probe.py`'s earlier, +smaller-scale finding — *"K×K = 0.999556, below bar"* — as the reason bar +B6(a) "can fire" at all. At real grid scale (200,000 pairs from the full +1,038,240-cell field, 3 independent real seasons, computed with `jc` not +`scipy`), **the K×K pair does not replicate that near-miss** — it clears +0.9996 with margin at all three. Two things can both be true without +contradiction: the bar was correctly falsifiable when written (a real failure +existed to justify it), and the failure does not hold at the scale/instrument +bar B6 actually specifies. The `p2_probe.py` number is not retracted — it +was measured on a real, smaller fixture and is left as-is — but it is no +longer read as "the substrate's fidelity is marginal"; at grid scale, on this +data, it is not. + +**`D-WXS-8` (bar B7) — control 16/16 PASS; primary 10/16 PASS; twin 2/6 PASS.** +Reported in full, nothing filtered: + +- **Control (per-variable floor must LOSE) — 16/16 PASS, at every cross-unit + pair, every season.** Per-variable ρ ranges **0.245–0.939**; shared-floor ρ + is **0.9987–0.9999** on the identical pairs. This is the KILL-gated claim — + *"if the shared floor loses to per-variable on ANY cross-unit pair, §4 + policy (a) is refuted"* — and it does not lose once. **The KILL does not + fire.** +- **Primary (cross-unit ρ_shared ≥ 0.9996) — 10/16 PASS, 6 FAIL.** Every + failure is a close miss, not a collapse: `0.998681`–`0.999611`, all still + ≥ 0.9986 and all still dramatically ahead of per-variable. The 6 failing + pairs skew toward wind and pressure (`10m_u/v_component_of_wind`, + `mean_sea_level_pressure`) rather than temperature — a pattern, not + reported as a proven cause. +- **Stay-silent twin — the two halves diverge, and only one holds.** + `|ρ_shared − ρ_pervar| ≤ 0.0001`: PASS at spring (0.000044) and summer + (0.000025), **FAIL at winter** (0.000174 — 1.7× the tolerance). **Zero + empty buckets: FAIL at all three seasons** (38, 39, 45 of 256 buckets + empty — 15–18%). This is the one finding this run adds that is not merely + "known-losable-control held" — the *literal* "zero empty buckets" half of + the original small-fixture claim (`p2_probe.py`, 1 timestep, 3 variables) + does **not** hold once the shared floor is calibrated across a wider, + real, multi-unit pooled window at grid scale. Read plainly: a + percentile-trimmed shared window necessarily has *some* slack for a single + variable's own narrower distribution — the direction is unsurprising; the + bar's literal "zero" was not re-verified before this run and is now known + to be a real, small, repeated gap rather than a re-confirmed exact zero. + +**What this does and does not license.** `D-WXS-8`'s own pre-registered KILL +(the control losing) is the ONLY clause that would retract the arc's core +product claim, and it did not fire — the shared canonical floor beats +per-variable floors on cross-unit comparability **unambiguously and by a wide +margin**, confirmed at real grid scale across 3 real seasons and 16 real +variable pairs spanning 4 physical units, none of it assumed. What the primary +and twin FAILs correctly block is treating **0.9996 exactly** and **literally +zero empty buckets** as *proven at grid scale for every pair* — they are not, +and no verdict above pretends otherwise. Downstream deliverables that need the +directional claim (shared floor is the right design) may proceed; anything +that would need the *exact* numeric thresholds met on every pair must treat +this as open. + +**Not run this session, named rather than silently deferred:** L64/L16 ladder +for the cross-unit pairs (bar B6's ladder was only computed for the K×K pair); +a fourth+ season; the wind/pressure-skew pattern in the primary FAILs was +observed, not tested as a hypothesis. + +Board: `STATUS_BOARD.md` `D-WXS-7`/`D-WXS-8` rows updated from *READY* to their +real outcomes; `EPIPHANIES.md` prepend recording the K×K non-replication and +the empty-bucket gap. + ### W4 — re-homing `D-CZ-8` (ζ + a range-normalised transfer metric) | D-id | deliverable | diff --git a/.gitignore b/.gitignore index 1c0b3cb5..f2432e1e 100644 --- a/.gitignore +++ b/.gitignore @@ -112,3 +112,15 @@ crates/lance-graph-planner/examples/data/rosetta/* # fresh clone can verify which bytes a recorded result was produced from. probes/weather-p1/fixture/**/*.npy probes/weather-p1/fixture/*.npy +# Same rule, same reason, for D-WXS-7/D-WXS-8's fetch: the store's own +# .zmetadata (fetched fresh by fidelity_probe_fetch.py, 48 KB, deterministic +# from the public store) and the 138 MB of raw pair-distance f64 arrays +# fidelity_probe_prep.py writes. `fidelity_pairs/manifest.json` and the +# per-season `*_meta.txt` sidecars ARE committed (small, the provenance a +# fresh clone needs); `fidelity_probe_availability.json` and +# `fidelity_probe_results.json` ARE committed (the actual measured receipts). +# Re-derive the raw arrays with: +# python3 probes/weather-p1/fidelity_probe_fetch.py +# python3 probes/weather-p1/fidelity_probe_prep.py +probes/weather-p1/zmeta.json +probes/weather-p1/fixture/fidelity_pairs/*.f64 diff --git a/crates/weather-poc/Cargo.lock b/crates/weather-poc/Cargo.lock index fdaedaca..156b1d27 100644 --- a/crates/weather-poc/Cargo.lock +++ b/crates/weather-poc/Cargo.lock @@ -2,6 +2,13 @@ # It is not intended for manual editing. version = 4 +[[package]] +name = "jc" +version = "0.1.0" + [[package]] name = "weather-poc" version = "0.1.0" +dependencies = [ + "jc", +] diff --git a/crates/weather-poc/Cargo.toml b/crates/weather-poc/Cargo.toml index 681d9acf..b95219aa 100644 --- a/crates/weather-poc/Cargo.toml +++ b/crates/weather-poc/Cargo.toml @@ -33,7 +33,24 @@ description = "Zarr -> NodeRow: the ERA5 0.25-degree global grid baked onto the [dependencies] +# ── jc AS A DEV-DEPENDENCY, PATH-SOURCED — different case from `helix` above ── +# Bar B6/B7 (plan sec 4 W3, D-WXS-7/D-WXS-8) require the fidelity metric to be +# "computed WITH jc::reliability::spearman" specifically — jc is the +# operator-named authority for this measurement (`.claude/plans/ +# weather-soa-bake-v1.md` sec 5's classification of `spearman(a,b)` as +# SUBSTRATE). This is safe as a PATH dep, unlike helix, for two reasons: +# 1. It is a dev-dependency: resolved only for `cargo test`/examples, NEVER +# for `cargo build --lib` (the default/production build this crate's +# zero-dep note is actually protecting). +# 2. `jc` is itself genuinely zero-external-dep BY DEFAULT (its own +# Cargo.toml: "Zero EXTERNAL deps in production"; its one path-dep, +# `sigker`, sits behind an opt-in, non-default feature) — so pulling it +# in does not chain into ndarray or any other heavy fork, and does not +# reintroduce the codex-P2/#504 trap (that trap is about an OPTIONAL +# dependency still being read at manifest resolution; a plain +# dev-dependency on an already-zero-dep sibling has no such edge). [dev-dependencies] +jc = { path = "../jc" } [lib] name = "weather_poc" diff --git a/crates/weather-poc/examples/fidelity_probe.rs b/crates/weather-poc/examples/fidelity_probe.rs new file mode 100644 index 00000000..0702ffae --- /dev/null +++ b/crates/weather-poc/examples/fidelity_probe.rs @@ -0,0 +1,308 @@ +//! `D-WXS-7` / `D-WXS-8` — bar B6/B7, RUN. +//! +//! Reads the raw pair-distance arrays [`crate`]'s companion Python stage +//! (`probes/weather-p1/fidelity_probe_fetch.py` + `fidelity_probe_prep.py`) +//! wrote from REAL, live-fetched ARCO-ERA5 grid data at three real seasons, +//! and computes every ρ with [`jc::reliability::spearman`] — per bar B6's own +//! wording, *"computed with `jc::reliability::spearman`"*, not a Python +//! re-implementation. Rust's role here is exactly this: the metric. Fetching +//! real data over HTTP is Python's role (`weather-poc` stays zero-dep in its +//! default `[lib]` build; `jc` is a **dev-dependency only** — see +//! `Cargo.toml`'s comment on why that is safe, unlike `helix`). +//! +//! Bars, verbatim from `.claude/plans/weather-soa-bake-v1.md` sec 4 W3: +//! +//! **Bar B6 (`D-WXS-7`)** — the K×K pair (or its within-variable degenerate +//! twin where only one K variable is available), at each season: +//! - (a) primary: ρ ≥ 0.9996 at resolution 256 +//! - (b) control: the shuffled-decode-table arm must score < 0.98 +//! - (c) can-it-differ: the 16/64/256 ladder must be **strictly monotone**, +//! checked BEFORE any verdict is read +//! - thesis-level KILL (reported, does not block): any arm below ρ ≈ 0.9 +//! +//! **Bar B7 (`D-WXS-8`)** — every cross-variable pair at each season: +//! - primary: cross-UNIT pairs must reach ρ ≥ 0.9996 on the shared floor +//! - control: the per-variable floor must LOSE (lower ρ) on every cross-unit +//! pair — a control that WINS refutes bar B7's premise +//! - stay-silent twin: within-variable, the shared floor must not cost +//! resolution — |ρ_shared − ρ_pervar| ≤ 0.0001 AND zero empty buckets +//! +//! This example does not soften a failing bar. Every verdict below is +//! printed exactly as measured, including any FAIL. + +use std::fs; +use std::path::Path; + +/// Absolute at compile time via `CARGO_MANIFEST_DIR`, not relative to +/// whatever directory `cargo run` happens to be invoked from. +fn pair_dir() -> String { + format!( + "{}/../../probes/weather-p1/fixture/fidelity_pairs", + env!("CARGO_MANIFEST_DIR") + ) +} +const SEASONS: [&str; 3] = ["winter", "spring", "summer"]; +const PRIMARY_RHO: f64 = 0.9996; +const SHUFFLE_CEILING: f64 = 0.98; +const THESIS_KILL_FLOOR: f64 = 0.9; +const TWIN_TOLERANCE: f64 = 0.0001; + +/// Read a flat little-endian `f64` array with no header — the format +/// `fidelity_probe_prep.py::write_f64` emits (`ndarray.astype(' Vec { + let bytes = fs::read(path).unwrap_or_else(|e| panic!("read {}: {e}", path.display())); + assert_eq!( + bytes.len() % 8, + 0, + "{}: not a whole number of f64s", + path.display() + ); + bytes + .chunks_exact(8) + .map(|c| f64::from_le_bytes(c.try_into().unwrap())) + .collect() +} + +fn spearman_or_panic(a: &[f64], b: &[f64], what: &str) -> f64 { + jc::reliability::spearman(a, b) + .unwrap_or_else(|| panic!("{what}: jc::reliability::spearman returned None (length mismatch or non-finite input) — a data-prep bug, not a measurement")) +} + +/// The `key:value` / `cross:key:unit_a:unit_b` sidecar +/// `fidelity_probe_prep.py` writes instead of JSON (see its own comment for +/// why — no parsing crate needed). +struct SeasonMeta { + within_var: String, + within_empty_buckets: u32, + kxk_name: String, + kxk_va: String, + kxk_vb: String, + cross_pairs: Vec<(String, String, String)>, // (pair_key, unit_a, unit_b) +} + +fn read_meta(season: &str) -> SeasonMeta { + let dir = pair_dir(); + let text = fs::read_to_string(format!("{dir}/{season}_meta.txt")) + .unwrap_or_else(|e| panic!("read {season}_meta.txt: {e}")); + let mut within_var = String::new(); + let mut within_empty_buckets = 0u32; + let mut kxk_name = String::new(); + let mut kxk_va = String::new(); + let mut kxk_vb = String::new(); + let mut cross_pairs = Vec::new(); + for line in text.lines() { + let mut parts = line.splitn(4, ':'); + let tag = parts.next().unwrap_or(""); + match tag { + "within_var" => within_var = parts.next().unwrap_or("").to_string(), + "within_empty_buckets" => { + within_empty_buckets = parts.next().unwrap_or("0").parse().unwrap_or(0) + } + "kxk_name" => kxk_name = parts.next().unwrap_or("").to_string(), + "kxk_va" => kxk_va = parts.next().unwrap_or("").to_string(), + "kxk_vb" => kxk_vb = parts.next().unwrap_or("").to_string(), + "cross" => { + let key = parts.next().unwrap_or("").to_string(); + let ua = parts.next().unwrap_or("").to_string(); + let ub = parts.next().unwrap_or("").to_string(); + cross_pairs.push((key, ua, ub)); + } + "" => {} + other => panic!("{season}_meta.txt: unknown tag {other:?}"), + } + } + SeasonMeta { + within_var, + within_empty_buckets, + kxk_name, + kxk_va, + kxk_vb, + cross_pairs, + } +} + +struct Verdict { + name: String, + pass: bool, + detail: String, +} + +fn main() { + let mut all_verdicts: Vec = Vec::new(); + + for season in SEASONS { + println!("\n== {season} =="); + let meta = read_meta(season); + let dir = pair_dir(); + + // ---- Bar B6: the K x K pair (or its within-variable degenerate) ---- + let truth = read_f64(Path::new(&format!("{dir}/{season}_kxk_truth.f64"))); + let rho16 = spearman_or_panic( + &truth, + &read_f64(Path::new(&format!("{dir}/{season}_kxk_code_L16.f64"))), + "kxk L16", + ); + let rho64 = spearman_or_panic( + &truth, + &read_f64(Path::new(&format!("{dir}/{season}_kxk_code_L64.f64"))), + "kxk L64", + ); + let rho256 = spearman_or_panic( + &truth, + &read_f64(Path::new(&format!("{dir}/{season}_kxk_code_L256.f64"))), + "kxk L256", + ); + let rho_shuffled = spearman_or_panic( + &truth, + &read_f64(Path::new(&format!("{dir}/{season}_kxk_code_shuffled.f64"))), + "kxk shuffled", + ); + println!( + " B6 pair: {} (va={} vb={})", + meta.kxk_name, meta.kxk_va, meta.kxk_vb + ); + println!(" rho(L16)={rho16:.6} rho(L64)={rho64:.6} rho(L256)={rho256:.6} rho(shuffled)={rho_shuffled:.6}"); + + let ladder_monotone = rho16 < rho64 && rho64 < rho256; + let v_ladder = Verdict { + name: format!("B6c[{season}] resolution ladder strictly monotone"), + pass: ladder_monotone, + detail: format!( + "rho16={rho16:.6} rho64={rho64:.6} rho256={rho256:.6} (need rho16= {PRIMARY_RHO}"), + pass: rho256 >= PRIMARY_RHO, + detail: format!("rho256={rho256:.6}"), + }; + let v_control = Verdict { + name: format!("B6b[{season}] shuffled control < {SHUFFLE_CEILING}"), + pass: rho_shuffled < SHUFFLE_CEILING, + detail: format!("rho_shuffled={rho_shuffled:.6}"), + }; + let v_thesis = Verdict { + name: format!("B6-thesis-kill[{season}] every arm >= {THESIS_KILL_FLOOR}"), + pass: [rho16, rho64, rho256] + .iter() + .all(|&r| r >= THESIS_KILL_FLOOR), + detail: format!( + "min(rho16,rho64,rho256)={:.6}", + rho16.min(rho64).min(rho256) + ), + }; + for v in [&v_primary, &v_control, &v_thesis] { + println!( + " {}: {} — {}", + v.name, + if v.pass { "PASS" } else { "FAIL" }, + v.detail + ); + } + all_verdicts.push(v_ladder); + all_verdicts.push(v_primary); + all_verdicts.push(v_control); + all_verdicts.push(v_thesis); + + // ---- Bar B7: every cross-variable pair, shared vs per-variable ---- + for (key, unit_a, unit_b) in &meta.cross_pairs { + let truth = read_f64(Path::new(&format!("{dir}/{season}_{key}_truth.f64"))); + let code_shared = read_f64(Path::new(&format!("{dir}/{season}_{key}_code_shared.f64"))); + let code_pervar = read_f64(Path::new(&format!("{dir}/{season}_{key}_code_pervar.f64"))); + let rho_shared = spearman_or_panic(&truth, &code_shared, &format!("{key} shared")); + let rho_pervar = spearman_or_panic(&truth, &code_pervar, &format!("{key} pervar")); + let same_unit = unit_a == unit_b; + println!( + " B7 {key:55} units=({unit_a},{unit_b}) rho_shared={rho_shared:.6} rho_pervar={rho_pervar:.6} {}", + if same_unit { "(same-unit, informational)" } else { "(cross-unit)" } + ); + if !same_unit { + all_verdicts.push(Verdict { + name: format!( + "B7-primary[{season}][{key}] cross-unit rho_shared >= {PRIMARY_RHO}" + ), + pass: rho_shared >= PRIMARY_RHO, + detail: format!("rho_shared={rho_shared:.6} units=({unit_a},{unit_b})"), + }); + all_verdicts.push(Verdict { + name: format!("B7-control[{season}][{key}] per-variable floor LOSES to shared"), + pass: rho_pervar < rho_shared, + detail: format!("rho_shared={rho_shared:.6} rho_pervar={rho_pervar:.6}"), + }); + } + } + + // ---- stay-silent twin: within-variable, shared must not cost ---- + let truth_w = read_f64(Path::new(&format!("{dir}/{season}_within_truth.f64"))); + let code_w_shared = read_f64(Path::new(&format!("{dir}/{season}_within_code_shared.f64"))); + let code_w_pervar = read_f64(Path::new(&format!("{dir}/{season}_within_code_pervar.f64"))); + let rho_w_shared = spearman_or_panic(&truth_w, &code_w_shared, "within shared"); + let rho_w_pervar = spearman_or_panic(&truth_w, &code_w_pervar, "within pervar"); + let diff = (rho_w_shared - rho_w_pervar).abs(); + println!( + " B7 stay-silent twin: within-var={} rho_shared={rho_w_shared:.6} rho_pervar={rho_w_pervar:.6} diff={diff:.6} empty_buckets={}", + meta.within_var, meta.within_empty_buckets + ); + all_verdicts.push(Verdict { + name: format!("B7-twin[{season}] |rho_shared - rho_pervar| <= {TWIN_TOLERANCE}"), + pass: diff <= TWIN_TOLERANCE, + detail: format!( + "rho_shared={rho_w_shared:.6} rho_pervar={rho_w_pervar:.6} diff={diff:.6}" + ), + }); + all_verdicts.push(Verdict { + name: format!("B7-twin[{season}] zero empty buckets under shared floor"), + pass: meta.within_empty_buckets == 0, + detail: format!("empty_buckets={}", meta.within_empty_buckets), + }); + } + + println!("\n== VERDICT SUMMARY (every one, no filtering) =="); + let mut n_pass = 0usize; + for v in &all_verdicts { + println!( + " [{}] {} ({})", + if v.pass { "PASS" } else { "FAIL" }, + v.name, + v.detail + ); + if v.pass { + n_pass += 1; + } + } + println!( + "\n{n_pass}/{} verdicts pass ({} FAIL)", + all_verdicts.len(), + all_verdicts.len() - n_pass + ); + + // Minimal hand-rolled JSON write (no serde — matches the crate's + // zero-dep-by-construction lib; this example already carries the one + // acceptable dev-dependency, jc, and should not add a second). + let mut json = String::from("{\n \"verdicts\": [\n"); + for (i, v) in all_verdicts.iter().enumerate() { + json.push_str(&format!( + " {{\"name\": {:?}, \"pass\": {}, \"detail\": {:?}}}{}\n", + v.name, + v.pass, + v.detail, + if i + 1 < all_verdicts.len() { "," } else { "" } + )); + } + json.push_str(" ]\n}\n"); + fs::write( + format!("{}/../fidelity_probe_results.json", pair_dir()), + &json, + ) + .expect("write fidelity_probe_results.json"); + println!("\nwrote probes/weather-p1/fixture/fidelity_probe_results.json"); +} diff --git a/probes/weather-p1/fidelity_probe_fetch.py b/probes/weather-p1/fidelity_probe_fetch.py new file mode 100644 index 00000000..d0d7f6a9 --- /dev/null +++ b/probes/weather-p1/fidelity_probe_fetch.py @@ -0,0 +1,101 @@ +"""D-WXS-7/D-WXS-8 fetch stage. Extends `fetch.py`'s proven pattern (real +ARCO-ERA5 Zarr v2 HTTP fetch, 404=NaN=valid-missing-chunk semantics) to +grid-scale scope: >=4 variables spanning >=2 distinct units, at each of >=3 +distinct seasons, as bar B7 (`.claude/plans/weather-soa-bake-v1.md` W3) +requires. + +The three (season, timestep, variable-set) triples below are NOT assumed -- +they were found by a live HEAD-request sweep of this script's own store +(probes/weather-p1/README.md section 1's own finding: this store is sparse +by design, and NOT just at one timestep -- an initial attempt at 4 fixed +calendar-season anchors in 2021 found only the SAME 3 variables present at +EVERY one of them; total_column_water_vapour / total_cloud_cover / +sea_surface_temperature / mean_sea_level_pressure / surface_pressure / +10m_v_component_of_wind were absent at all 4. `10m_wind_speed` does not +exist in this store at all -- confirms the pre-existing FINDING in +`.claude/knowledge/weather-normalized-substrate.md` section 2. A follow-up +sweep of 24 candidate timesteps spread across the whole 1959-2023 archive +found these three, each independently satisfying >=4 variables / >=2 units, +in three genuinely different calendar seasons.""" +import json, urllib.request, numpy as np, numcodecs, datetime as dt, os + +B = "https://storage.googleapis.com/gcp-public-data-arco-era5/ar/1959-2023_01_10-full_37-1h-0p25deg-chunk-1.zarr" +meta = json.load(open('zmeta.json'))['metadata'] +EPOCH = dt.datetime(1959, 1, 1) + +# (season_label, time_index, {variable: unit}) -- each triple confirmed +# present by a real HEAD sweep before this script existed (see module doc). +SEASONS = { + 'winter': (96000, { + '2m_temperature': 'K', '2m_dewpoint_temperature': 'K', + '10m_u_component_of_wind': 'm/s', + 'mean_sea_level_pressure': 'Pa', 'total_cloud_cover': '1', + }), + 'spring': (98000, { + '2m_temperature': 'K', '2m_dewpoint_temperature': 'K', + '10m_u_component_of_wind': 'm/s', + 'total_column_water_vapour': 'kg/m2', 'sea_surface_temperature': 'K', + }), + 'summer': (102000, { + '2m_temperature': 'K', + '10m_u_component_of_wind': 'm/s', '10m_v_component_of_wind': 'm/s', + 'total_cloud_cover': '1', 'sea_surface_temperature': 'K', + }), +} + + +def get(var, t): + """One (var, t) chunk. 404 -> all-fill (NaN here); valid Zarr v2 + semantics for a missing chunk, not a fetch failure (codex P2 on #920, + documented in probes/weather-p1/README.md section 1).""" + za = meta[f'{var}/.zarray'] + comp = za['compressor'] + key = f'{var}/{t}.0.0' + req = urllib.request.Request(f'{B}/{key}') + try: + raw = urllib.request.urlopen(req, timeout=300).read() + except urllib.error.HTTPError as e: + if e.code != 404: + raise + fill = za.get('fill_value') + fill = np.nan if fill is None else fill + a = np.full(za['chunks'][1:], fill, dtype=za['dtype']) + return a, 0 + dec = numcodecs.get_codec(comp).decode(raw) + a = np.frombuffer(dec, dtype=za['dtype']).reshape(za['chunks'])[0] + return a, len(raw) + + +def main(): + os.makedirs('fixture', exist_ok=True) + report = {} + for season, (t, varmap) in SEASONS.items(): + target = (EPOCH + dt.timedelta(hours=t)).isoformat() + os.makedirs(f'fixture/season_{season}', exist_ok=True) + report[season] = {'target_utc': target, 'time_index': t, 'vars': {}} + n_finite_units = set() + for var, unit in varmap.items(): + a, ncomp = get(var, t) + finite = np.isfinite(a) + n_finite = int(finite.sum()) + usable = n_finite == a.size + status = 'usable' if usable else ('partial' if n_finite > 0 else 'absent') + report[season]['vars'][var] = { + 'unit': unit, 'compressed_bytes': ncomp, 'n_finite': n_finite, + 'total': int(a.size), 'status': status, + } + if usable: + np.save(f'fixture/season_{season}/{var}.npy', a.astype(np.float32)) + n_finite_units.add(unit) + print(f"{season:8s} {var:28s} unit={unit:6s} status={status:8s} " + f"n_finite={n_finite:>8,}/{a.size:>8,} compressed={ncomp:>9,}B") + n_usable = sum(1 for v in report[season]['vars'].values() if v['status'] == 'usable') + print(f" -> {season}: {n_usable} usable vars, {len(n_finite_units)} distinct units " + f"(bar B7 needs >=4 vars, >=2 units) " + f"{'OK' if n_usable >= 4 and len(n_finite_units) >= 2 else 'SHORT'}\n") + json.dump(report, open('fixture/fidelity_probe_availability.json', 'w'), indent=2) + print("wrote fixture/fidelity_probe_availability.json") + + +if __name__ == '__main__': + main() diff --git a/probes/weather-p1/fidelity_probe_prep.py b/probes/weather-p1/fidelity_probe_prep.py new file mode 100644 index 00000000..43e7bece --- /dev/null +++ b/probes/weather-p1/fidelity_probe_prep.py @@ -0,0 +1,214 @@ +"""D-WXS-7/D-WXS-8 prep stage: quantize the fetched grid-scale fields and +write raw (truth_distance, code_distance) pair arrays for the Rust compute +stage (`crates/weather-poc/examples/fidelity_probe.rs`) to score with +`jc::reliability::spearman` -- per bar B6 (`.claude/plans/weather-soa-bake-v1.md` +W3), the metric must be COMPUTED WITH jc, not with scipy; this stage only +prepares the arrays. + +Quantization formula matches `crates/weather-poc/src/floor.rs::CalibratedFloor` +exactly (itself a documented re-expression of `helix::quantize::RollingFloor`): +N uniform buckets over a percentile-trimmed [lo, hi] window, decode to bucket +centre. Re-implemented here in Python for the SAME reason `floor.rs` is +zero-dep by construction (`.claude/plans/weather-soa-bake-v1.md` sec 2.3) -- +this stage fetches real data over HTTP, which the Rust crate deliberately +cannot do, so the two languages meet at a data file, not a dependency edge +(the D-WXS-12 shape: parity is measured, not imported). + +Standardization (z = (anom - mean) / std) matches probes/weather-p1/p2_probe.py +exactly, extended from 1 timestep/3 variables to 3 REAL seasons (see +fidelity_probe_fetch.py's module doc for how those seasons and their variable +sets were found by a live availability sweep, not assumed).""" +import json +import struct +import numpy as np + +RNG_SEED = 20260813 # pre-registered, distinct from D-CZ's 20260812 +LO_PCT, HI_PCT = 0.4, 99.6 # matches floor.rs LO_PERCENTILE/HI_PERCENTILE +N_PAIRS = 200_000 # matches p2_probe.py's N +LADDER = [16, 64, 256] # bar B6(c)'s resolution ladder + +SEASONS = { + 'winter': ['2m_temperature', '2m_dewpoint_temperature', '10m_u_component_of_wind', + 'mean_sea_level_pressure', 'total_cloud_cover'], + 'spring': ['2m_temperature', '2m_dewpoint_temperature', '10m_u_component_of_wind', + 'total_column_water_vapour'], + 'summer': ['2m_temperature', '10m_u_component_of_wind', '10m_v_component_of_wind', + 'total_cloud_cover'], +} +UNITS = { + '2m_temperature': 'K', '2m_dewpoint_temperature': 'K', 'sea_surface_temperature': 'K', + '10m_u_component_of_wind': 'm/s', '10m_v_component_of_wind': 'm/s', + 'mean_sea_level_pressure': 'Pa', 'total_column_water_vapour': 'kg/m2', + 'total_cloud_cover': '1', +} + + +def load_z(season, var): + """Load a fetched field, return its z-standardized flat array (anomaly / + std, matching p2_probe.py's transform -- identity-after-scaling, §12.1 + measured arctanh as actively harmful on this bell-shaped anomaly shape).""" + a = np.load(f'fixture/season_{season}/{var}.npy').astype(np.float64).ravel() + assert np.isfinite(a).all(), f"{season}/{var}: caller must pre-filter to usable fields" + z = (a - a.mean()) / a.std() + return z + + +def quantize(z, lo, hi, n_levels, decode_perm=None): + """CalibratedFloor::quantize/bucket_center, re-expressed. `decode_perm` + is a permutation applied to the CENTRES table (not the index assignment) + -- the "shuffled decode table" bar B6(b) names: same bucket a value lands + in, but a scrambled mapping from bucket -> represented value. Returns + (index_array_u16, decoded_value_array_f64).""" + lo, hi = float(lo), float(hi) + if hi <= lo: + hi = lo + 1e-9 + width = (hi - lo) / n_levels + idx = np.clip(((z - lo) / width).astype(np.int64), 0, n_levels - 1) + centres = lo + (np.arange(n_levels) + 0.5) * width + if decode_perm is not None: + centres = centres[decode_perm] + return idx, centres[idx] + + +def percentile_window(z): + lo, hi = np.percentile(z, [LO_PCT, HI_PCT]) + return float(lo), float(hi) + + +def write_f64(path, arr): + """Flat little-endian f64, no header -- the Rust reader is a std-only + `f64::from_le_bytes` loop, matching the crate's zero-dep philosophy.""" + arr.astype('= 2: + va, vb = k_vars[0], k_vars[1] + pair_name = f"{va} x {vb} (K x K)" + else: + va = vb = k_vars[0] if k_vars else varlist[0] + pair_name = f"{va} (within, only-K-available)" + + ia = rng.integers(0, n_cells, N_PAIRS) + ib = rng.integers(0, n_cells, N_PAIRS) + truth = np.abs(z[va][ia] - z[vb][ib]) + write_f64(f"{out_dir}/{season}_kxk_truth.f64", truth) + for n_levels in LADDER: + idx_a, _ = quantize(z[va], shared_lo, shared_hi, n_levels) + idx_b, _ = quantize(z[vb], shared_lo, shared_hi, n_levels) + code = np.abs(idx_a[ia].astype(np.float64) - idx_b[ib].astype(np.float64)) + write_f64(f"{out_dir}/{season}_kxk_code_L{n_levels}.f64", code) + # shuffle control at the top resolution (256): permute the DECODE + # table (not the index assignment), decode both sides through it, + # take |decoded_a - decoded_b| as the "code distance" a reader with + # a scrambled codebook would compute. + perm = rng.permutation(256) + _, dec_a = quantize(z[va], shared_lo, shared_hi, 256, decode_perm=perm) + _, dec_b = quantize(z[vb], shared_lo, shared_hi, 256, decode_perm=perm) + shuffled_code = np.abs(dec_a[ia] - dec_b[ib]) + write_f64(f"{out_dir}/{season}_kxk_code_shuffled.f64", shuffled_code) + season_manifest['pairs']['kxk'] = {'name': pair_name, 'va': va, 'vb': vb} + + # ---- bar B7: every cross-variable pair, shared vs per-variable floor, + # at resolution 256 (the shipped resolution) ---- + cross_pairs = {} + for i, va2 in enumerate(varlist): + for vb2 in varlist[i + 1:]: + key = f"{va2}_x_{vb2}" + ia2 = rng.integers(0, n_cells, N_PAIRS) + ib2 = rng.integers(0, n_cells, N_PAIRS) + truth2 = np.abs(z[va2][ia2] - z[vb2][ib2]) + write_f64(f"{out_dir}/{season}_{key}_truth.f64", truth2) + + idx_a_sh, _ = quantize(z[va2], shared_lo, shared_hi, 256) + idx_b_sh, _ = quantize(z[vb2], shared_lo, shared_hi, 256) + code_sh = np.abs(idx_a_sh[ia2].astype(np.float64) - idx_b_sh[ib2].astype(np.float64)) + write_f64(f"{out_dir}/{season}_{key}_code_shared.f64", code_sh) + + lo_a, hi_a = per_var_window[va2] + lo_b, hi_b = per_var_window[vb2] + idx_a_pv, _ = quantize(z[va2], lo_a, hi_a, 256) + idx_b_pv, _ = quantize(z[vb2], lo_b, hi_b, 256) + code_pv = np.abs(idx_a_pv[ia2].astype(np.float64) - idx_b_pv[ib2].astype(np.float64)) + write_f64(f"{out_dir}/{season}_{key}_code_pervar.f64", code_pv) + + cross_pairs[key] = {'va': va2, 'vb': vb2, 'unit_va': UNITS[va2], 'unit_vb': UNITS[vb2], + 'same_unit': UNITS[va2] == UNITS[vb2]} + print(f" cross pair {va2:28s} x {vb2:28s} units=({UNITS[va2]},{UNITS[vb2]})") + season_manifest['pairs']['cross'] = cross_pairs + + # ---- stay-silent twin: within-variable, shared floor must not cost + # resolution vs per-variable floor. One representative variable + # (the first in varlist) at resolution 256. ---- + v0 = varlist[0] + truth_w = np.abs(z[v0][ia_w] - z[v0][ib_w]) + write_f64(f"{out_dir}/{season}_within_truth.f64", truth_w) + idx_sh, _ = quantize(z[v0], shared_lo, shared_hi, 256) + code_w_sh = np.abs(idx_sh[ia_w].astype(np.float64) - idx_sh[ib_w].astype(np.float64)) + write_f64(f"{out_dir}/{season}_within_code_shared.f64", code_w_sh) + lo0, hi0 = per_var_window[v0] + idx_pv0, _ = quantize(z[v0], lo0, hi0, 256) + code_w_pv = np.abs(idx_pv0[ia_w].astype(np.float64) - idx_pv0[ib_w].astype(np.float64)) + write_f64(f"{out_dir}/{season}_within_code_pervar.f64", code_w_pv) + # empty-bucket count under the shared floor, for this variable + occ = np.bincount(idx_sh, minlength=256) + season_manifest['within_variable_control'] = { + 'var': v0, 'shared_empty_buckets': int((occ == 0).sum()), + } + + manifest['seasons'][season] = season_manifest + + # Plain key:value sidecar for the Rust reader -- deliberately NOT + # JSON, so the Rust example needs no parsing crate (matches the + # crate's zero-dep-by-construction philosophy; only `jc` is added, + # as a dev-dependency, for the one thing Rust must do: compute + # Spearman WITH jc, per bar B6's own wording). + with open(f'{out_dir}/{season}_meta.txt', 'w') as f: + f.write(f"within_var:{v0}\n") + f.write(f"within_empty_buckets:{season_manifest['within_variable_control']['shared_empty_buckets']}\n") + f.write(f"kxk_name:{pair_name}\n") + f.write(f"kxk_va:{va}\n") + f.write(f"kxk_vb:{vb}\n") + for key, info in cross_pairs.items(): + f.write(f"cross:{key}:{info['unit_va']}:{info['unit_vb']}\n") + + json.dump(manifest, open(f'{out_dir}/manifest.json', 'w'), indent=2) + print(f"\nwrote {out_dir}/manifest.json + raw f64 pair arrays + per-season meta.txt") + + +if __name__ == '__main__': + main() diff --git a/probes/weather-p1/fixture/fidelity_pairs/manifest.json b/probes/weather-p1/fixture/fidelity_pairs/manifest.json new file mode 100644 index 00000000..a680a32c --- /dev/null +++ b/probes/weather-p1/fixture/fidelity_pairs/manifest.json @@ -0,0 +1,246 @@ +{ + "seed": 20260813, + "n_pairs": 200000, + "ladder": [ + 16, + 64, + 256 + ], + "lo_percentile": 0.4, + "hi_percentile": 99.6, + "seasons": { + "winter": { + "n_cells": 1038240, + "units": { + "2m_temperature": "K", + "2m_dewpoint_temperature": "K", + "10m_u_component_of_wind": "m/s", + "mean_sea_level_pressure": "Pa", + "total_cloud_cover": "1" + }, + "shared_window": [ + -2.5779856641839727, + 2.3637323027935224 + ], + "pairs": { + "kxk": { + "name": "2m_temperature x 2m_dewpoint_temperature (K x K)", + "va": "2m_temperature", + "vb": "2m_dewpoint_temperature" + }, + "cross": { + "2m_temperature_x_2m_dewpoint_temperature": { + "va": "2m_temperature", + "vb": "2m_dewpoint_temperature", + "unit_va": "K", + "unit_vb": "K", + "same_unit": true + }, + "2m_temperature_x_10m_u_component_of_wind": { + "va": "2m_temperature", + "vb": "10m_u_component_of_wind", + "unit_va": "K", + "unit_vb": "m/s", + "same_unit": false + }, + "2m_temperature_x_mean_sea_level_pressure": { + "va": "2m_temperature", + "vb": "mean_sea_level_pressure", + "unit_va": "K", + "unit_vb": "Pa", + "same_unit": false + }, + "2m_temperature_x_total_cloud_cover": { + "va": "2m_temperature", + "vb": "total_cloud_cover", + "unit_va": "K", + "unit_vb": "1", + "same_unit": false + }, + "2m_dewpoint_temperature_x_10m_u_component_of_wind": { + "va": "2m_dewpoint_temperature", + "vb": "10m_u_component_of_wind", + "unit_va": "K", + "unit_vb": "m/s", + "same_unit": false + }, + "2m_dewpoint_temperature_x_mean_sea_level_pressure": { + "va": "2m_dewpoint_temperature", + "vb": "mean_sea_level_pressure", + "unit_va": "K", + "unit_vb": "Pa", + "same_unit": false + }, + "2m_dewpoint_temperature_x_total_cloud_cover": { + "va": "2m_dewpoint_temperature", + "vb": "total_cloud_cover", + "unit_va": "K", + "unit_vb": "1", + "same_unit": false + }, + "10m_u_component_of_wind_x_mean_sea_level_pressure": { + "va": "10m_u_component_of_wind", + "vb": "mean_sea_level_pressure", + "unit_va": "m/s", + "unit_vb": "Pa", + "same_unit": false + }, + "10m_u_component_of_wind_x_total_cloud_cover": { + "va": "10m_u_component_of_wind", + "vb": "total_cloud_cover", + "unit_va": "m/s", + "unit_vb": "1", + "same_unit": false + }, + "mean_sea_level_pressure_x_total_cloud_cover": { + "va": "mean_sea_level_pressure", + "vb": "total_cloud_cover", + "unit_va": "Pa", + "unit_vb": "1", + "same_unit": false + } + } + }, + "within_variable_control": { + "var": "2m_temperature", + "shared_empty_buckets": 38 + } + }, + "spring": { + "n_cells": 1038240, + "units": { + "2m_temperature": "K", + "2m_dewpoint_temperature": "K", + "10m_u_component_of_wind": "m/s", + "total_column_water_vapour": "kg/m2" + }, + "shared_window": [ + -2.427651942113806, + 2.578200723629849 + ], + "pairs": { + "kxk": { + "name": "2m_temperature x 2m_dewpoint_temperature (K x K)", + "va": "2m_temperature", + "vb": "2m_dewpoint_temperature" + }, + "cross": { + "2m_temperature_x_2m_dewpoint_temperature": { + "va": "2m_temperature", + "vb": "2m_dewpoint_temperature", + "unit_va": "K", + "unit_vb": "K", + "same_unit": true + }, + "2m_temperature_x_10m_u_component_of_wind": { + "va": "2m_temperature", + "vb": "10m_u_component_of_wind", + "unit_va": "K", + "unit_vb": "m/s", + "same_unit": false + }, + "2m_temperature_x_total_column_water_vapour": { + "va": "2m_temperature", + "vb": "total_column_water_vapour", + "unit_va": "K", + "unit_vb": "kg/m2", + "same_unit": false + }, + "2m_dewpoint_temperature_x_10m_u_component_of_wind": { + "va": "2m_dewpoint_temperature", + "vb": "10m_u_component_of_wind", + "unit_va": "K", + "unit_vb": "m/s", + "same_unit": false + }, + "2m_dewpoint_temperature_x_total_column_water_vapour": { + "va": "2m_dewpoint_temperature", + "vb": "total_column_water_vapour", + "unit_va": "K", + "unit_vb": "kg/m2", + "same_unit": false + }, + "10m_u_component_of_wind_x_total_column_water_vapour": { + "va": "10m_u_component_of_wind", + "vb": "total_column_water_vapour", + "unit_va": "m/s", + "unit_vb": "kg/m2", + "same_unit": false + } + } + }, + "within_variable_control": { + "var": "2m_temperature", + "shared_empty_buckets": 39 + } + }, + "summer": { + "n_cells": 1038240, + "units": { + "2m_temperature": "K", + "10m_u_component_of_wind": "m/s", + "10m_v_component_of_wind": "m/s", + "total_cloud_cover": "1" + }, + "shared_window": [ + -3.0595366003914215, + 2.5050001104258226 + ], + "pairs": { + "kxk": { + "name": "2m_temperature (within, only-K-available)", + "va": "2m_temperature", + "vb": "2m_temperature" + }, + "cross": { + "2m_temperature_x_10m_u_component_of_wind": { + "va": "2m_temperature", + "vb": "10m_u_component_of_wind", + "unit_va": "K", + "unit_vb": "m/s", + "same_unit": false + }, + "2m_temperature_x_10m_v_component_of_wind": { + "va": "2m_temperature", + "vb": "10m_v_component_of_wind", + "unit_va": "K", + "unit_vb": "m/s", + "same_unit": false + }, + "2m_temperature_x_total_cloud_cover": { + "va": "2m_temperature", + "vb": "total_cloud_cover", + "unit_va": "K", + "unit_vb": "1", + "same_unit": false + }, + "10m_u_component_of_wind_x_10m_v_component_of_wind": { + "va": "10m_u_component_of_wind", + "vb": "10m_v_component_of_wind", + "unit_va": "m/s", + "unit_vb": "m/s", + "same_unit": true + }, + "10m_u_component_of_wind_x_total_cloud_cover": { + "va": "10m_u_component_of_wind", + "vb": "total_cloud_cover", + "unit_va": "m/s", + "unit_vb": "1", + "same_unit": false + }, + "10m_v_component_of_wind_x_total_cloud_cover": { + "va": "10m_v_component_of_wind", + "vb": "total_cloud_cover", + "unit_va": "m/s", + "unit_vb": "1", + "same_unit": false + } + } + }, + "within_variable_control": { + "var": "2m_temperature", + "shared_empty_buckets": 45 + } + } + } +} \ No newline at end of file diff --git a/probes/weather-p1/fixture/fidelity_pairs/spring_meta.txt b/probes/weather-p1/fixture/fidelity_pairs/spring_meta.txt new file mode 100644 index 00000000..a9b7906c --- /dev/null +++ b/probes/weather-p1/fixture/fidelity_pairs/spring_meta.txt @@ -0,0 +1,11 @@ +within_var:2m_temperature +within_empty_buckets:39 +kxk_name:2m_temperature x 2m_dewpoint_temperature (K x K) +kxk_va:2m_temperature +kxk_vb:2m_dewpoint_temperature +cross:2m_temperature_x_2m_dewpoint_temperature:K:K +cross:2m_temperature_x_10m_u_component_of_wind:K:m/s +cross:2m_temperature_x_total_column_water_vapour:K:kg/m2 +cross:2m_dewpoint_temperature_x_10m_u_component_of_wind:K:m/s +cross:2m_dewpoint_temperature_x_total_column_water_vapour:K:kg/m2 +cross:10m_u_component_of_wind_x_total_column_water_vapour:m/s:kg/m2 diff --git a/probes/weather-p1/fixture/fidelity_pairs/summer_meta.txt b/probes/weather-p1/fixture/fidelity_pairs/summer_meta.txt new file mode 100644 index 00000000..351dcff0 --- /dev/null +++ b/probes/weather-p1/fixture/fidelity_pairs/summer_meta.txt @@ -0,0 +1,11 @@ +within_var:2m_temperature +within_empty_buckets:45 +kxk_name:2m_temperature (within, only-K-available) +kxk_va:2m_temperature +kxk_vb:2m_temperature +cross:2m_temperature_x_10m_u_component_of_wind:K:m/s +cross:2m_temperature_x_10m_v_component_of_wind:K:m/s +cross:2m_temperature_x_total_cloud_cover:K:1 +cross:10m_u_component_of_wind_x_10m_v_component_of_wind:m/s:m/s +cross:10m_u_component_of_wind_x_total_cloud_cover:m/s:1 +cross:10m_v_component_of_wind_x_total_cloud_cover:m/s:1 diff --git a/probes/weather-p1/fixture/fidelity_pairs/winter_meta.txt b/probes/weather-p1/fixture/fidelity_pairs/winter_meta.txt new file mode 100644 index 00000000..b1dbdb50 --- /dev/null +++ b/probes/weather-p1/fixture/fidelity_pairs/winter_meta.txt @@ -0,0 +1,15 @@ +within_var:2m_temperature +within_empty_buckets:38 +kxk_name:2m_temperature x 2m_dewpoint_temperature (K x K) +kxk_va:2m_temperature +kxk_vb:2m_dewpoint_temperature +cross:2m_temperature_x_2m_dewpoint_temperature:K:K +cross:2m_temperature_x_10m_u_component_of_wind:K:m/s +cross:2m_temperature_x_mean_sea_level_pressure:K:Pa +cross:2m_temperature_x_total_cloud_cover:K:1 +cross:2m_dewpoint_temperature_x_10m_u_component_of_wind:K:m/s +cross:2m_dewpoint_temperature_x_mean_sea_level_pressure:K:Pa +cross:2m_dewpoint_temperature_x_total_cloud_cover:K:1 +cross:10m_u_component_of_wind_x_mean_sea_level_pressure:m/s:Pa +cross:10m_u_component_of_wind_x_total_cloud_cover:m/s:1 +cross:mean_sea_level_pressure_x_total_cloud_cover:Pa:1 diff --git a/probes/weather-p1/fixture/fidelity_probe_availability.json b/probes/weather-p1/fixture/fidelity_probe_availability.json new file mode 100644 index 00000000..02a38e91 --- /dev/null +++ b/probes/weather-p1/fixture/fidelity_probe_availability.json @@ -0,0 +1,125 @@ +{ + "winter": { + "target_utc": "1969-12-14T00:00:00", + "time_index": 96000, + "vars": { + "2m_temperature": { + "unit": "K", + "compressed_bytes": 2337482, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "2m_dewpoint_temperature": { + "unit": "K", + "compressed_bytes": 2378237, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "10m_u_component_of_wind": { + "unit": "m/s", + "compressed_bytes": 3321010, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "mean_sea_level_pressure": { + "unit": "Pa", + "compressed_bytes": 2183019, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "total_cloud_cover": { + "unit": "1", + "compressed_bytes": 2586061, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + } + } + }, + "spring": { + "target_utc": "1970-03-07T08:00:00", + "time_index": 98000, + "vars": { + "2m_temperature": { + "unit": "K", + "compressed_bytes": 2341435, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "2m_dewpoint_temperature": { + "unit": "K", + "compressed_bytes": 2381534, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "10m_u_component_of_wind": { + "unit": "m/s", + "compressed_bytes": 3325465, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "total_column_water_vapour": { + "unit": "kg/m2", + "compressed_bytes": 3033384, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "sea_surface_temperature": { + "unit": "K", + "compressed_bytes": 1351901, + "n_finite": 686364, + "total": 1038240, + "status": "partial" + } + } + }, + "summer": { + "target_utc": "1970-08-21T00:00:00", + "time_index": 102000, + "vars": { + "2m_temperature": { + "unit": "K", + "compressed_bytes": 2324886, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "10m_u_component_of_wind": { + "unit": "m/s", + "compressed_bytes": 3329793, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "10m_v_component_of_wind": { + "unit": "m/s", + "compressed_bytes": 3351924, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "total_cloud_cover": { + "unit": "1", + "compressed_bytes": 2536586, + "n_finite": 1038240, + "total": 1038240, + "status": "usable" + }, + "sea_surface_temperature": { + "unit": "K", + "compressed_bytes": 1313463, + "n_finite": 686364, + "total": 1038240, + "status": "partial" + } + } + } +} \ No newline at end of file diff --git a/probes/weather-p1/fixture/fidelity_probe_results.json b/probes/weather-p1/fixture/fidelity_probe_results.json new file mode 100644 index 00000000..693527f0 --- /dev/null +++ b/probes/weather-p1/fixture/fidelity_probe_results.json @@ -0,0 +1,60 @@ +{ + "verdicts": [ + {"name": "B6c[winter] resolution ladder strictly monotone", "pass": true, "detail": "rho16=0.986011 rho64=0.998926 rho256=0.999909 (need rho16= 0.9996", "pass": true, "detail": "rho256=0.999909"}, + {"name": "B6b[winter] shuffled control < 0.98", "pass": true, "detail": "rho_shuffled=0.023855"}, + {"name": "B6-thesis-kill[winter] every arm >= 0.9", "pass": true, "detail": "min(rho16,rho64,rho256)=0.986011"}, + {"name": "B7-primary[winter][2m_temperature_x_10m_u_component_of_wind] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999171 units=(K,m/s)"}, + {"name": "B7-control[winter][2m_temperature_x_10m_u_component_of_wind] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999171 rho_pervar=0.597928"}, + {"name": "B7-primary[winter][2m_temperature_x_mean_sea_level_pressure] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999093 units=(K,Pa)"}, + {"name": "B7-control[winter][2m_temperature_x_mean_sea_level_pressure] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999093 rho_pervar=0.817051"}, + {"name": "B7-primary[winter][2m_temperature_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999909 units=(K,1)"}, + {"name": "B7-control[winter][2m_temperature_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999909 rho_pervar=0.939481"}, + {"name": "B7-primary[winter][2m_dewpoint_temperature_x_10m_u_component_of_wind] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999203 units=(K,m/s)"}, + {"name": "B7-control[winter][2m_dewpoint_temperature_x_10m_u_component_of_wind] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999203 rho_pervar=0.587247"}, + {"name": "B7-primary[winter][2m_dewpoint_temperature_x_mean_sea_level_pressure] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999167 units=(K,Pa)"}, + {"name": "B7-control[winter][2m_dewpoint_temperature_x_mean_sea_level_pressure] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999167 rho_pervar=0.806954"}, + {"name": "B7-primary[winter][2m_dewpoint_temperature_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999907 units=(K,1)"}, + {"name": "B7-control[winter][2m_dewpoint_temperature_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999907 rho_pervar=0.941228"}, + {"name": "B7-primary[winter][10m_u_component_of_wind_x_mean_sea_level_pressure] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.998681 units=(m/s,Pa)"}, + {"name": "B7-control[winter][10m_u_component_of_wind_x_mean_sea_level_pressure] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.998681 rho_pervar=0.751437"}, + {"name": "B7-primary[winter][10m_u_component_of_wind_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999211 units=(m/s,1)"}, + {"name": "B7-control[winter][10m_u_component_of_wind_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999211 rho_pervar=0.551341"}, + {"name": "B7-primary[winter][mean_sea_level_pressure_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999150 units=(Pa,1)"}, + {"name": "B7-control[winter][mean_sea_level_pressure_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999150 rho_pervar=0.708683"}, + {"name": "B7-twin[winter] |rho_shared - rho_pervar| <= 0.0001", "pass": false, "detail": "rho_shared=0.999916 rho_pervar=0.999742 diff=0.000174"}, + {"name": "B7-twin[winter] zero empty buckets under shared floor", "pass": false, "detail": "empty_buckets=38"}, + {"name": "B6c[spring] resolution ladder strictly monotone", "pass": true, "detail": "rho16=0.984137 rho64=0.998792 rho256=0.999895 (need rho16= 0.9996", "pass": true, "detail": "rho256=0.999895"}, + {"name": "B6b[spring] shuffled control < 0.98", "pass": true, "detail": "rho_shuffled=0.020747"}, + {"name": "B6-thesis-kill[spring] every arm >= 0.9", "pass": true, "detail": "min(rho16,rho64,rho256)=0.984137"}, + {"name": "B7-primary[spring][2m_temperature_x_10m_u_component_of_wind] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999611 units=(K,m/s)"}, + {"name": "B7-control[spring][2m_temperature_x_10m_u_component_of_wind] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999611 rho_pervar=0.559432"}, + {"name": "B7-primary[spring][2m_temperature_x_total_column_water_vapour] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999903 units=(K,kg/m2)"}, + {"name": "B7-control[spring][2m_temperature_x_total_column_water_vapour] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999903 rho_pervar=0.309949"}, + {"name": "B7-primary[spring][2m_dewpoint_temperature_x_10m_u_component_of_wind] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999655 units=(K,m/s)"}, + {"name": "B7-control[spring][2m_dewpoint_temperature_x_10m_u_component_of_wind] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999655 rho_pervar=0.448606"}, + {"name": "B7-primary[spring][2m_dewpoint_temperature_x_total_column_water_vapour] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999896 units=(K,kg/m2)"}, + {"name": "B7-control[spring][2m_dewpoint_temperature_x_total_column_water_vapour] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999896 rho_pervar=0.245867"}, + {"name": "B7-primary[spring][10m_u_component_of_wind_x_total_column_water_vapour] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999572 units=(m/s,kg/m2)"}, + {"name": "B7-control[spring][10m_u_component_of_wind_x_total_column_water_vapour] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999572 rho_pervar=0.659447"}, + {"name": "B7-twin[spring] |rho_shared - rho_pervar| <= 0.0001", "pass": true, "detail": "rho_shared=0.999899 rho_pervar=0.999855 diff=0.000044"}, + {"name": "B7-twin[spring] zero empty buckets under shared floor", "pass": false, "detail": "empty_buckets=39"}, + {"name": "B6c[summer] resolution ladder strictly monotone", "pass": true, "detail": "rho16=0.969079 rho64=0.997344 rho256=0.999684 (need rho16= 0.9996", "pass": true, "detail": "rho256=0.999684"}, + {"name": "B6b[summer] shuffled control < 0.98", "pass": true, "detail": "rho_shuffled=0.020245"}, + {"name": "B6-thesis-kill[summer] every arm >= 0.9", "pass": true, "detail": "min(rho16,rho64,rho256)=0.969079"}, + {"name": "B7-primary[summer][2m_temperature_x_10m_u_component_of_wind] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999652 units=(K,m/s)"}, + {"name": "B7-control[summer][2m_temperature_x_10m_u_component_of_wind] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999652 rho_pervar=0.362330"}, + {"name": "B7-primary[summer][2m_temperature_x_10m_v_component_of_wind] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999591 units=(K,m/s)"}, + {"name": "B7-control[summer][2m_temperature_x_10m_v_component_of_wind] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999591 rho_pervar=0.461251"}, + {"name": "B7-primary[summer][2m_temperature_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999824 units=(K,1)"}, + {"name": "B7-control[summer][2m_temperature_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999824 rho_pervar=0.924452"}, + {"name": "B7-primary[summer][10m_u_component_of_wind_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": true, "detail": "rho_shared=0.999613 units=(m/s,1)"}, + {"name": "B7-control[summer][10m_u_component_of_wind_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999613 rho_pervar=0.620800"}, + {"name": "B7-primary[summer][10m_v_component_of_wind_x_total_cloud_cover] cross-unit rho_shared >= 0.9996", "pass": false, "detail": "rho_shared=0.999570 units=(m/s,1)"}, + {"name": "B7-control[summer][10m_v_component_of_wind_x_total_cloud_cover] per-variable floor LOSES to shared", "pass": true, "detail": "rho_shared=0.999570 rho_pervar=0.686075"}, + {"name": "B7-twin[summer] |rho_shared - rho_pervar| <= 0.0001", "pass": true, "detail": "rho_shared=0.999695 rho_pervar=0.999719 diff=0.000025"}, + {"name": "B7-twin[summer] zero empty buckets under shared floor", "pass": false, "detail": "empty_buckets=45"} + ] +}