Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude/board/STATUS_BOARD.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ with a control that can lose and a stay-silent twin.
| D-WXS-1 | field manifest v1 — (facet, pair, byte) → (variable, level, unit, floor id), a committed data artifact, ClassView-side | 0 | **SHIPPED 2026-08-13** — `data/field_manifest_v1.tsv` (22 rows = F0 5 pairs + F1/F2 3 pairs each, reserved slots emit NO row) + `manifest.rs`, 13/13; collision guard **disable-verified** (removed → only `colliding_entries_are_rejected` fails, BOTH stay-silent twins stay green). Bar B0's end-to-end half (mutating an entry changes written bytes) DEFERRED — the bake does not exist yet | slot purity §2; bar B0 |
| D-WXS-1a | variable census as a committed re-runnable probe (17 surface + 91 upper-air + 14 static = 122 fields; 92,044 six-hourly steps) | 0 | **SHIPPED 2026-08-13** — `era5_variable_census.py` + `.json`; `--selftest` PASS on all 10 constants, orchestrator-rerun independently; guard disable-verified (one constant broken → exit 1, correct message) | ends the chat-only-figure defect for the census |
| D-WXS-2 | key codec `(lat,lon) ↔ NodeGuid` — HEEL 16° tile / HIP within-tile / TWIG dormant; ragged tiles; lon-wrap range-SET | 1 | **SHIPPED 2026-08-13** — `key.rs`, 5/5 green; exhaustive 1,038,240-cell round-trip + collision-free; both bar-B1 halves **disable-verified** by the orchestrator (zeroing the HIP lat byte kills 3 tests incl. collision + ragged; removing the seam split kills the wrap twin while the non-wrap twin stays green) | a 16° box becomes a HEEL-prefix scan; bar B1 |
| D-WXS-2a | **NEW — row-major vs Morton, pre-registered comparison.** The shipped key assigns one WHOLE byte per axis; OGAR's cascade doctrine specifies the axis bytes **nibble-interleaved** (Morton). §1.2 deviated from the canon it cites and did not say so — now recorded as plan §1.3a. The prefix-scan claim holds under both; what differs is neighbour locality (`lat ± 1` is 1440 cells away under row-major) and how many ranges a non-tile-aligned box needs | 1 | **Bar PRE-REGISTERED 2026-08-13** (plan §1.3b, committed before the run). Half A (pure key-space: range count + neighbour locality; arms SHIPPED/MORTON/CONTROL-BAD) is runnable NOW. Half B (the ζ stencil) is gated on D-WXS-9 → D-WXS-0 | gates any downstream assumption of Morton locality — measured against the ζ stencil (D-WXS-9), metric stated before the run |
| D-WXS-2a | **NEW — row-major vs Morton, pre-registered comparison.** The shipped key assigns one WHOLE byte per axis; OGAR's cascade doctrine specifies the axis bytes **nibble-interleaved** (Morton). §1.2 deviated from the canon it cites and did not say so — now recorded as plan §1.3a. The prefix-scan claim holds under both; what differs is neighbour locality (`lat ± 1` is 1440 cells away under row-major) and how many ranges a non-tile-aligned box needs | 1 | **CLOSED by KILL 2026-08-13** — half A run, `examples/layout_probe.rs` + `.json`. **primary FAIL** (MORTON won ONE metric, not both: neighbour locality 16 vs 32 = 2x better, range count 212.50 vs 140.00 = ~1.5x worse). **control PASS** (CONTROL-BAD 3100 ranges / 15862 neighbour — the metrics do measure locality). **stay-silent twin PASS** (tile-aligned = exactly 1 range under BOTH, `[1,1,1,1]`). Pre-registered consequence applied: **no code change, no migration**. §1.3b's word "harmless" corrected in §1.3c — MORTON wins on exactly the metric half B would care about, so the honest reading is *no unambiguous win in key space* | gates any downstream assumption of Morton locality — measured against the ζ stencil (D-WXS-9), metric stated before the run |
| D-WXS-3 | shared canonical floor calibration (global 0.4–99.6 pct, frozen per epoch, stamped in dataset metadata) | 1 | **SHIPPED 2026-08-13** — `floor.rs`; bar B2 **disable-verified** (widening the "narrow" control floor kills only the control, twin stays green); version-stamp mismatch detected, ±½-bucket round-trip asserted. **AMENDED same day** (`E-A-TOTAL-FUNCTION-THAT-CANNOT-REFUSE-IS-A-CORRUPTION-PATH-1`): `saturation_of` folded non-finite input into the metric — `quantize` sends `NaN`/`-inf`→0 and `+inf`→255, all **rim** buckets, so an all-`NaN` population scored **1.0** ("fully saturated") where the truth is "no data at all". Now returns `SaturationScore {fraction, finite, non_finite}` — reported, never folded and never silently dropped. `calibrate` checked **CLEAN** (already filters `is_finite`) | bar B2 — this is the bar's own INSTRUMENT, so the defect would have corrupted a measurement, not a value |
| D-WXS-3b | **NEW — the L4 lane (pack/unpack ONE 16-byte facet).** The plan gave the lane a worker in §6.2 but **no D-id in §4's ladder** — it jumped D-WXS-3 → D-WXS-4. Added here as the pack/unpack half the bake will call | 1 | **SHIPPED 2026-08-13** — `lane.rs`, 33/33 crate-wide; 4 disables verified by the orchestrator (lo/hi swap → the swap test; hard-coded slot → 3 tests incl. manifest-load-bearing; version guard bypassed → the version test; unmapped slots emitting values → the reserved-slot test). The lane names no ERA5 variable in its own source — the caller's closure owns that **AMENDED same day (codex P1, PR #948):** `pack_facet` accepted non-finite readings; `quantize` maps them to valid-looking buckets, so a missing ARCO-ERA5 chunk (all-`NaN` — **valid store semantics**, and five W1 variables 404 at the arc's own fixture timestep) would have been stored as plausible low-bucket measurements. Now `LaneError::NonFiniteValue`, covering `±inf` too since they land on the rim. Disable-verified | precursor to bar B3; §2.6 slot purity as code |
| D-WXS-4 | the bake: one timestep → 1,038,240 NodeRows → ONE Lance version | 1 | Queued — **blocked behind D-WXS-0** (must refuse to write without a minted classid) | bar B3; the missing path |
Expand Down
69 changes: 69 additions & 0 deletions .claude/plans/weather-soa-bake-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,75 @@ whether a difference exists **in key space at all**. If half A kills, half B is
moot; if half A confirms, half B still has to run before any migration. Stated
now so a green half A is not later read as a mandate.

### §1.3c `D-WXS-2a` half A — RUN 2026-08-13. **The KILL fires. Closed, no code change.**

Probe: `crates/weather-poc/examples/layout_probe.rs` (+ `--json` output committed
beside it). Bar: §1.3b, committed **before** this run (`d83b4d3e`). Selftest run
first, green — including the check the brief demanded be *worked out rather than
assumed*: a full longitude row under SHIPPED is **23 ranges** (one per longitude
tile), not 1.

#### Measured

| arm | rc median (non-tile-aligned) | neighbour-locality median |
|---|---|---|
| **SHIPPED** | **140.00** | 32.00 |
| **MORTON** | 212.50 | **16.00** |
| CONTROL-BAD | 3100.00 | 15862.00 |

Box set: 20 boxes — 4 tile-aligned, 8 non-tile-aligned, 4 seam-crossing,
4 pole-adjacent. Neighbour sample: deterministic stride 677, n = 1534.

#### The three verdicts, as pre-registered

- **primary — FAIL.** The bar required MORTON to beat SHIPPED on **both**
metrics. It won one and lost one: **2× better** on neighbour locality
(16 vs 32) and **~1.5× worse** on range count (212.50 vs 140.00).
- **control — PASS.** CONTROL-BAD is far worse on both (3100 ranges;
neighbour distance 15862). The metrics therefore measure locality rather
than nothing — without this the primary result would be unreadable.
- **stay-silent twin — PASS.** Every tile-aligned box is **exactly one range
under both** arms (`[1,1,1,1]` each). §1.2's load-bearing claim — *a 16° box
is a HEEL-prefix scan* — holds, and shows **no difference where the plan
claims none**.

#### Consequence (the pre-registered one, unchanged)

**`D-WXS-2a` closes. No code change. §1.3a's layout stands as shipped.** The
migration is not opened.

#### ⚠ A correction to §1.3b's own wording, not a softening of the verdict

§1.3b phrased the KILL as *"the deviation is **harmless for this workload**"*.
The data shows that phrasing was imprecise, and it is corrected here rather
than left to read as more than was measured:

**MORTON wins, 2×, on exactly the metric half B would care about.** Neighbour
locality is what a ζ stencil spends; range count is what a box scan spends. So
the honest statement is **"no unambiguous win in key space"**, not "harmless".

The verdict is untouched: the bar said *both*, it got *one*, and one is a FAIL.
Re-reading a split as a win is the failure this arc has a rule against
(*"re-normalising a metric until it stops showing the confound is how a
confound becomes a finding"*). What changes is only my own summary sentence,
which claimed more than the measurement.

**The prior this leaves for half B**, if the ζ stencil is ever measured under
both layouts (gated on `D-WXS-9` → `D-WXS-0`): MORTON is expected to win there,
by roughly 2×, because that is what the locality metric already shows. That is
a **prior, not a result** — and a 2× locality win still would not by itself
justify a migration, because the same measurement shows it costs ~1.5× on
range count.

#### Why SHIPPED's numbers look the way they do (arithmetic, checked)

SHIPPED orders `[lat_tile, lon_tile, lat_hip, lon_hip]`, so within one tile the
order is `lat_hip`-major / `lon_hip`-minor. A `lon ± 1` neighbour is therefore
**1** apart and a `lat ± 1` neighbour is **64** apart; the median over the
4-neighbourhood is the median of `[1, 1, 64, 64]` = **32** — exactly the
measured value. The two-level blocked order §1.3a identified (rather than the
flat "row-major" I first wrote) is what keeps its range count competitive.

### §1.4 classid — a mint decision, NOT taken here

`0x0F = Geo` already exists in the OGAR domain table; free domains are `0x03–0x06`
Expand Down
60 changes: 60 additions & 0 deletions crates/weather-poc/examples/layout_probe.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
{
"arms": ["shipped", "morton", "control_bad"],
"sample_stride": 677,
"box_set": [
{"kind": "tile_aligned", "lat_lo": 128, "lat_hi": 192, "lon_lo": 0, "lon_hi": 64},
{"kind": "tile_aligned", "lat_lo": 320, "lat_hi": 384, "lon_lo": 640, "lon_hi": 704},
{"kind": "tile_aligned", "lat_lo": 512, "lat_hi": 576, "lon_lo": 1344, "lon_hi": 1408},
{"kind": "tile_aligned", "lat_lo": 192, "lat_hi": 256, "lon_lo": 960, "lon_hi": 1024},
{"kind": "non_tile_aligned", "lat_lo": 10, "lat_hi": 37, "lon_lo": 5, "lon_hi": 53},
{"kind": "non_tile_aligned", "lat_lo": 100, "lat_hi": 233, "lon_lo": 200, "lon_hi": 410},
{"kind": "non_tile_aligned", "lat_lo": 50, "lat_hi": 521, "lon_lo": 33, "lon_hi": 999},
{"kind": "non_tile_aligned", "lat_lo": 400, "lat_hi": 405, "lon_lo": 700, "lon_hi": 1300},
{"kind": "non_tile_aligned", "lat_lo": 20, "lat_hi": 680, "lon_lo": 900, "lon_hi": 905},
{"kind": "non_tile_aligned", "lat_lo": 65, "lat_hi": 129, "lon_lo": 63, "lon_hi": 129},
{"kind": "non_tile_aligned", "lat_lo": 200, "lat_hi": 205, "lon_lo": 1, "lon_hi": 2},
{"kind": "non_tile_aligned", "lat_lo": 300, "lat_hi": 450, "lon_lo": 1100, "lon_hi": 1150},
{"kind": "seam_crossing", "lat_lo": 100, "lat_hi": 200, "lon_lo": 1400, "lon_hi": 40},
{"kind": "seam_crossing", "lat_lo": 300, "lat_hi": 350, "lon_lo": 1430, "lon_hi": 10},
{"kind": "seam_crossing", "lat_lo": 500, "lat_hi": 600, "lon_lo": 1439, "lon_hi": 1},
{"kind": "seam_crossing", "lat_lo": 50, "lat_hi": 70, "lon_lo": 1300, "lon_hi": 100},
{"kind": "pole_adjacent", "lat_lo": 0, "lat_hi": 30, "lon_lo": 100, "lon_hi": 200},
{"kind": "pole_adjacent", "lat_lo": 700, "lat_hi": 721, "lon_lo": 500, "lon_hi": 600},
{"kind": "pole_adjacent", "lat_lo": 0, "lat_hi": 1, "lon_lo": 0, "lon_hi": 1440},
{"kind": "pole_adjacent", "lat_lo": 715, "lat_hi": 721, "lon_lo": 900, "lon_hi": 950}
],
"results": {
"shipped": {
"by_kind": {
"tile_aligned": {"range_counts": [1, 1, 1, 1], "median": 1.0000, "max": 1},
"non_tile_aligned": {"range_counts": [27, 269, 963, 19, 660, 130, 5, 150], "median": 140.0000, "max": 963},
"seam_crossing": {"range_counts": [200, 99, 198, 45], "median": 148.5000, "max": 200},
"pole_adjacent": {"range_counts": [61, 42, 23, 6], "median": 32.5000, "max": 61}
},
"neighbour_locality_median": 32.0000
},
"morton": {
"by_kind": {
"tile_aligned": {"range_counts": [1, 1, 1, 1], "median": 1.0000, "max": 1},
"non_tile_aligned": {"range_counts": [57, 289, 1053, 47, 660, 136, 5, 295], "median": 212.5000, "max": 1053},
"seam_crossing": {"range_counts": [208, 100, 200, 68], "median": 150.0000, "max": 208},
"pole_adjacent": {"range_counts": [65, 46, 90, 13], "median": 55.5000, "max": 90}
},
"neighbour_locality_median": 16.0000
},
"control_bad": {
"by_kind": {
"tile_aligned": {"range_counts": [4096, 4096, 4096, 4096], "median": 4096.0000, "max": 4096},
"non_tile_aligned": {"range_counts": [1296, 13440, 61824, 3000, 320, 4224, 5, 3200], "median": 3100.0000, "max": 61824},
"seam_crossing": {"range_counts": [5120, 1000, 128, 4800], "median": 2900.0000, "max": 5120},
"pole_adjacent": {"range_counts": [3000, 2100, 1440, 300], "median": 1770.0000, "max": 3000}
},
"neighbour_locality_median": 15862.0000
}
},
"verdicts": [
{"name": "primary", "pass": false, "detail": "morton_rc_median=212.50 vs shipped_rc_median=140.00 (non-tile-aligned); morton_nb_median=16.00 vs shipped_nb_median=32.00"},
{"name": "control", "pass": true, "detail": "control_rc_median=3100.00 vs shipped=140.00/morton=212.50 (non-tile-aligned); control_nb_median=15862.00 vs shipped=32.00/morton=16.00"},
{"name": "stay-silent twin", "pass": true, "detail": "shipped_tile_aligned_counts=[1, 1, 1, 1]; morton_tile_aligned_counts=[1, 1, 1, 1] (every entry must be 1)"}
]
}
Loading
Loading