diff --git a/.claude/board/EPIPHANIES.md b/.claude/board/EPIPHANIES.md index 5b4fd209..6941004d 100644 --- a/.claude/board/EPIPHANIES.md +++ b/.claude/board/EPIPHANIES.md @@ -1,3 +1,472 @@ +## 2026-08-11 — E-THE-BYTE-WAS-ONLY-THE-SELECTOR-THE-PAIR-IS-THE-CARRIER-1 + +**Status:** FINDING `[H]` — operator correction + `l4_rail_probe.py` (commit +96e86b90); report §6.1. EXPLORATORY, not an EV. + +**Operator:** *"was ist mit 6× Palette256:Palette256 centroid, was ja die +Verteilung anzeigen soll — palette256 alleine ist ja nur 'attention header'"*. + +**Every encoding in this arc treated "one scalar → one byte" as the unit. The +shipped carrier is a PAIR.** le-contract §3 row L4 is `6 × (8:8)`, +`palette256²` — "each byte pair indexes the 256×256 palette distance/compose +tables; similarity = ONE table read". The single byte is the **selector**; the +**pair** is a cell in the centroid tile, and the tile is where the +distribution lives. I had built one rail and called it the carrier — the +rolling-floor cascade is the sanctioned §3 "area : location in stacked +exactness" reading, but of ONE rail out of six. + +**Measured: the 12-byte facet is LOSSLESS against the f64 spine.** Carve D +(dipole rail + 10 ring bytes *spread over the full radius*, missing rings +interpolated) reproduces the f64 constrained spine to four decimals on both +storms — R² 0.9434 / 0.9090, |D − f64| = 0.0000. R² is demonstrably sensitive +(carve B, the same budget spent on 12 rings with no dipole rail, collapses to +0.635 / 0.294), so this is recovery, not insensitivity. + +**Two of four pre-registered bars FAILED as written, and both failures taught +more than the pass.** + +1. **L1 failed** (0.0222 vs a 0.02 bar) and the decomposition names the cause + exactly: **quantization +0.0000, dropped rings +0.0222**. *The carrier's + PRECISION is free; its CAPACITY was the entire miss.* Spending the same 12 + bytes across the full radius erases it. Generalizable: when a byte-budget + fit misses, decompose before widening — the two costs point at opposite + fixes (a bigger codebook vs a better carve), and here the codebook was + already perfect. + +2. **L3 failed, and so did my proposed rescue** — the more useful half. + Fisher-z centroid axes are **5× WORSE** than uniform on the ring means + (18.07 vs 3.84 Pa). I hypothesised the population was wrong (ranks against + the 24 encoded values rather than the field) and measured that too: **19.00 + Pa, no rescue.** So the mechanism is not population size but *what the read + is for*: ring means are a smooth NARROW-BAND quantity sitting + mid-distribution, and a rim-stretch spends levels in tails where no ring + mean lives. + +**The demarcation this forces, which is the entry's real content:** it does +NOT contradict `three_register_probe`'s R4, where Fisher-z is **8.3× TIGHTER** +than plain rank in the storm tail (24.74 vs 204.54 Pa) on the raw field. Same +substrate, opposite verdicts. **Fisher-z wins a RANK/TAIL read and loses an +INTERPOLATE/LEVEL read.** Which is exactly why le-contract says a ClassView +**MAY** declare an analytic codebook — per class, by measurement, not as a +default. This corrects my own over-generalization, made earlier the same +session, that Fisher-z is *the* L4 codebook axis. + +**A ninth vacuous falsifier, and the mechanism is worth naming.** L4x +(shared-vs-per-storm codebook) passed on its first run **comparing an array +against itself**: a uniform codebook is fixed by its population's min/max +alone, so because storm 1's profile range strictly CONTAINS storm 2's, storm +1's "own" codebook IS the pooled codebook. It looked like a real comparison +only because an earlier variant (fisherz, which depends on the whole rank +distribution rather than the endpoints) had produced *differing* numbers — +**switching to the codebook the previous bar had just NAMED AS BEST is what +made the test vacuous.** Degeneracy flags now report both directions; the +informative one gives **620.79 Pa vs 4.48 Pa, a 139× penalty**, strong +evidence the codebook must be global — the "one table read" property the +carrier exists for. + +**Rule:** *when a bar's inputs are derived from the same population, check +they are not the same OBJECT before reading its verdict.* Equal numbers on +both sides of a comparison are the signature, and "it passed" is the least +informative way to find out. + +## 2026-08-11 — E-THE-HEADLINE-NUMBER-MEASURED-A-MODEL-NOBODY-CLAIMED-1 + +**Status:** FINDING `[G]` — external review of PR #926 (14 CodeRabbit + 2 Codex +findings); re-measured and corrected in `comet_tail_probe.py` and +`COMET_TAIL_REPORT.md` §1 / §8b (commit 5302828e). + +**The arc's most-repeated number was measured on a model the arc never +claimed.** The compression headline — "center + ~12 ring means + ONE dipole +(2 values) = 93–97 % of in-disk variance" — came from a `decompose()` that +fits `a1[b]`, `b1[b]` **per ring**: 12 rings × 2 = **24** free dipole +parameters, so the published 0.972/0.926 belongs to a **36-parameter** model, +not the **14-value** one the storage claim describes. Measured properly, the +constrained 2-parameter dipole (one amplitude slope + one bearing — the +linear-background form the report's own §2 derives) gives **0.943 / 0.909**. +Corrected headline: **90.9–94.3 %, not 93–97 %.** + +The finding SURVIVES — 14 values still lift a storm from 29–63 % to 91–94 % — +but claim and measurement had drifted apart by ~2.5× in parameter count across +six probes and several report rewrites, and nobody in-session noticed. **The +tell was available the whole time:** the report described the representation +in one place ("2 values") and the code produced another ("per-ring"), and no +test tied the two together. + +**Two review findings IMPROVED results rather than damaging them**, which is +the argument for external review as more than ceremony: (a) sunflower E2 was +handing the grid arm up to 25 % more samples than the spiral (every in-disk +lattice point instead of exactly n) — with equal budgets enforced the **spiral +now wins at every N**, where the arc had recorded "parity"; the original result +was PESSIMISTIC. (b) The voxel-chess palette arm compared palette-derived +geostrophic winds against RAW observations, because `geo_corr` closed over +module-level `u`/`v` — a hybrid, not the pre-registered palette result. + +**A fourth vacuous assertion, in this arc's own documented house style:** E6's +`rises_then_decays` required only an interior maximum plus a lower final +value, so it accepted a profile that DECREASED before rising to the peak — and +the committed run did exactly that (12.190 → 12.163 m/s before the 525 km +peak) while reporting `true`. Found by a reviewer, not by the author, which is +the same asymmetry `E-ZERO-FOR-ELEVEN-...` already recorded. + +**Rule:** *a number that appears in a headline must be produced by code whose +parameter count matches the headline's own description of the object.* Where +prose says "N values", the probe should EMIT N and the report should print it +— which `comet_tail_probe.py` now does +(`n_params_profile_wn1_constrained`), so the two cannot silently drift again. + +## 2026-08-11 — E-SPINE-FOUND-MODERATORS-MISSING-1 + +**Status:** OPERATOR RULING (framing) + FINDING `[H]` — report §9, commit +paired with the PR opening. Reframes the whole comet-tail chain's verdict. + +**Operator:** *"Wir haben ein Spine gefunden — die Stellschrauben müssen noch +mit den Variablen der bekannten Modelle moduliert werden. Uns fehlen die +Moderatoren; aber wir haben bereits das Gerüst, um das Zentrum und die +Dynamik zu modellieren. Außerdem haben wir Feuchtigkeit und Abregnen im +Aufwind an der Kollision zwischen den Gebieten nicht modelliert — das ist +eine Art Entropie bei Verdunstung und Abregnen."* + +**Why this reading is defensible — stated at the strength the evidence +actually supports:** a 0.68–0.73 directional main effect whose residual were +random would be a dying claim — but this chain's residual runs MONOTONICALLY +with a measured variable (the 92–102° height ladder, 3–5× apparatus noise). +*Main effect + structured residual + identified covariate* is **consistent +with a missing moderator and requires independent validation**. It does NOT by +itself exclude model misspecification, centre/label error, selection effects, +or chance. + +> **Correction (CodeRabbit on PR #926, 2026-08-11).** This paragraph +> originally read "is the signature of a missing moderator, NOT of a null. A +> null does not produce a ladder." That overstated what a monotonic residual +> can establish — it supports the hypothesis, it does not discriminate it from +> the alternatives above. Corrected in place per the append-only rule's +> allowance for regrading; the directional predictor stays SUGGESTIVE and +> unpromoted either way, which is what the PR objective already said. + +**The three-part decomposition now on record (report §9):** +1. **Spine `[G]`** — center + ~12 ring means + 1 wn-1 dipole = **90.9–94.3 %** + of in-disk variance, unshaken across 3 independent samples / 41+ storms / + 1980–2021. **~14 logical model values** (12 ring means + a 2-value + dipole) plus a center address. *(Corrected 2026-08-12, CodeRabbit PR #926: + this read "~14 bytes + an address", conflating the MODEL size with a + CARRIER budget. The measured encoding is a 12-byte `6×(8:8)` L4 facet — + see report §6.1; the byte budget belongs in the encoding section, not + in the spine statement.)* *(Regraded in place per the same + allowance used at line 67 above: this line printed 93–97 %, which + `E-THE-HEADLINE-NUMBER-MEASURED-A-MODEL-NOBODY-CLAIMED-1` — the entry + directly above — corrects to the 14-value model's real figure. Flagged by + CodeRabbit on PR #926 as an internal inconsistency with that entry, and it + was one.)* +2. **Dry moderators `[H]`** — measured in this chain, not yet wired: + steering level (THE ladder; CT-F16 = score the dipole against + steering-level motion instead of 6h surface displacement), displacement/ + label noise, friction/surface type, latitude/regime. +3. **Moist sector `[S]`** — not modeled at all, and "entropy" is technically + the right word: rain-out in the collision-zone updraft is irreversible + moist entropy production (θe the state variable, precipitation the sink — + Emanuel/Pauluis frame). Tractable NOW: the WB2 store carries + specific_humidity/temperature (θe), TCWV, total_precipitation_6hr, + vertical_velocity — and θe/TCWV are scalar fields, so the SAME ring/wn-1 + decomposition applies verbatim. CT-M1..M3 named as falsifiers; the July + failures (wn1_frac 0.19–0.36) are plausibly the diabatically-dominated + storms, making diabatic dominance itself a computable intake gate. + +**The brutal step (operator-directed, `[S]`):** learn the moderator matrix on +the substrate's own proven machinery — the spine as board state, moderators +as `W` in domino.rs' symbiont `C = A·W` tile-GEMM (stencil-as-GEMM already +byte-proven on real WB2 in ndarray `geostrophic_stencil.rs`), recurrence over +6h spine states via the workspace's byte-parity int8 LSTM (E-OCR-LSTM-1). +Explicit physics as spine, learned weights as moderators — the NeuralGCM- +shaped hybrid at 512 B/storm, gated by disjoint-decade train/test + the +plan-§8 audit. + +## 2026-08-11 — E-MY-OWN-PRE-REGISTRATION-HAD-A-GAP-AND-I-NAMED-IT-1 + +**Status:** FINDING `[G]` — CT-F14, `comet_tail_f14.py` / `.json`, report +§5.11. Direct follow-up to E-THE-RESCUE-THAT-WEAKENED-ITSELF... below; this +entry is about a defect in *my own pre-registration design*, caught and +corrected by the same discipline it should have applied from the start. + +**CT-F14 was pre-registered and committed to git BEFORE it ran** (`4f1a1b4f`) +— fixed dates, fixed bar (n≥20, ≥0.70), and a fixed interpretation table for +a "combined 3-sample" figure, all decided before any output existed. Run: 19 +of 85 mechanically-generated candidates qualified (one short of the n=20 +floor) → correctly **NO-VERDICT by the pre-registered rule.** The pooled +3-sample figure (n=26, 19/26=0.731) crossed the pre-committed p<0.05 +"established" threshold (p=0.0145). + +**By the letter of my own pre-registration, this should have been reported +as "established, ready for the audit-gate queue."** It was not, because a +sensitivity check — run precisely because the last entry demands applying +scrutiny to results that help as readily as to ones that hurt — found +something the pre-registration never anticipated: **CT-F14's own qualifying +subset, taken alone, sits at 0.684 (13/19), BELOW the 0.70 bar, p=0.0835 — +not significant.** The single test this whole probe existed to produce does +not independently support the claim it was built to test. The "established" +pooled figure is being carried by two small prior fragments (n=4 at 0.75, +n=3 at 1.00) blended with a properly-powered new sample that came in lower. +Dropping just the smallest fragment (n=3, fully saturated) still barely +clears p<0.05 (0.0466) — so the pooled crossing is not purely an artifact of +one tiny subsample, but the component that mattered most (the large, careful +new test) disagrees with the pooled verdict on its own terms. + +**The gap, named plainly: my pre-registration specified thresholds for a +pooled figure without specifying what to do if the new, properly-powered +sample and the pooled figure disagreed.** I did not write a rule for this +exact configuration because I did not anticipate it — I expected CT-F14 to +either clearly pass or clearly fail on its own, not to fall one storm short +of its power floor while *also* landing under the bar. Finding that gap +after the fact and exploiting it silently (reporting only the pooled +"established" number, which the letter of my pre-registration technically +licensed) would have been exactly the failure mode this arc's discipline +exists to prevent — just moved one level up, from cherry-picking a result to +cherry-picking which of two valid readings of a pre-committed rule to report. +**Named instead: graded the verdict down to "still suggestive," and +recorded the pre-registration gap itself as the finding**, alongside a +second, smaller walk-back (§5.10's striking monsoon-band exclusion pattern, +4/5 in the small sample, thinned to 2/21 at 4× the exclusion count in this +larger one — the small-n-looked-like-a-pattern theme recurring one level +down from the main directional claim). + +**The reusable lesson, sharper than the previous entry's:** *pre-registration +protects against post-hoc rationalization of the DATA. It does not +automatically protect against post-hoc selection among several VALID readings +of the rule itself, when the rule turns out to admit more than one — that +requires the same discipline applied one level up, at read time, not just at +design time.* After four probes, three independent samples, and 41 total +storms, the honest position is: structural claim solid throughout, directional +claim genuinely undetermined — not because no test was run, but because the +one test built to settle it came back below its own bar. + +## 2026-08-11 — E-THE-RESCUE-THAT-WEAKENED-ITSELF-UNDER-SCRUTINY-1 + +**Status:** FINDING `[G]` — CT-F10/F11/F13, `comet_tail_f10_f11.py` / `.json`, +report §5.10. Direct follow-up to E-N-EQUALS-TWO-... below; the discipline +this entry demonstrates is the point as much as the numbers. + +**A second, fully independent blind sample REVERSED the previous sample's +verdict on its own bar — 8/10 = 0.80 (vs the first sample's 6/10 = 0.60) on +15 mechanically-generated candidate dates (fixed start + fixed 411-day +stride, chosen before any code ran, landing entirely in 1980–1995, zero +overlap with the 2015–2021 sample).** This is exactly the moment a session is +tempted to declare victory. Instead: proper statistics first. Neither sample +clears a conventional two-sided 0.05 alone (sample 1 p=0.754; sample 2 +p=0.109). **Pooled across both fully independent samples: 14/20 = 0.70, +one-sided p≈0.058 — landing almost exactly on the pre-set 0.70 bar by +coincidence, and still short of significance.** The reversal is real, but it +does not resolve anything on its own; it converts a FAIL into a +STILL-BORDERLINE with a doubled sample size. + +**Then the two competing explanations from the previous entry were BOTH +checked against the new evidence — and the answer inverted which one looked +better supported.** §5.9 (previous) favored regime-contamination (a monsoon +low corrupting one sample) as the likely driver of the sign-consistency gap. +Checked directly: retroactively applying the wn1_frac≥0.40 regime filter to +**sample 1's own data** removes its two lowest-structure storms +(2020-07-20, 2019-10-25) — and **both were negative-signed, agreeing with +the prediction.** Removing two hits, not two misses, drops sample 1 from +6/10 (0.60) to **4/8 = exactly 0.500.** The regime explanation does not +survive contact with its own predicted mechanism — checked, not assumed, and +reported as a weakening even though it was MY preferred explanation from the +prior entry. + +**Meanwhile the apparatus explanation (motion-bearing noise on slow storms) +strengthened under the same treatment.** Pooling the displacement≥250km/6h +subset from BOTH independent samples (n=4+3=7, spanning two decades) gives +**6/7 = 0.857, one-sided p≈0.0625** — the single most consistent number in +the entire chain, though n=7 keeps it suggestive rather than decisive, and +this pooling was not itself pre-registered before either sample ran (stated +plainly). An unplanned bonus: 4 of 5 storms excluded from the second sample +by the pre-existing trackability gate cluster at 26–33°N/67–134°E in +June–September — the exact monsoon-season geography flagged as +contamination-prone twice before (§5.7's blind land storm, §5.9's worst +storm) — showing the trackability gate is already doing real filtering work +for free, on a completely different axis than the wn1_frac regime filter. + +**The reusable lesson, and why this entry exists separately from the last +one:** *when a follow-up result HELPS your preferred story, run the same +scrutiny you'd run if it hurt it — apply the retroactive check, do the +statistics, report the weakening if the mechanism doesn't survive contact +with its own prediction.* This session's regime-contamination hypothesis was +mine, felt right after §5.9, and did not survive being checked against the +data that was supposed to support it. The number that DID hold up (apparatus, +6/7) is not the flashier "we found the real mechanism" story — it is the +more mundane "the label itself is noisy when storms move slowly" story, and +it earned its position by surviving a check the flashier story failed. +CT-F14 (a single properly-powered displacement-filtered sample, n≥25–30) is +now the correctly-scoped next step — not a third exploratory rerun of a +sample that already ran twice. + +## 2026-08-11 — E-N-EQUALS-TWO-LOOKED-LIKE-PHYSICS-AND-WAS-HALF-COIN-FLIP-1 + +**Status:** FINDING `[G]` — CT-F5 + CT-N, `comet_tail_f5_n10.py` / `.json`, +report §5.8–5.9. Closes the arc's own standing item ("n ≥ 10 storms before +any constant"); supersedes nothing, differentiates everything. + +**CT-F5 (small, closes a named defect):** the walking-center fix (search near +the *previous* level, not always the surface, 250 km/step) corrects storm 2's +CT-F1 saturation defect. Its winning level (500 hPa, −4.5°) is reached by a +**0.0 km step**, so the pass is untouched by the saturation that appears one +step later, outside the tested band. Storm 2 goes from NO-VERDICT to a clean +pass; storm 1 reproduces bit-identically (verified post-hoc — the F5c field +was pre-registered but never coded, a gap admitted rather than smoothed over). + +**CT-N (the headline): a blind 10-storm sample split the comet-tail claim into +two claims with DIFFERENT truth values, and only one of them survives.** +wn1-dominance and explanatory power (E1/E4) replicate cleanly at scale +(median wn1_frac 0.72, median R² 0.90, both clear their bars) — **the signed +left-of-motion prediction that made storms 1–2 exciting does not** (6/10 = +0.60 same-sign, against a 0.70 bar; a naive p=0.5 null already gives +P(≥6/10)≈0.38, i.e. statistically unremarkable). What looked like 2/2 at +p=0.0625 was, on independent storms, close to a coin flip. CT-N5 (does a +wind-based center generalize the way it did for storm 2 alone?) also fails +at 6/10; CT-F9 (does land-fraction asymmetry correlate with the unexplained +residual, testing candidate 2's real mechanism) returns a clean non-support +(corr = −0.30). + +**Two post-hoc leads, explicitly NOT used to rescue the FAIL:** restricting to +the 4 storms with displacement ≥ 250 km/6h (closer to storms 1–2's own +regime) lifts sign-consistency to 3/4 = 0.75; dropping only the single most +extreme low-displacement/near-polar outlier lifts it to 6/9 = 0.667. Both +point at plausible, testable confounds — motion-bearing noise at small +displacement, and storm-type contamination in a purely blind sample (one +selected storm at 32°N in July is very plausibly a monsoon low, not a +baroclinic system, and has the sample's worst wn1/R²). Filed as CT-F10 +(pre-registered displacement filter) and CT-F11 (pre-registered regime +filter) — a properly designed rerun, not a re-scoring of this one. + +**The reusable lesson, stated once so it doesn't need re-deriving:** a probe +result at n=2 can be right about STRUCTURE (a real, dominant, well-explained +mode) while being wrong about GENERALITY (whether its sign/direction is a +rule or a coincidence of which two examples were picked) — and only a genuine +n≥10 blind sample separates the two. Every earlier gate in this chain (F3's +apparatus check, F4's independent-definition check, F7's blind land-storm +selection) was local to storms 1–2; CT-N is the first gate that tests whether +storms 1–2 were representative at all, and the honest answer is "partially." + +## 2026-08-11 — E-A-JITTER-AMPLITUDE-YOU-CHOSE-IS-NOT-AN-UNCERTAINTY-YOU-MEASURED-1 + +**Status:** FINDING `[G]` — CT-F4 + CT-F7, `comet_tail_f4_f7.py` / `.json`, +report §5.6–5.7. Amends E-THE-OFFSET-WAS-THE-APPARATUS-... below, which stands +as written (it was correct for the amplitude it tested). + +**My apparatus test condemned a number using a sensitivity amplitude I picked +out of the air.** CT-F3 jittered the storm center by ±100 km and found the +alignment error moved 29.4° — so the −40° offset was declared unmeasurable. +But ±100 km was never *measured*; it was a plausible-sounding round number. +The non-circular question is **how far apart independent center definitions +actually land**, and that is the uncertainty. Four definitions across three +physical fields (sub-grid MSLP min; ∇²p centroid; **10m vorticity** centroid; +sub-grid z850 min) agree to **20 km** / **73 km**, and the answers they give +span **2.3°** / **6.5°** — so the real apparatus noise is ≈ ±3–7° and the +offset **is** measurable. F3 was not wrong; its amplitude was unjustified. + +**Two structural lessons, both reusable:** +1. *Prefer a measured disagreement to a chosen perturbation.* Independent + method variants are a free, non-arbitrary uncertainty estimate — and where + both exist, the isotropic jitter was a 2× **over**estimate, because real + variants cluster along a preferred axis rather than scattering evenly. +2. *An anti-vacuity guard must be allowed to refuse a PASS you want.* Storm 1 + scored a 2.3° spread — but its four centers agreed to 20 km, below the + 31.9 km grid diagonal, so the pre-registered CT-F4c guard returned + NO-VERDICT rather than banking a free pass. Storm 2, whose definitions + genuinely disagreed (73 km = 2.4× the diagonal), is the one that carries + the result. The guard cost me the tidier of the two numbers, which is + what tells me it was real. + +**Also, the friction candidate was partly MIS-SPECIFIED and the data said so +before I did.** CT-F7 measured cross-isobar inflow over land (blind storm +selection): **+34.2° land vs +20.5° ocean inside the same disk** — a paired +contrast that controls for depth/latitude/curvature, textbook magnitudes, both +bars passed. But F2/F7 bound the rotation of the **wind** relative to the +isobars, whereas the CT-E3 offset is a rotation of the **pressure dipole**; +friction does not rotate the pressure field except at second order. And an +unplanned pairing already in the F2 output points the wrong way for it: storm 1 +is 1 % land (+14.7° inflow), storm 2 is 46 % land (+22.0°), yet their offsets +are −42.0° and −40.2° — **the more frictional storm has the smaller offset.** +`[S]`, n=2, undesigned, recorded as suggestive. + +Net: candidate 1 (baroclinic tilt) leading and near-unopposed; candidate 3 +(center bias) bounded at ≈±5°; candidate 2 re-scoped, with CT-F9 written to +test the mechanism it *should* have been about (Ekman pumping vs land-fraction +**asymmetry** across the disk, not mean land fraction). Binding constraint is +now n ≥ 10 storms, not the apparatus. + +## 2026-08-11 — E-THE-OFFSET-WAS-THE-APPARATUS-THE-LADDER-WAS-THE-PHYSICS-1 + +**Status:** FINDING `[G]` — CT-F1/F2/F3 run, `comet_tail_followup.py` / `.json`, +report §5. Follow-up to E-CYCLONE-ASYMMETRY-IS-ONE-DIPOLE-1 below. + +**I ran my own apparatus gate first and it FAILED, which killed the number I +was about to explain.** The −42°/−40° common offset moves by up to **29.4°** +under a ±100 km center jitter — comparable to the offset itself. So the offset +magnitude is inside the apparatus noise and no constant may be fitted from it; +CT-E3 is re-graded in place (the ±45° left-of-motion HIT stands, the magnitude +does not). + +**But the mechanism test found a much bigger signal than the thing it was sent +to explain.** Sweeping geopotential over all 13 levels (one chunk, so the +yes/no became a ladder): the alignment error climbs **monotonically** from +≈ −40° at 1000 hPa through zero in the mid-troposphere (≈600–650 hPa storm 1, +≈400–500 hPa storm 2), spread 101.8°/91.6° — **3–5× the apparatus noise**, and +exactly the baroclinic-tilt/steering-level prediction. Friction was +independently bounded: measured 10m cross-isobar inflow +14.7°/+13.0° over +ocean (right sign, textbook magnitude), so it owns ≈⅓ of the offset at most. + +**Three transferable rules, each earned here:** +1. *Apparatus before mechanism, and mean it.* Had F3 run second, both F1 and F2 + would have been written up as explanations of a number that isn't there. +2. *A failed gate generates the next probe, not an exemption.* The 3–5× size + comparison is recorded as a post-hoc observation and routes to CT-F4 + (sub-grid center fit) — NOT used to override the failed bar. That override + is the "indictment fired → post-hoc rescue" anti-pattern already on this + arc's open-P1 list. +3. *Add diagnostics, never bars, after a run.* A diagnostic added post-run-1 + caught F1's own defect — its center finder **saturating at the 600 km search + radius** on storm 2's upper levels (it locked onto a different system), which + had produced a physically absurd "best level = 100 hPa". Verdict corrected to + NO-VERDICT on that path rather than reported as a refutation. + +The headline claim is untouched: wn-1 dominance, the R² lift, and +left-of-motion never depended on the offset. What died is the constant I would +otherwise have shipped. + +## 2026-08-11 — E-CYCLONE-ASYMMETRY-IS-ONE-DIPOLE-1 + +**Status:** FINDING `[G]` for the measurements (n=2, pre-registered, committed +unmodified, `db57aac0`); `[H]` for generalization; the −40° offset mechanism +and the encoding consequence are `[S]`. Full documentation: +`probes/weather-p1/COMET_TAIL_REPORT.md`. + +**The non-axisymmetric residual of a translating extratropical low is almost +entirely wavenumber-1, and the dipole's orientation is predicted by the +storm's own motion vector.** Measured on two real storms (WB2 ERA5 MSLP, +2021-06-15 12Z→18Z): wn-1 carries 0.924 / 0.895 of the azimuthal residual; +ring-profile + one dipole lifts in-disk R² from 0.635/0.294 to +**0.972/0.926**; low pole lands left-of-motion (NH geostrophic prediction) +within ±45° on 2/2 (null 0.0625), with a *common* −42°/−40° offset — +steering-level/baroclinic-tilt rotation is the prime `[S]` candidate, Ekman +friction second, center-finder bias to be excluded first (CT-F1..F3 +pre-registered in the report, not run). + +**Why it matters here:** (a) it RESOLVES the sunflower probe's axisymmetry +FAIL — the missing third of the storm WAS the tail, one mode, not many; (b) +the compact form `center + ~12 ring means + ONE dipole ≈ 93–97 %` fits the +highheelbgz (start, stride, length) address shape, and the dipole encodes the +motion — a single-frame motion predictor is the named next falsifier, gated +behind the plan-§8 adversarial audit per E-ZERO-FOR-ELEVEN below. + +> **Regraded 2026-08-11 (CodeRabbit, PR #926).** Two claims in this entry are +> superseded by later entries and are corrected here rather than left to +> mislead: (a) **`≈ 93–97 %` → `90.9–94.3 %`** — that figure was a +> 36-parameter per-ring fit, not the 14-value form this sentence describes +> (`E-THE-HEADLINE-NUMBER-MEASURED-A-MODEL-NOBODY-CLAIMED-1`); (b) **"the +> dipole encodes the motion" is NOT established** — the falsifier named here +> was subsequently run four times over three independent samples (n=41), and +> the largest and best-powered of them (CT-F14, n=19) gives 0.684 at p=0.0835. +> The directional claim stays SUGGESTIVE. The structural half of this entry +> (wn-1 dominance, the axisymmetry resolution) is untouched. + ## 2026-08-11 — E-ZERO-FOR-ELEVEN-THE-AUTHOR-CANNOT-AUDIT-HIS-OWN-FALSIFIERS-1 **Status:** FINDING `[G]` — measured by the 13-agent verify/attack pass on diff --git a/probes/weather-p1/COMET_TAIL_REPORT.md b/probes/weather-p1/COMET_TAIL_REPORT.md new file mode 100644 index 00000000..d9feff01 --- /dev/null +++ b/probes/weather-p1/COMET_TAIL_REPORT.md @@ -0,0 +1,1213 @@ +# Comet-Tail Report — Wavenumber-1 Asymmetry of Translating Cyclones + +> **Audience:** meteorology-literate product lead (geostrophy, Ekman layer, +> steering flow, azimuthal wavenumber decomposition assumed known). +> **Status:** EXPLORATORY probe result, all pre-registered bars met (2/2 +> storms). NOT a promoted EV — the audit gate (plan §8) has not run on it. +> **Provenance:** `comet_tail_probe.py` / `.json`, commit `db57aac0`. +> Data: WeatherBench2 ERA5 6h 0.25° (`1959-2022-6h-1440x721.zarr`), +> `mean_sea_level_pressure`, t=91246→91247 (2021-06-15 12Z→18Z). +> **Grades:** `[G]` measured/textbook · `[H]` bounded inference · `[S]` +> speculation, labelled as such. + +--- + +## 1. Executive summary + +An axisymmetric (ring-profile) model of an extratropical low leaves ~30–70 % +of the in-disk MSLP variance unexplained. This probe shows that on two real +storms, **89–92 % of that azimuthal residual is a single wavenumber-1 dipole** +— the "comet tail" — whose orientation is **predicted by the storm's own +motion vector** (low pole left of motion, NH), and whose amplitude grows +~linearly with radius, exactly the signature of a linear background pressure +gradient advecting the vortex. `[G]` for the measurements; `[H]` for the +generalization beyond n=2. + +> **⚠ UPDATE 2026-08-11 (§5.9, CT-N).** A blind 10-storm sample was run +> specifically to test whether the n=2 signed prediction generalizes. **The +> magnitude and explanatory power (§5.9's N2/N3/N4) hold up well at scale — +> the sign consistency of §4's CT-E3 does not** (6/10 = 60 % vs a 70 % bar, +> essentially a coin flip once you leave the two original storms). Read the +> rest of this report — especially the "left-of-motion, signed" framing in §2 +> and §4 — as demonstrated **on storms 1–2 specifically**, not yet established +> as a general rule. §5.9 has the honest breakdown, including a post-hoc lead +> (motion-bearing noise at low displacement) that may explain part of the gap +> and is *not* used to override the failed bar. +> +> **⚠⚠ UPDATE 2026-08-11 (§5.10, a SECOND independent sample).** A fresh, +> mechanically-generated 1980–1995 sample scores 8/10 = 0.80 unfiltered — +> reversing §5.9's verdict on its own bar. Proper statistics keep this +> **borderline, not resolved**: neither sample alone clears a two-sided 0.05, +> and the combined pooled figure (14/20) lands right at the noise floor +> (one-sided p ≈ 0.058). The one number that *does* strengthen — pooling both +> samples' large-displacement storms, 6/7 = 0.857 — supports an **apparatus** +> explanation (motion-bearing noise on slow-moving storms) over the +> **regime-contamination** explanation §5.9 initially favored: checked +> directly, the regime filter does NOT rescue sample 1 (drops it to exactly +> chance, 0.500). Read §4/§5.9's "not established" verdict as still current, +> now with a specific, falsifiable next step (CT-F14) rather than a closed +> question. +> +> **⚠⚠⚠⚠ CORRECTION 2026-08-11 (external review of PR #926) — THE +> COMPRESSION NUMBERS IN THIS REPORT WERE FROM THE WRONG MODEL.** Codex +> flagged, and re-measurement confirms, that `decompose()` fits `a1[b]`, +> `b1[b]` **per ring** — 12 rings × 2 = **24** free dipole parameters, not the +> **2** ("amplitude slope + bearing") the storage claim describes. So the +> published R² 0.972 / 0.926 belongs to a **36-parameter** model, while the +> claimed 14-value representation is a different, more constrained one. Both +> are now measured (`comet_tail_probe.py`, `R2_profile_wn1_constrained_2param`): +> +> | model | params | storm 1 | storm 2 | +> |---|---:|---:|---:| +> | ring profile only | 12 | 0.635 | 0.294 | +> | + per-ring dipole (what was published) | 36 | 0.972 | 0.926 | +> | **+ constrained 2-param dipole (what was CLAIMED)** | **14** | **0.943** | **0.909** | +> +> **The corrected headline is 90.9–94.3 %, not 93–97 %.** The structural +> finding survives — 14 values still lift a storm from 29–63 % to 91–94 % — +> but every "93–97 %" in this document is an overstatement of ~2.5× in +> parameter count, and is superseded by the table above. The constrained model +> is the physically motivated one (a linear background gives exactly one +> amplitude slope and one bearing, §2), so this is a correction of the +> MEASUREMENT to the claim, not a retreat from the claim. +> +> **⚠⚠⚠ UPDATE 2026-08-11 (§5.11, CT-F14, the properly-powered test).** The +> pre-registered decision rule technically fires "established" on a pooled +> 3-sample figure (p=0.0145) — but the single test this whole exercise was +> designed to produce, CT-F14 alone (n=19, the largest and most carefully +> powered sample in the chain), does **NOT** independently clear +> significance (p=0.0835) and its raw rate (0.684) sits below the 0.70 bar. +> Applying the same scrutiny used on §5.10's reversal: **graded down from +> "established" to "still suggestive"**, and the pre-registration's own +> pooling rule is flagged as having a real gap (no contingency for a large +> new sample disagreeing with small prior fragments). The directional claim +> remains **not established** after four probes and three independent +> samples (n=41 total storms). The structural claim (wn1 dominance, +> explanatory power) is untouched throughout. + +Product consequence `[S]`: a storm's pressure field compresses to a +**center address (metadata) + 14 logical fit values** — ~12 ring means plus a +2-value dipole — at **90.9–94.3 %** +variance explained (the 14-value constrained model — see the correction block +above; the 93–97 % figure belonged to a 36-parameter model and is superseded). +The dipole's orientation is *suggestively* related to the motion vector but +**that relation is NOT established** (§5.11, CT-F14: 0.684, p=0.0835 on the +largest sample) — so the compression claim stands on its own, and the +"single-frame motion predictor" reading does not yet. The representation is a +natural fit for the substrate's 3-integer spiral addressing (`highheelbgz`), +and those 14 logical values are **measured to fit the L4 carrier**: the +encoding is a **12-byte `6×(8:8)` facet** (10 ring bytes spread over the radius +with 2 interpolated, plus a 2-byte dipole rail), recovering the f64 spine to +within 0.07 Pa RMSE (`l4_rail_probe.py`, §6.1). **14 logical values ≠ 14 +bytes** — the model size and the carrier budget are different quantities and +are kept separate throughout. + +--- + +## 2. Physical basis (existing theory, nothing invented) + +All `[G]`, textbook dynamic meteorology: + +1. **A translating vortex is a vortex embedded in a steering flow.** In the + vortex-relative frame the environment appears as a superposed background + flow (the "airplane relative wind" framing). +2. **Geostrophy makes the geometry signed.** If the vortex translates with + the geostrophic steering flow `v_g = (1/fρ) k×∇p`, the background pressure + gradient is *perpendicular* to the motion, with the **low pole 90° to the + LEFT of the motion vector** (Northern Hemisphere; sign flips in SH). +3. **A linear background gradient is pure wavenumber-1.** Around any circle + centered on the vortex, a linear field `p_bg = a·x + b·y` has zero ring + mean and projects entirely onto `cos(θ−θ₀)` with amplitude + `√(a²+b²)·r` — i.e. it survives ring-profile removal *completely* as a + wn-1 residual growing linearly in radius. + +So the theory yields three independent, falsifiable predictions: dominance of +wn-1 in the residual, a *signed* orientation locked to the motion vector, and +`a₁(r) ∝ r`. The probe tests all three. + +Caveat stated up front `[G]`: real extratropical cyclones are baroclinic and +are steered by the mid-tropospheric flow, not the surface gradient alone; the +surface-level prediction is therefore expected to hold *up to a systematic +rotation* (see §6). The probe's ±45° tolerance was chosen to admit that +rotation while still rejecting the null (uniform orientation) at 0.25 per +storm. + +--- + +## 3. Data and method + +- **Domain:** disk of R = 1200 km around the detected center, planar + local-tangent geometry with `cos(lat_center)` zonal metric; 100 km rings. +- **Center finding:** deepest zonal-anomaly MSLP minimum; at t+6h the center + is re-found within a 600 km search radius (trackability gate CT-E2). +- **Decomposition:** per-ring mean → radial profile `p̄(r)`; per-ring + least-squares wavenumber-1 fit + `a₁(r)·cos θ + b₁(r)·sin θ` on the residual; amplitude-weighted mean dipole + bearing across rings → the **low-pole bearing** (`+π` from the high pole of + the fitted dipole). +- **Motion:** bearing of the 6h center displacement; predicted low pole = + motion bearing + 90° CCW. +- **Storms:** (1) 55.75N 334.5E — the arc's reference storm; (2) 67.0N 28.0E + — an independent center found by the preceding go-territory probe, used as + replication. + +--- + +## 4. Falsification design and results + +All bars pre-registered in the probe docstring *before* the run; the run was +committed unmodified. Null model for CT-E3: uniform dipole orientation → +P(hit ±45°) = 0.25 per storm; 2/2 joint = 0.0625. n=2 is stated, not hidden. + +| Bar | Pre-registered criterion | Storm 1 (55.75N) | Storm 2 (67N) | Verdict | +|---|---|---|---|---| +| **CT-E1** | wn-1 ≥ 0.40 of azimuthal-residual variance | **0.924** | **0.895** | PASS — wn-1 is not merely dominant, it is nearly *all* of the asymmetry | +| **CT-E2** | trackable: displacement ≥ 100 km / 6h within 600 km | 279 km | 440 km | PASS | +| **CT-E3** | **signed:** low pole = motion + 90° CCW, within ±45° | error **−42.0°** | error **−40.2°** | PASS 2/2, same side both storms | +| **CT-E4** | profile + wn-1 explains ≥ 0.80 of in-disk variance | 0.635 → **0.972** | 0.294 → **0.926** | PASS | +| **CT-E5** | observation (no bar): `a₁(r) ∝ r` | corr **0.800** | corr **0.998** | consistent with linear-background signature | + +**Interpretation discipline:** CT-E1/E4/E5 are unsigned goodness-of-fit +results — a skeptic could attribute them to "any smooth large-scale gradient." +CT-E3 is the load-bearing test: it is *signed by an independent quantity* +(the motion vector, measured from a different pair of fields), and both storms +land on the predicted side. That is what elevates this from curve-fitting to +physics `[H at n=2]`. + +> **⚠ CT-E3 RE-GRADED 2026-08-11 by CT-F3 (§5.1), as §5's pre-registration +> required.** The ±45° HIT stands — but the *offset magnitude* does not. A +> ±100 km center jitter moves the alignment error by up to **29.4°** (storm 1), +> which is comparable to the offset itself. So "−42° / −40°" is **NOT a robust +> number**: read CT-E3 as *"the low pole lies left-of-motion, within an +> apparatus uncertainty of roughly ±15°"*, and read the specific offset as +> unresolved at this centering precision. The *height dependence* of the error +> (§5.2) is a separate and much larger signal and is not affected by this +> re-grade. +> +> > **⚠⚠ RE-GRADE AMENDED 2026-08-11 by CT-F4 (§5.6).** The note above is +> > kept verbatim because it was correct *for the amplitude it tested* — but +> > ±100 km was an amplitude I **chose**, not one I measured. Four +> > independent center definitions (three different physical fields) turn out +> > to agree to **20 km** on storm 1 and **73 km** on storm 2, and the +> > alignment error they produce spans only **2.3° / 6.5°**. So the operative +> > apparatus uncertainty is **≈ ±3–7°, not ±15°**, and the −42°/−40° offset +> > **is** measurable above it. F3 was not wrong; its jitter amplitude was +> > unjustified — which is exactly what F4 was pre-registered to find out. + +**What this resolves:** the earlier sunflower-lattice probe failed its +axisymmetry bar (E1: 0.639 < 0.70) — the missing third of the storm was +unexplained. It *was* the tail: adding one dipole per ring takes the same +storm from R² 0.635 to 0.972. The golden-spiral/ring encoding was not wrong, +it was incomplete by exactly one mode. + +--- + +## 5. The systematic −40° offset — follow-up work (NOT yet run) + +Both storms miss the naive 90°-left prediction by **−42° and −40°** — nearly +identical magnitude, same rotation sense. With n=2 this is an observation +`[S]`, but a common offset of matched size is the signature of a *systematic +mechanism*, not noise. Three candidates, ranked: + +1. **Steering-level / baroclinic-tilt rotation** `[S]`, prime candidate. + Extratropical cyclones translate with the mid-tropospheric (≈500–700 hPa) + steering flow. The *surface* background gradient is rotated relative to + the steering-level gradient by the thermal wind (the system's westward + tilt with height). The probe measured the dipole at MSLP but the motion is + set aloft — a fixed rotation between the two is expected, not anomalous. +2. **Ekman / surface-friction turning** `[S]`. Boundary-layer friction turns + the surface flow 10–30° cross-isobar toward low pressure (more over land, + less over open ocean) and drives Ekman pumping that distorts the surface + pressure asymmetry. Magnitude range is plausibly consistent with −40° in + combination with (1), unlikely to explain it alone over ocean. +3. **Center-finder bias** `[S]`, must be excluded before believing either + mechanism. "Deepest zonal-anomaly point" ≠ circulation center; a center + displaced along-track biases the fitted dipole orientation. + +**RUN 2026-08-11** — `comet_tail_followup.py` / `.json`. Bars pre-registered in +that probe's docstring before the run; committed unmodified except for one +`grad_p` shape bug fix and one added *diagnostic* field (both recorded in the +probe's own RUN LOG; **no bar was added, removed, or loosened**). + +### 5.1 CT-F3 — apparatus — **FAILED the gate** (candidate 3 is live) + +Six center choices per storm: MSLP minimum (baseline), a ∇²p-centroid +(geostrophic-vorticity proxy), and ±100 km jitters along- and across-track. + +| Storm | baseline | ∇²p centroid | jitter range | spread | verdict | +|---|---|---|---|---|---| +| 1 (55.75N) | −42.0° | −42.0° (same grid point) | −25.4° … −54.8° | **29.4°** | APPARATUS-DOMINATED | +| 2 (67N) | −40.2° | −38.6° (toward zero, marginal) | −29.8° … −49.3° | **19.4°** | SURVIVES-WITH-UNCERTAINTY | + +Worst spread 29.4° **> the 20° bar → gate FAILED.** Per the pre-registration +this forces two things, both done: CT-E3 is re-graded in §4, and CT-F1/CT-F2 +below are reported as *measured but gated* — not as settled verdicts. + +The mechanism is understood and was anticipated: a 100 km miscentering of a +monopole injects a wn-1 by construction. What the test establishes is that the +**offset magnitude is inside the apparatus noise**, so no offset constant may +be derived from it. `[G]` + +*Post-hoc observation, explicitly NOT a rescue of the failed gate:* the +level-dependence signal in §5.2 is 92–102°, i.e. **3–5× this apparatus noise**. +That does not un-fail F3 — it means the right next probe is F3 re-run *at the +level where the error crosses zero*, with a sub-grid center fit. Recording the +comparison and letting the failed gate generate the next probe is the +disciplined move; overriding the gate on the strength of it would be the +"indictment fired, post-hoc rescue" anti-pattern this arc already has on its +open-P1 list. + +### 5.2 CT-F1 — steering level — **strong signal, formally mixed** (candidate 1 favoured) + +The store ships all 13 pressure levels in one chunk, so the yes/no became a +sweep. Alignment error vs the *same surface-measured motion bearing*: + +| level | storm 1 (own ctr) | storm 2 (sfc ctr) | +|---|---|---| +| 1000 hPa | −40.5° | −39.7° | +| 925 | −32.5° | −29.9° | +| 850 | −23.8° | −22.3° | +| 700 | −8.1° | −12.9° | +| 600 | **−2.1°** | −7.7° | +| 500 | +2.9° | −2.8° | +| 400 | +8.7° | **+1.0°** | +| 300 | +7.8° | +2.0° | + +**Both storms show a smooth, monotone climb from ≈ −40° at the surface through +zero in the mid-troposphere** — storm 1 crosses at ~600–650 hPa, storm 2 at +~400–500 hPa. Spread across levels: 101.8° / 91.6°. This is exactly the +baroclinic-tilt/steering-level prediction: the surface gradient is rotated +relative to the steering-level gradient, and the rotation unwinds with height. +`[H]` — the shape is unambiguous, n is still 2. + +Formal bar bookkeeping, stated rather than smoothed: storm 1 **PASSES** as +written (minimum at 600 hPa, |err| 2.1°). Storm 2's *own-center* column +minimises at 100 hPa (+2.0°), which trips the pre-registered `dead-absurd` +flag — **but the added diagnostic shows why:** at 50–400 hPa the center finder +**saturated at its 600 km search radius** (586–599 km), i.e. it never found a +co-located upper center and locked onto a different system. Those rows are +apparatus, not physics. The surface-center column (unsaturated by +construction) is the one tabulated above and behaves like storm 1. This is a +real defect in F1's own apparatus, found by a diagnostic added after run 1; +the honest verdict for storm 2 is **NO-VERDICT on the own-center path**, not +"dead". + +### 5.3 CT-F2 — friction — **bounded, candidate 2 is a contributor not the cause** + +Measured 10m cross-isobar inflow angle (positive = turned toward the low, the +NH friction sign), rings 300–1000 km, |v10| > 3 m/s: + +| storm | n | land frac | median α | IQR | ocean-only median | +|---|---|---|---|---|---| +| 1 | 6552 | 0.01 | **+14.7°** | +10.1 … +18.2 | +14.7° (n=6517) | +| 2 | 8960 | 0.46 | +22.0° | +10.5 … +34.7 | **+13.0°** (n=4674) | + +Sign is positive on both storms, as predicted. Magnitude is textbook for open +ocean (10–30°). The bar that matters: **13–15° ≪ 40°**, so friction alone +**cannot** own the offset — at most about a third of it. `[G]` for the +measurement, `[H]` for the attribution. + +*Apparatus can-it-fire check, unplanned but load-bearing:* storm 2 is 46 % +land, and its all-points median (+22.0°) is substantially larger than its +ocean-only median (+13.0°) — friction turning is stronger over land, exactly +as textbook. The measurement therefore discriminates a known physical contrast +in the right direction, which is evidence it is measuring what it claims. + +### 5.6 CT-F4 — sub-grid center — **the blocking item CLEARS** (candidate 3 mostly retired) + +`comet_tail_f4_f7.py` / `.json`. F3 showed sensitivity at an amplitude I chose. +F4 asks the non-circular question instead: **how far apart do independent +center definitions actually land?** That disagreement *is* the center +uncertainty. Four definitions, deliberately not variants of one idea — A: +sub-grid MSLP minimum (2D quadratic on the 3×3); B: ∇²p centroid (pressure +curvature); C: 10m relative-vorticity centroid (**wind** field); D: 850 hPa +geopotential minimum, sub-grid (different field *and* altitude). + +| | A (MSLP) | B (∇²p) | C (10m ζ) | D (z850) | max separation | error spread | +|---|---|---|---|---|---|---| +| storm 1 | −42.5° | −41.0° | −41.8° | −43.4° | **20.3 km** | **2.3°** | +| storm 2 | −39.8° | −38.6° | −35.1° | −41.6° | **73.0 km** | **6.5°** | + +- **Storm 2 PASSES CT-F4a** cleanly (6.5° ≤ 10° bar) with the anti-vacuity + guard satisfied — its four definitions genuinely disagree (73 km = 2.4× the + 29.9 km grid diagonal), so the test had real room to fail and did not. +- **Storm 1 is NO-VERDICT on F4a by my own CT-F4c guard**: its four definitions + agree to 20.3 km, *below* the 31.9 km grid diagonal. A 2.3° spread among + centers that coincide proves nothing about method sensitivity, so the guard + correctly refuses the free pass. Storm 1's bound comes instead from F4b + (below), and is labelled as derived, not as an F4a result. +- **CT-F4b sensitivity curve** (spread in ° vs jitter amplitude in km), + monotone on both as pre-registered: + + | | 25 km | 50 km | 100 km | 200 km | + |---|---|---|---|---| + | storm 1 | 6.5° | 13.8° | 30.9° | 77.2° | + | storm 2 | 5.0° | 9.8° | 19.3° | 35.8° | + + Read against the *measured* uncertainties: storm 1's 20 km sits below the + 25 km point → **≤ 6.5°**; storm 2's 73 km would predict ~14° isotropically, + yet the direct F4a measurement is 6.5°. The gap is informative: real center + definitions do **not** scatter isotropically — they cluster along a preferred + axis, so the four-direction jitter is an **upper bound**, not an estimate. + That is the second reason F3's number over-stated the problem. `[H]` + +**Consequence:** the offset is measurable with an error bar of roughly ±3–7°. +CT-F4 was the blocking item ahead of any constant-fitting; it is cleared for +storm 2 and bounded for storm 1. Candidate 3 (center-finder bias) drops from +"live and not excluded" to **bounded at ≈±5°, i.e. ~⅛ of the offset**. `[G]` + +*One observation worth a follow-up, n=1:* on storm 2 the **wind-based** center +(C) sits ~65 km north of the pressure minimum and moves the error the furthest +toward zero (−35.1° vs −39.8°). Whether the circulation center is +systematically the better reference for this test is untested. + +### 5.7 CT-F7 — friction over LAND — **both bars pass, and it re-scopes candidate 2** + +Operator-requested replication over land. Storm selected **blind to its +inflow**: deepest NH zonal-anomaly low whose 300–1000 km ring is ≥ 70 % land. +Selection returned **28.50N 67.50E** (ring 80 % land, anomaly −2609 Pa). +Orography guard dropped 826 of 2642 land points (31 %) above 1000 m, since +MSLP over high terrain is an extrapolated fiction whose gradient would corrupt +the geostrophic reference. + +| | n | median α | verdict | +|---|---|---|---| +| **land** points (oro-guarded) | 1816 | **+34.2°** (IQR reported in JSON) | | +| **ocean** points, *same disk* | 743 | **+20.5°** | | +| CT-F2 reference (storm 1, 99 % ocean) | 6517 | +14.7° | | + +- **F7a PASSES** (34.2° ≥ 14.7+8, inside [20,50]) — but this half is + confounded by latitude: at 28.5N, *f* is roughly half its 56N value, so + ageostrophic effects are inherently larger. Its own storm's ocean points + (+20.5°) already run above storm 1's (+14.7°) for that reason. +- **F7b PASSES** and is the number to trust: **+34.2° land vs +20.5° ocean + inside the same disk**, a +13.7° paired contrast that controls for depth, + latitude and curvature by construction. Both classes clear the n ≥ 500 bar, + so the half is genuinely evaluable rather than NO-VERDICT. +- **F7d is False** — 34.2° < the 35° threshold, though not by much. Over + *land* friction turning is roughly 2.4× its ocean value, so the friction + bound is **surface-type dependent and must not be applied globally**, exactly + as pre-warned. + +Two honest caveats. **(i)** The blind selection was blind to storm *type* as +well as to the answer: at 28.5N in mid-June this is a monsoon-season thermal +low, not an extratropical cyclone. It is a legitimate test of the *inflow +apparatus* over land, and a weaker analogue of storms 1–2 dynamically. +**(ii)** The apparatus passes its own can-it-fire test twice now — land/ocean +within one storm here, and land/ocean within storm 2 in §5.3 — in the same +direction both times. + +> **Self-correction, and it matters more than the pass.** F2/F7 measure the +> **wind** turning relative to the isobars. The CT-E3 offset is a rotation of +> the **pressure dipole** relative to the motion. Friction rotates the surface +> wind *within* a given pressure field; it does not rotate the pressure field +> itself, except at second order through Ekman-pumping feedback. So candidate 2 +> as originally written in §5 was **partly mis-specified** — F2/F7 bound a +> mechanism that was never the leading route to a pressure-dipole rotation. +> `[H]` + +> **An unplanned paired natural experiment, post-hoc but clean.** Storm 1 is +> **1 % land** with +14.7° inflow; storm 2 is **46 % land** with +22.0° inflow +> — a real 7.3° difference in actual surface friction. Their offsets are +> **−42.0° and −40.2°**, within 1.8° of each other, and the *more* frictional +> storm has the *smaller* offset. If friction drove the offset the two should +> separate substantially; they do not. This was not designed — it fell out of +> the land fractions already in the CT-F2 output — and it is n=2, so it is +> recorded as suggestive, not decisive. `[S]` + +### 5.8 CT-F5 — walking-center geopotential sweep — **fixes storm 2's saturation defect** + +`comet_tail_f5_n10.py` / `.json`. §5.2 flagged that CT-F1's storm-2 own-center +path had a real apparatus defect: searching for each level's center within a +*fixed* 600 km radius of the *surface* center saturated at 5–7 of 13 levels +(offsets 586–599 km, essentially pinned at the search wall), producing a +physically-absurd "best level = 100 hPa." + +**Fix:** walk the center level-by-level, searching near the *previous level's* +found center (radius 250 km per step, surface-anchored, 1000 hPa → 50 hPa) — +the center tracks continuously along the tilt axis instead of jumping the +whole tilt from the surface in one hop. + +| bar | criterion | storm 1 | storm 2 | +|---|---|---|---| +| **CT-F5a** | best \|error\| in 400–850 hPa ≤ 20° | −2.1° @ 600 hPa — **PASS** | **−4.5° @ 500 hPa — PASS** | +| CT-F5b | no single step > 250 km | max step 243.3 km — PASS | max step **250.0 km — FAIL** (exactly at the cap, at the 500→400 hPa transition) | +| CT-F5c | storm 1 reproduces original within 10° | **0.0° difference** (identical) | n/a | + +**F5a is the one that mattered, and it passes cleanly.** Storm 2's winning +level (500 hPa, −4.5°) was reached by a **0.0 km step** — the 600 hPa position +already coincided with the 500 hPa center — so the value that clears the bar +is untouched by any saturation. The saturation CT-F5b caught happens one step +*later*, moving from 500 hPa to 400 hPa, i.e. **outside** the band the bar +evaluates. Both storms also show a large, likely-unrelated excursion at +50 hPa (+101° / +92°, wn1_frac dropping to 0.78) — near-stratospheric, outside +the pre-registered 400–850 hPa band, not interpreted further here. + +*Housekeeping, stated honestly:* CT-F5c was pre-registered but never coded as +an explicit pass/fail field; verified post-hoc from the printed numbers — +storm 1's walking result at every 400–850 hPa level is *bit-identical* to the +original F1 sweep (both used the surface center throughout, since storm 1 +never needed to move before 600 hPa), so F5c passes trivially. Recorded as a +gap in this probe's own execution, not smoothed over. + +**Net: storm 2's dead-absurd/NO-VERDICT status from §5.2 is corrected to a +genuine PASS**, driven by an unsaturated intermediate level. The height ladder +(§5.2) is confirmed rather than weakened by fixing this defect. + +### 5.9 CT-N — n=10 blind storm sample — **the headline result of this arc** + +`comet_tail_f5_n10.py` / `.json`. Ten independent synoptic times (2015–2021, +all four seasons, NH), each storm found **blind** (no hint, no inspection +before recording — deepest zonal-anomaly MSLP low, 25–75° lat). Storm 1's +anchor date is included and its t-index reproduces the arc's pinned +T0 = 91246 exactly (guard asserted in code before anything else runs). + +*Data-boundary finding, unplanned:* the store's own filename claims +"1959-2022" coverage; its actual last valid timestep is **2021-12-31 18Z**, +six months short. One planned date (2022-02-14) 404'd against this; a bounds +guard was added (report + exclude, never crash) and that date swapped for an +in-range one. No pre-registered bar was touched by this fix. + +| storm | disp (km/6h) | E1 wn1_frac | E4 R² | E3 error | F8 error@vort-ctr | shrinks? | +|---|---:|---:|---:|---:|---:|:---:| +| 2019-03-05 | 455 | 0.72 | 0.930 | −67.8° | −109.6° | no | +| 2020-07-20 | 406 | 0.23 | 0.323 | −39.0° | −36.8° | yes | +| 2021-06-15 (anchor) | 277 | 0.92 | 0.972 | −41.3° | −40.6° | yes | +| 2020-01-10 | 250 | 0.75 | 0.906 | +5.0° | +4.5° | yes | +| 2019-10-25 | 185 | 0.38 | 0.894 | −107.6° | −103.3° | yes | +| 2017-11-30 | 158 | 0.53 | 0.908 | −49.1° | −101.2° | no | +| 2014-09-12 | 156 | 0.49 | 0.871 | +3.2° | −12.2° | no | +| 2015-12-25 | 132 | 0.76 | 0.887 | −19.7° | −15.0° | yes | +| 2018-08-08 | 128 | 0.87 | 0.919 | +19.4° | +18.4° | yes | +| 2016-04-18 | 113 | 0.73 | 0.830 | **+165.7°** | +166.6° | no | + +All 10 trackable (CT-E2 ≥ 100 km); none excluded. + +| bar | criterion | result | verdict | +|---|---|---|---| +| **CT-N1** | sign consistency ≥ 0.70 | **6/10 = 0.60** | **FAIL** | +| CT-N2 | magnitude (observation, no bar) | median \|error\| = **40.2°**, IQR [19.5°, 63.2°] | — | +| CT-N3 | median wn1_frac ≥ 0.40 | **0.723** | PASS | +| CT-N4 | median R² ≥ 0.80 | **0.900** | PASS | +| CT-N5 / F8 | vort-center shrinks error ≥ 0.70 | **6/10 = 0.60** | **FAIL** | +| CT-F9 | corr(land-dipole amp, unexplained residual) | **−0.295** | does not support candidate 2 | + +**The wn1-dominance and explanatory-power claims (E1/E4, CT-N3/N4) replicate +robustly at scale — the signed left-of-motion claim (CT-E3, CT-N1) does +not.** This is the honest headline: what looked like a clean 2/2 confirmation +at p = 0.0625 is, on 10 independent storms, statistically indistinguishable +from a coin flip (a naive binomial null at p=0.5 already gives P(≥6/10) ≈ +0.38 — nowhere near rejecting "no signed relationship"). CT-N5/F8 shows the +earlier observation that a wind-based center shrinks the error (seen on +storm 2 in §5.6) also does **not** generalize. CT-F9 gives a clean, +unambiguous non-support for the Ekman-pumping-residual route to candidate 2. + +**Post-hoc stratification — a lead for the next probe, explicitly NOT used to +override CT-N1's FAIL.** Sorting by displacement (a proxy for how well the +motion *bearing* itself is determined — small 6h displacement means a large +relative error on the direction label CT-E3 is scored against): + +- Restricting to the 4 storms with displacement ≥ 250 km (closest to storms + 1–2's own 277/440 km regime): sign consistency rises to **3/4 = 0.75**, + clearing the original 0.70 bar. +- Dropping only the single most extreme case (2016-04-18: 113 km + displacement, near-polar 75°N where the planar `cos(lat)` approximation is + already flagged degrading in §7, error +165.7° — essentially orthogonal to + the prediction): sign consistency rises to **6/9 = 0.667**, still short but + closer. + +Two candidate confounds, named rather than smuggled into the verdict: **(a) +motion-bearing noise at low displacement** — a mechanical apparatus effect, +symmetric in principle; **(b) storm-type contamination in a purely blind +sample** — 2020-07-20 (32°N, 84.6°E, mid-monsoon-season) has the sample's +worst wn1_frac (0.23) *and* R² (0.32), consistent with a monsoon/thermal low +rather than a baroclinic extratropical system the whole steering-flow argument +targets (the same caveat CT-F7's land storm already carried, §5.7). Neither +is fitted or applied here. The properly pre-registered next step is a +displacement-and/or-regime-filtered n ≥ 10 sample designed *in advance* to +test candidate (a) and (b) separately, not a re-scoring of this one. + +### 5.10 CT-F10/F11/F13 — a second independent sample — **borderline, in a specific and honest way** + +`comet_tail_f10_f11.py` / `.json`. §5.9 named two post-hoc leads and filed +them as pre-registered reruns rather than a re-scoring. This is that rerun, +on a **fresh, mechanically-generated** date set — fixed start (1980-02-10) ++ fixed stride (411 days, chosen for no reason tied to any outcome) × +15 candidates, landing entirely in **1980–1995**, zero overlap with the +2015–2021 sample. The stride was picked once, before any code ran that could +see a result; the dates were never inspected before recording. + +**Attrition was real and different this time:** 5 of 15 candidates (33 %) +failed CT-E2 trackability — versus 0 of 10 in the first sample. **4 of those +5** cluster at 26–33°N, 67–134°E in June–September — the same South/East-Asian +monsoon-season geography already flagged as contamination-prone (§5.9's worst +storm at 32°N/85°E; §5.7's blind land storm at 28.5°N/68°E). The 5th +(1995-11-12, 54°N/161°E, November) sits outside that band and is an ordinary +mid-latitude exclusion. This is a genuine, unplanned, cross-sample-consistent +pattern about **where blind NH-wide selection breaks**, not yet exploited by +any filter. + +| test | n | sign-neg fraction | median \|error\| | one-sided p (H₀: p=0.5) | verdict | +|---|---:|---:|---:|---:|---| +| **CT-F13** (raw, unfiltered — direct replication check) | 10 | **0.80** (8/10) | 41.3° | 0.055 | **PASS** (≥0.70 bar) | +| CT-F10 (disp ≥ 250 km) | 3 | 1.00 (3/3) | — | — | **NO-VERDICT** (n<6, pre-registered minimum) | +| **CT-F11** (wn1_frac ≥ 0.40) | 9 | **0.89** (8/9) | 41.9° | — | **PASS** | +| CT-F12 (both filters) | 3 | 1.00 (3/3) | — | — | reported only, n too small | + +**The headline number reverses §5.9's verdict on its own bar** — 0.80 vs the +first sample's 0.60. That reversal is exactly why it needs more scrutiny, not +less, and the honest statistics do not let it stand alone: + +- **Neither sample is significant on its own** at a conventional two-sided + 0.05: sample 1 (6/10) two-sided p = 0.754; sample 2 (8/10) two-sided + p = 0.109. One-sided (the physically motivated direction, pre-registered + from §2 onward) sample 2 gives p = 0.055 — genuinely borderline, not a + clean win. +- **Combined across both fully independent samples: 14/20 = 0.70, + one-sided p = 0.058.** Still borderline. Landing exactly on the 0.70 bar is + a coincidence worth naming, not a rescue: the bar was picked before either + sample ran. +- **The regime filter (CT-F11) does NOT rescue sample 1 — checked, and it + makes things worse.** Applying wn1_frac ≥ 0.40 retroactively to sample 1 + removes 2020-07-20 and 2019-10-25, **both of which were negative-signed** + (−39.0° and −107.6°). Removing two "hits" drops sample 1 from 6/10 (0.60) + to **4/8 = 0.500 — exactly chance.** This directly contradicts the + regime-contamination story as an explanation for sample 1's specific FAIL: + its two lowest-wn1_frac storms happened to agree with the prediction, not + disagree with it. Recorded as measured, not smoothed into the flattering + reading. +- **The strongest single number in this whole probe chain is a post-hoc + combination, clearly labeled as such — not pre-registered:** pooling the + displacement ≥ 250 km subsets from *both* independent samples (n=4 + n=3 = + 7, spanning two different decades) gives **6/7 = 0.857 negative, + one-sided p = 0.0625.** Displacement is a *mechanism-motivated* filter + (it bounds motion-bearing noise, §5.9), not an outcome-based one, which is + why this number carries more weight than F13's raw 0.80 despite being + numerically less extreme — but n=7 is still small and this combination + was not itself pre-registered before either sample ran. + +**Net read, stated as precisely as the evidence allows:** the signed +left-of-motion claim is **not dead** (§5.9 was right to fail CT-N1 as +written, and this rerun does not overturn that FAIL under its own bar with +proper statistics) but it is **not established** either. What *does* survive +scrutiny is a consistent pattern across two independent decades: **storms +with well-determined motion (large 6h displacement) show a much stronger, +borderline-significant left-of-motion signature (6/7 combined) than storms +with poorly-determined motion or unfiltered blind selection (14/20, exactly +at the noise floor).** That is evidence *for* the apparatus explanation (§5.9 +candidate (a), motion-bearing noise) and *against* the regime explanation +(§5.9 candidate (b), storm-type contamination) as the dominant driver — the +opposite weighting from what seemed most plausible after §5.9 alone, and +worth stating exactly because it cuts against the tidier story. + +**~~CT-F14 (new, not run)~~ — RUN, see §5.11.** A properly powered, +pre-registered, displacement-filtered-ONLY sample — the single test that would +move the combined 6/7 either toward significance or back to noise. **Result: +19 qualifying storms, 0.684 (13/19), p=0.0835 → NO-VERDICT** (one short of its +own n ≥ 20 floor, and it would have failed the 0.70 bar anyway). + +> **Two drifts recorded rather than tidied away** (CodeRabbit, PR #926; note +> the finding was reported as already-addressed and was **not** — verified +> against the file, which is why it is fixed here). (a) This forward-looking +> text said "not run" after §5.11 had run it. (b) It specified **n ≥ 25–30** +> while the test as actually pre-registered and executed used a **n ≥ 20** +> floor. The floor was loosened between planning and execution and nobody +> noticed — the same claim-vs-measurement drift as +> `E-THE-HEADLINE-NUMBER-MEASURED-A-MODEL-NOBODY-CLAIMED-1`, in the sample-size +> dimension instead of the parameter-count one. Worth knowing when reading +> §5.11's "fell one storm short": short of **20**, not of the 25–30 this +> paragraph originally asked for. +**CT-F15 (new, not run):** geo-fence the blind selection away from +26–33°N/67–134°E in June–September (or add an explicit baroclinicity +proxy at intake) and re-check whether CT-E2's own trackability gate keeps +doing this filtering job for free, as the exclusion pattern above suggests. + +### 5.11 CT-F14 — the properly-powered test — **fell one storm short of its own floor, and didn't clear the bar either** + +`comet_tail_f14.py` / `.json`. §5.10 named CT-F14 as the correctly-scoped +next step: a single, properly powered (n≥20), displacement-filtered-only, +pre-registered sample — committed to git (`4f1a1b4f`) *before execution*, +including the exact interpretation thresholds for a pooled three-sample +figure, so the read could not be tuned after seeing results. + +85 mechanically-generated candidates (fixed start 1996-01-15, fixed 61-day +stride, chosen before writing the loop), landing 1996–2010, zero overlap +with either prior sample. 64/85 trackable (75 % — closer to sample 1's 100 % +than sample 2's 67 %); **19 storms qualified** at displacement ≥ 250 km/6h — +one short of the pre-registered n=20 floor. + +| test | n | sign-neg fraction | one-sided p | verdict | +|---|---:|---:|---:|---| +| **CT-F14 alone** | 19 | 0.684 (13/19) | 0.0835 | **NO-VERDICT** (n<20, pre-registered floor) — and would have **FAILED** the 0.70 bar anyway if the floor were ignored | +| Pooled, all 3 independent samples | 26 | **0.731** (19/26) | **0.0145** | crosses the pre-committed <0.05 "established" threshold | + +**The pre-registered rule technically fires "established."** The honest +next step — applying the same scrutiny used on §5.10's favorable reversal — +is to check how much that crossing depends on which components are in the +pool, since two of the three components are very small: + +| pooled subset | n | fraction | one-sided p | +|---|---:|---:|---:| +| All three (pre-registered) | 26 | 0.731 | **0.0145** | +| Excluding the smallest/most saturated prior subsample (sample 2, n=3, 3/3) | 23 | 0.696 | 0.0466 | +| **CT-F14 alone (the properly-powered test)** | 19 | 0.684 | **0.0835** | +| The two small prior subsamples alone (n=7) | 7 | 0.857 | 0.0625 | + +**Two things are true at once, and both need to be said plainly.** (1) The +pooled figure is not purely an artifact of the smallest fragment — dropping +it still leaves p=0.0466, barely under 0.05. (2) **The single test this +whole probe was designed to produce — CT-F14 alone, n=19, the largest and +most carefully powered sample in the entire chain — does NOT independently +support the claim** (p=0.0835, "suggestive" by the arc's own pre-committed +scale, and its raw rate sits below the 0.70 bar). The "established" reading +depends on treating three heterogeneous samples (different eras, different +mechanical generators, sizes 4/3/19) as one undifferentiated pool of +Bernoulli trials — a legitimate but not the only reasonable pooling choice, +and one this probe's pre-registration did not anticipate needing to defend +against a properly-powered *new* sample landing lower than small prior +fragments. + +**Honest verdict, overriding the letter of the pre-committed rule where +scrutiny disagrees with it:** this is a **fragile pass, not an established +finding.** Graded down from "ready for the audit-gate queue" to +**suggestive, same tier as before CT-F14 ran** — because the properly-powered +component alone does not clear significance, and that component is the one +this whole exercise existed to produce. Recording this as a **gap in the +pre-registration itself**, not smoothed over: the interpretation thresholds +were written for the pooled figure without a contingency for "the new, +larger sample disagrees with the small prior fragments" — a real design +blind spot, named so a future pre-registration in this arc specifies it. + +**A second, unplanned walk-back, in the same direction.** §5.10 flagged a +striking pattern — 4 of 5 trackability exclusions in the 1980–1995 sample +clustered at 26–33°N/67–134°E in June–September. This sample's 21 +exclusions show the **same band catching only 2 of 21** — a much weaker +signal at 4× the exclusion count. The earlier 4/5 was very likely a small-n +inflation of a real-but-modest effect, not a strong reproducible pattern. +CT-F15 (the proposed geo-fence) is downgraded accordingly — worth a light +touch, not a structural fix. + +**What remains solid, restated:** the structural claim (wn1 dominance, +explanatory power) is untouched by any of this — CT-F14's own qualifying +subset has median wn1_frac 0.60, median R² 0.90, consistent with N3/N4. +Only the *directional/predictive* claim is affected, and it moves from +"borderline" (§5.10) to **"borderline, and the properly-powered test that +was supposed to settle it did not."** + +### 5.4 Where that leaves the three candidates + +> **Read this table as a within-storm-1/2 candidate ranking for the OFFSET +> MECHANISM.** §5.9 (CT-N) found the sign of the offset itself does not +> reliably generalize past those two storms — so the table below explains a +> phenomenon whose *universality* is now in question, not a confirmed +> atmospheric constant. Both findings stand together: storms 1–2's offset is +> real and its mechanism is best explained by candidate 1; whether *most* +> storms have a comparably-signed offset at all is open. + +*(Updated after F4/F7 — the F1–F3 column is kept so the movement is visible.)* + +| candidate | after F1–F3 | after F4 + F7 | +|---|---|---| +| 1 — steering-level / baroclinic tilt | Favoured | **Leading, and now near-unopposed** `[H]`. Unchanged evidence (92–102° monotone height ladder crossing zero mid-troposphere on both storms), but its two rivals have shrunk. | +| 2 — Ekman surface friction | Bounded contributor | **Re-scoped and demoted** `[H]`. The bound tightened (ocean 13–20°, land 34°) but the mechanism was partly mis-specified: friction rotates the *wind*, not the *pressure dipole*. The unplanned 1 %-vs-46 %-land pairing (§5.7) points the wrong way for it. | +| 3 — center-finder bias | LIVE, not excluded | **Bounded at ≈±5°** `[G]`, ~⅛ of the offset. Independent center definitions agree to 20 / 73 km and the error spans 2.3° / 6.5°. | + +Candidates remain **not mutually exclusive**. The residual budget is now +roughly: tilt ≈ most of the 40°, centering ≈ ±5°, friction ≈ second-order on +this quantity. No stronger attribution is claimed at n = 2. + +### 5.5 Next falsifiers + +- ~~**CT-F4** (sub-grid center, was blocking)~~ — **RUN, §5.6.** Storm 2 + PASSES; storm 1 NO-VERDICT by the anti-vacuity guard, bounded ≤ 6.5° via the + F4b curve. The apparatus is not what makes storms 1–2's own offset + unreliable. +- ~~**CT-F5**~~ — **RUN, §5.8.** Walking-center fix; storm 2's saturation + defect corrected, F5a passes cleanly. +- ~~**CT-F7** (friction over land)~~ — **RUN, §5.7.** Both bars pass; the + candidate it tests is re-scoped rather than confirmed. +- ~~**CT-F8** (wind-center generalization)~~ — **RUN, §5.9 (as CT-N5).** + Does **not** generalize: 6/10 = 0.60 vs the 0.70 bar. Storm 2's own + improvement (§5.6) was a single case, not a pattern. +- ~~**CT-N** (n=10 sample)~~ — **RUN, §5.9.** The headline result: sign + consistency of the offset itself is **6/10 = 0.60**, statistically + indistinguishable from chance. Magnitude/explanatory-power claims (E1, E4) + replicate; the signed claim (E3) does not, at this n. +- **CT-F6 (still open):** the crossing level (§5.2/5.8) as the observable — + *prediction:* it tracks the deep-layer mean steering level. Needs the same + n ≥ 10-with-regime-control treatment CT-N just showed is necessary. +- **CT-F9 (Ekman-pumping mechanism)** — **RUN, §5.9.** corr = −0.295, does + not support the residual-correlation pathway for candidate 2. +- ~~**CT-F10** (displacement-filtered rerun)~~ — **RUN, §5.10.** NO-VERDICT + standalone (n=3, below the pre-registered minimum) — but pooled with + sample 1's own displacement-filtered subset, 6/7 = 0.857, the strongest + number in the chain (still small-n, still not pre-registered as a pooled + test). +- ~~**CT-F11** (regime-filtered rerun)~~ — **RUN, §5.10.** Passes on sample 2 + (8/9), but **checked directly against sample 1 and found NOT to rescue + it** — retroactively applied, it drops sample 1 to exactly 0.500. The + regime-contamination explanation is weaker evidence than it looked after + §5.9 alone. +- **CT-F13 (raw replication, RUN as part of §5.10):** 8/10 on an independent + sample, reversing §5.9's verdict on its own bar — but not significant on + proper statistics (two-sided p=0.109), and the pooled combined figure with + sample 1 (14/20) sits right at the noise floor. +- **~~CT-F14 (new, from §5.10, NOT yet run)~~ — RUN, §5.11:** the + correctly-scoped step — a single, pre-registered + **displacement-filtered-only** sample. **Executed with a n ≥ 20 floor (not + the n ≥ 25–30 asked for here — see the drift note in §5.10); 19 storms, + 0.684, p=0.0835, NO-VERDICT.** The directional claim is therefore *not* + gated-and-pending; it is measured and unsupported at this power. +- **CT-F15 (new, from §5.10, NOT yet run):** geo-fence blind selection away + from 26–33°N/67–134°E in June–September (4 of 5 exclusions in the second + sample clustered there) and check whether this simply reproduces what + CT-E2's trackability gate is already doing for free. +- **Sample size:** the apparatus (CT-F4/F5) is good enough that an offset + constant COULD be fitted for storms in the large-displacement regime — but + §5.10 means even that regime is only *suggestively* supported (n=7, p≈0.06 + pooled) and CT-F14 is the gate before any constant is fitted, not general + n ≥ 10 in the abstract. + +--- + +## 6. Product / encoding consequence `[S]` + +> **⚠ Read with §5.9–5.11 AND the compression correction in §1.** The figures +> below say 93–97 %; the honest number for the 14-value model they describe is +> **90.9–94.3 %** (the 93–97 % belongs to a 36-parameter per-ring fit). The +> compression is real and generalized cleanly across every sample this arc ran +> (N3/N4, and CT-F14's own qualifying subset: median wn1_frac 0.60, R² 0.90). The +> *motion-encoding* half ("the dipole encodes the motion") depends on the +> signed relationship CT-F14 — the properly-powered test built to settle it — +> did NOT independently establish (§5.11). Treat the compression as ready +> for the audit-gate queue; treat the predictor as **suggestive at best**, +> not gated-and-pending but **not yet earned**. + +If CT-F1..F3 hold up, the compact representation of a surface low is: + +``` +storm ≈ CENTER (place) — 1 address + + p̄(r) ring-profile means — ~12 bytes (12 × 100 km rings, u8-quantizable per voxel-chess probe: u8 max dev 0.0047) + + (a₁,b₁) ONE dipole vector — 2 values (amplitude slope + bearing) + = 90.9–94.3 % of in-disk MSLP variance [corrected 2026-08-11; the + 93–97 % previously printed here was a 36-parameter per-ring fit, not + this 14-value model — see the §1 correction] +``` + +- Maps directly onto `highheelbgz`'s 3-integer spiral **address** form + (start, stride, length) + a short payload — the storm is stored as a place + plus ~14 bytes, values recomputed on demand, instead of a raster crop. +- **The dipole encodes the motion.** `bearing(low pole) − 90° (− offset)` + estimated the 6h displacement *direction* within tolerance on both storms + **from a single timestep**. Named next falsifier: the single-frame motion + predictor, evaluated on n ≥ 10 storms against the observed 6h track, with + the offset constant fitted on a disjoint training set. Not executed — + awaiting go, and gated behind the adversarial audit per the arc's standing + rule (exploratory probes are not EVs; the 0/11 lesson). + + > **⚠ Superseded by §5.11.** "The dipole encodes the motion" is the claim + > CT-F14 was built to settle and did **not** independently establish + > (0.684, p=0.0835, n=19 — the largest sample in the chain). Left in place + > per append-only, read as **suggestive, not established**. + +### 6.1 It fits the ACTUAL carrier — 12 bytes as `6 × (8:8)` `[H]` + +Operator correction, 2026-08-11: *"was ist mit 6× Palette256:Palette256 +centroid, was ja die Verteilung anzeigen soll — palette256 alleine ist ja nur +'attention header'"*. Every encoding in this arc had treated **one scalar → +one byte** as the unit. The shipped carrier is a **pair**: le-contract §3 row +L4 is `6 × (8:8)`, `palette256²` — "each byte pair indexes the 256×256 palette +distance/compose tables; similarity = ONE table read". A single byte is the +*selector*; the **pair** is a cell in the centroid tile, and the tile is where +the distribution lives. I had built one rail and called it the carrier. + +`l4_rail_probe.py`, four pre-registered bars, **two failed as written**. +Numbers below are the CORRECTED ones — see the ⚠ note at the end of this +section; an earlier version of this table used a `var()`-based R² that +flattered every biased carve. + +| carve | 12 B? | storm 1 R² | storm 2 R² | RMSE Pa (s1) | **bias Pa (s1)** | +|---|:--:|---:|---:|---:|---:| +| f64 constrained spine (reference) | — | 0.94344 | 0.90905 | 241.64 | +0.00 | +| **D — dipole rail + 10 ring bytes spread over the full radius** | **yes** | **0.94340** | **0.90903** | **241.71** | **+1.59** | +| A — dipole rail + rings 0–9, outer rings held | yes | 0.91287 | 0.90177 | 299.91 | **+92.76** | +| B — all 12 rings, no dipole rail | yes | 0.63479 | 0.29426 | — | — | + +**The 12-byte facet recovers the f64 spine to within 0.07 Pa RMSE (0.03 %), +carrying a +1.59 Pa mean bias.** It is **NOT lossless** — an earlier version of +this section said so, on the strength of an R² that agreed to four decimals, +and that was the wrong statistic to conclude it from (⚠ below). R² is +nonetheless demonstrably *sensitive* here — carve B, the same byte budget spent +differently, collapses to 0.63/0.29 — so carve D is genuine recovery, not +insensitivity. + +> **⚠ CORRECTION 2026-08-12 (CodeRabbit, PR #926) — R² was computed with +> `var()` instead of the uncentered MSE, at 11 sites across 8 probes.** +> `1 − var(y−ŷ)/var(y)` silently discards the squared MEAN residual, so any +> BIASED reconstruction is flattered. Measured consequences: +> - **Where a ring-mean profile is present the effect is exactly zero** +> (`mean(resid)` = 1e-12 by construction), so **every f64 headline in this +> report — 0.9434 / 0.9090 / 0.972 / 0.926 — is unchanged.** +> - **Carve A moved 0.9212 → 0.9129** and its `loss_dropped_rings` 0.0222 → +> 0.0306. It holds the two outer rings at a fixed value; `var()` could not +> see the resulting **+92.76 Pa** offset at all. +> - Carve D moved 0.943406 → 0.943403 (2.4e-06). +> +> **The deeper lesson, which is why the wording changed and not just the +> digits:** in-disk variance here is ~1e5 Pa², so a systematic offset of tens +> of Pa perturbs R² in the 5th decimal. **R² is structurally near-blind to +> exactly the defect that matters for an encoder**, and "lossless" was inferred +> from the one statistic that could not detect the loss. The probe now reports +> **RMSE and mean bias in Pa alongside every R²**, because those are what +> distinguish the carves. + +Three results worth more than the headline: + +- **L1 FAILED as written** (storm 1: **0.0306** against a 0.02 bar) and the + decomposition names the cause exactly: **quantization +0.0000, dropped + rings +0.0306**. The carrier's *precision* is free; its *capacity* was the + entire miss. Spending the same 12 bytes across the full radius (carve D) + rather than on the inner 10 plus a held edge erases it — and in Pa the gap + is far starker than R² suggests: carve A's held outer rings cost **+92.76 Pa + of bias**, carve D's **+1.59 Pa**. +- **L3 FAILED, and so did my proposed rescue.** Fisher-z centroid axes are + **5× worse** than uniform on the ring means (18.07 vs 3.84 Pa). I + hypothesised the population was wrong — ranks taken against the 24 encoded + values instead of the field — and measured that too (L3b): **19.00 Pa, no + rescue.** Mechanism: ring means are a smooth *narrow-band* quantity sitting + mid-distribution, so a rim-stretch spends levels in tails where no ring mean + lives. This does **not** contradict the three-register result (§6.2, Fisher-z + 8.3× *tighter* in the storm tail on the raw field) — it **demarcates** it: + Fisher-z wins a **rank/tail** read and loses an **interpolate/level** read. + Which is precisely why le-contract says a ClassView **MAY** declare an + analytic codebook — per class, by measurement. Corrects my own + over-generalization that Fisher-z is *the* L4 codebook axis. +- **L4x was vacuous on its first run.** A uniform codebook is fixed by its + population's min/max alone, so because storm 1's profile range strictly + *contains* storm 2's, storm 1's "own" codebook **is** the pooled codebook — + the bar compared an array against itself and passed for free. It looked + real only because an earlier variant gave differing numbers. Both directions + now carry an explicit degeneracy flag; the informative one (storm 2's + codebook applied to storm 1) gives **620.79 Pa vs 4.48 Pa shared, a 139× + penalty** — strong evidence the codebook must be global, which is exactly + the "one table read" property the carrier exists for. + +Scope: 2 storms, 1 timestep, 1 variable. This measures **structural fit to the +carrier**, and says nothing about forecast skill. + +### 6.2 Three registers over one byte — and why there is no absolute anchor `[H]` + +Operator, 2026-08-11: *"du lebst noch in der Vorstellung, dass alles absolut +ist — die Relativitätstheorie widerlegt sogar das"* + *"in der Statistik ist es +gold wert, alles auf Palette256 normalized zu haben"*. + +The correction: I had encoded pressure against a fixed 1000 hPa reference and +called it "a stable convention". It is an **absolute anchor, and there is +none** — 994 hPa is unremarkable over Iceland and a record in the subtropics. +What is meaningful is a value's **position in its own distribution**. + +`three_register_probe.py`, one byte per value, global ERA5 MSLP (n=1,038,240), +errors in Pa so the three are comparable at all: + +| band | n | A affine | B rank | C Fisher-z of rank | +|---|---:|---:|---:|---:| +| storm tail (bottom 1 %) | 10,383 | 10.71 | 204.54 | **24.74** | +| lower shoulder (1–10 %) | 93,450 | 10.76 | 51.11 | 45.38 | +| bulk (40–60 %) | 207,737 | 10.79 | **2.33** | 16.49 | +| high tail (top 1 %) | 10,387 | 10.71 | 88.27 | **6.86** | + +All five pre-registered bars PASS. The register that wins **depends on the +band, and no register wins everywhere**: A is flat by construction (ratio +1.01 — the control); B is 4.6× tighter than A in the bulk and 19× *worse* in +the storm tail; C is **8.3× tighter than B** in the storm tail, which is the +decisive comparison for a storm substrate, since a storm *is* a tail event. + +**R5, the "statistical gold" bar:** after rank-normalisation the same `u8` +denotes the same rarity in MSLP, 2 m temperature and 10 m wind — max spread +across all 256 bytes **0.00043 vs one bucket 0.00391**. For the *absolute* +register the same comparison is not merely worse, it is **undefined**: Pa, K +and m/s share no unit, so byte 128 of each denotes no common quantity. That +is the concrete cash value of rank-normalising onto one palette — cross- +variable distance becomes *defined*, and one LUT serves every field. + +Read §6.1's L3 against this table and the demarcation is exact: **Fisher-z +wins the tail read here (24.74 vs 204.54) and loses the level read there +(18.07 vs 3.84)**. Same substrate, opposite verdicts, and the discriminator is +what the read is *for* — not which codec is "better". + +Scope limit, stated rather than hidden: the reference distribution is the +**global field at one timestep**, the population a rolling floor would have +observed. Operational extremity wants a multi-year climatology, and a +two-sided/climatological calibration of the C register is an open item. + +## 7. Limitations and non-claims + +- **n=2, one synoptic time, one season, NH only.** No claim of climatological + generality; SH sign flip untested. +- **MSLP only.** No upper-air, no moisture, no intensity change — this is a + *structure* result, not a forecast skill result. +- **Planar geometry** within 1200 km (`cos(lat)` metric); fine at 55–67°N for + this radius, degrades toward the pole. +- **The 6h displacement is the label, not a forecast** — CT-E3 tests + consistency between two simultaneous measurements (dipole at t0, motion + t0→t1); the *predictor* framing is future work (§6). +- The ±45° bar was set to admit the expected baroclinic rotation. That was + the right call for a first signed test, but it means the current result + cannot distinguish candidates 1–3 in §5 — that is exactly what CT-F1..F3 + are for. + +## 8. Status and promotion path + +1. ~~Probe, pre-registered, run, committed~~ — DONE (`db57aac0`). +2. ~~This report~~ — DONE (`d9a98b86`). +3. ~~CT-F3 first (apparatus before mechanism), then CT-F1/CT-F2~~ — DONE. + **F3 FAILED its gate**; CT-E3 re-graded in §4; F1 strongly favours the + steering-level mechanism; F2 bounds friction to ~⅓ of the offset. +4. ~~**CT-F4 (sub-grid center) is the blocking item**~~ — **RUN and CLEARED** + (§5.6). Apparatus uncertainty ≈ ±3–7°; an offset constant is fittable *for + storms in the regime storms 1–2 sit in*. **CT-F5** (§5.8) fixed the + remaining saturation defect. **CT-F7** (§5.7) bounded and re-scoped + candidate 2 (friction rotates wind, not the pressure dipole). +5. ~~**n ≥ 10 storm sample**~~ — **RUN, §5.9.** The offset's **sign** does not + generalize on that first blind sample (6/10, indistinguishable from + chance), while its magnitude/explanatory-power claims do. CT-F8's + wind-center generalization and CT-F9's Ekman-pumping mechanism also + tested here: neither holds up at scale. +6. ~~**CT-F10 / CT-F11 / CT-F13**~~ — **RUN, §5.10. A second independent + sample, borderline in a specific direction.** A fresh 1980–1995 sample + scores 8/10 unfiltered — but proper statistics keep the combined figure + right at the noise floor (14/20, one-sided p≈0.058), and the + regime-filter explanation is checked and found NOT to rescue sample 1 + (drops it to exactly chance). What *does* strengthen is the + displacement-filtered pooled subset (6/7, p≈0.0625) — apparatus, not + regime, is now the better-supported explanation for the gap. +7. ~~**CT-F14**~~ — **RUN, §5.11. The single properly-powered test does NOT + independently support the claim** (n=19, one short of its own n≥20 floor; + 0.684, p=0.0835). The pooled 3-sample figure technically crosses the + pre-committed <0.05 threshold (p=0.0145), but the honest, scrutinized + verdict grades this down to **still suggestive** — the pre-registration's + own pooling rule had a gap, named rather than exploited. +8. Adversarial audit gate (plan §8) before any of it is promoted to EV / + product claim — moot for the directional claim until it clears its own + properly-powered test; live for the structural (compression) claim now. + +**Net effect of the full follow-up chain on the headline claim, stated +plainly after four probes and 41 total storms across three independent +samples.** The structural claim — wn-1 dominance, the ring-profile + dipole +compression, R² lift — **generalizes cleanly and has not been shaken once** +across every sample this arc ran. The directional/predictive claim — the +signed left-of-motion orientation that made storms 1–2 exciting — has now +been tested four separate ways (apparatus §5.6, land-fraction mechanism +§5.7, blind n=10 §5.9, a reversal on independent n=10 §5.10, a +properly-powered n=19 §5.11) and **still does not clear a real bar on its +own strongest test.** The offset's status across the chain: **dead (F3) → +alive with a ±3–7° error bar (F4) → apparently general (misread of an n=10 +in isolation) → reversed on a second n=10 → pooled-and-technically-passing +but not independently supported by the one test built to settle it (F14).** +That is not noise in the writing — it is the honest trajectory of a +borderline effect being measured with increasing rigor, and every step was a +genuine gate: a result that *helped* the claim (§5.10's reversal, CT-F14's +pooled crossing) got exactly the scrutiny a result that hurt it would have +gotten, and in both cases the scrutiny found reasons for caution that a less +careful pass would have missed. **Current position: the compression is +ready for the audit-gate queue now; the predictor is not, and should not be +represented as more than "suggestive" until a properly-powered test clears +its own bar without pooling assistance.** §6 is marked accordingly. + +## 8b. External review of PR #926 — what it changed (2026-08-11) + +16 findings (14 CodeRabbit + 2 Codex). Four changed measured numbers; the rest +were latent bugs or labelling. Recorded because two of them make the arc's own +results BETTER and one makes the headline WORSE — the review is not a formality. + +**Changed published numbers:** + +1. **The compression claim was measured on the wrong model** (Codex P1) — see + the §1 correction. 93–97 % → **90.9–94.3 %** for the 14-value model actually + claimed. The most consequential finding in the review. +2. **Sunflower E2 was not a controlled comparison** (Codex P2 + CodeRabbit): + `grid_pts(n)` returned every in-disk lattice point, so the grid arm ran on + 80 samples against the spiral's 64 (293 vs 256, 1085 vs 1024) — and + nearest-neighbour reconstruction improves with samples, so the arm being + compared was systematically advantaged. With EXACTLY n enforced the verdict + **improves in the spiral's favour**: 234.5 vs 269.0 Pa (n=64), 119.1 vs + 123.3 (n=256), 58.9 vs 59.9 (n=1024) — the spiral now wins at every budget, + where the earlier write-up recorded "parity". The original result was + PESSIMISTIC, not optimistic. +3. **The voxel-chess palette arm was a hybrid** (CodeRabbit): `geo_corr` + received the palette-derived geostrophic winds but closed over the + module-level RAW `u`/`v`, so "u8 max dev 0.0047" compared palette + geostrophy against raw observations — not the pre-registered palette + result. The observed fields are now explicit parameters and the palette arm + passes `u8`/`v8`. +4. **go_territory's explained variance re-centred the residual** (CodeRabbit): + `res.var()` subtracts the residual mean after every atom, excluding it from + the error. Fixed to a fixed centered-field denominator over the residual + mean-square; K=10 matched moves 0.530 → **0.523**, and **no verdict flips** + (A-E1 and A-E2 still fail their bars). + +**Vacuous assertion caught** (CodeRabbit): E6's `rises_then_decays` required +only an interior maximum plus a lower final value — it accepted a profile that +DECREASED before rising to the peak, which the committed run literally did +(12.190 → 12.163 m/s before the 525 km peak) while reporting `true`. Now +asserts monotone rise to the peak and decay after it, with a stated 0.05 m/s +tolerance; the run still passes, but now because the profile is Rankine-shaped +rather than because the test could not fail. + +**Latent bugs fixed (no committed run hit them, so no numbers move):** +`find_center` returned grid cell (0,0) when a `near`-limited mask was fully +masked, instead of `None`; `subgrid_min`'s 3×3 slice did not wrap in longitude +and would have raised on any centre at the 0° seam. Both are now guarded in +all six probes. + +**Labelling / provenance:** CT-F12 can no longer emit `pass: true` below its +evaluable minimum; F7d's threshold now matches the 40 deg its own key and +docstring pre-register (it tested 35); `comet_tail_followup.json` persists the +per-storm centre / bearing / displacement instead of `"storms": {}`; +`go_territory_probe.json` is written beside the probe rather than the cwd. + +## 9. Reframe — the spine is found; the moderators are missing (operator, 2026-08-11) + +Operator ruling on how to read the whole chain, and it is quantitatively +better than my "borderline" framing: + +> *"Wir haben ein Spine gefunden — die Stellschrauben müssen noch mit den +> Variablen der bekannten Modelle moduliert werden. Uns fehlen die +> Moderatoren; aber wir haben bereits das Gerüst, um das Zentrum und die +> Dynamik zu modellieren."* + +**Why this framing is a working hypothesis supported by a structured +residual — not spin, and not an established reading either.** A directional +main effect at 0.68–0.73 sign consistency +whose residual were *random* would be a dying claim. This chain's residual +is not random: it runs **monotonically with a measured variable** — the +height ladder (§5.2/5.8), ≈ −40° at 1000 hPa climbing smoothly through zero +in the mid-troposphere, spread 92–102°, 3–5× the measured apparatus noise, +on both storms it was measured on. *Main effect + structured residual + +identified covariate* is **consistent with a missing moderator and requires +independent validation**. `[H]` at the ladder's n=2; `[G]` that the framing +follows if the ladder replicates. + +> **Corrected 2026-08-11, and again 2026-08-12 (CodeRabbit, PR #926).** Round +> one softened the BODY; the **heading still said "it is the statistically +> correct reading"** — the same overclaim, one line above its own correction, +> which is the third instance in this document of fixing a claim in one place +> and leaving its twin. Round two rewrote the heading to "a working hypothesis +> supported by a structured residual". The original body read "*is the +> signature of a missing moderator, NOT of a null. A null does not produce a +> ladder.*" That overstates what a monotonic residual establishes: it supports +> the hypothesis without discriminating it from model misspecification, +> centre/label error, selection effects, or chance. The identical sentence was +> corrected in `EPIPHANIES.md` (E-SPINE-FOUND-MODERATORS-MISSING-1) and this +> copy was missed — **the same claim living in two places, fixed in one.** + +### 9.1 What is established (the spine) `[G]` + +**Center (place) + ring profile (~12 values) + one wn-1 dipole (2 values) += 90.9–94.3 % of in-disk MSLP variance** (the 14-value constrained model; the +93–97 % once printed here was the 36-parameter per-ring fit — §1) — replicated +across three independent +samples spanning 1980–2021, 41+ storms, four seasons, never shaken once +(N3/N4; §5.11's own subset: median wn1_frac 0.60, R² 0.90). This is a +skeleton that models the **center and the first asymmetry mode of the +dynamics** in **14 logical values** plus a center address — encoded, per +§6.1, in a **12-byte** `6×(8:8)` facet (values and bytes are different +quantities; this line said "~14 bytes" until 2026-08-12) — which is, as the operator notes, +already more explicit structure than a learned model exposes. + +### 9.2 The DRY moderators — measured in this chain, not yet wired `[H]` + +| moderator | measured evidence | wiring | +|---|---|---| +| **Steering level** (baroclinic tilt) | the 92–102° monotone height ladder, zero-crossing 400–650 hPa (§5.2/5.8) | score the dipole against the *steering-level* motion (500–700 hPa flow) instead of the 6h surface displacement — the single most promising fix, **CT-F16** | +| **Displacement magnitude** (label noise) | 6/7 pooled at ≥250 km vs 14/20 unfiltered; CT-F14 0.684 | model the motion-bearing *uncertainty* explicitly instead of a hard cutoff | +| **Surface type / friction** | +14° ocean vs +34° land inflow, paired within one disk (§5.7) | a wind-level correction; second-order on the pressure dipole | +| **Latitude / f, regime** | the low-wn1 July cases; the 75°N outlier | intake covariates, already computed per storm | + +### 9.3 The MOIST sector — not modeled at all (operator, same ruling) `[S]` + +> *"Außerdem haben wir Feuchtigkeit und Abregnen im Aufwind an der Kollision +> zwischen den Gebieten nicht modelliert — das ist eine Art Entropie bei +> Verdunstung und Abregnen."* + +Correct, and the "entropy" word is the *technically* right one, not a +metaphor. Everything in this chain is **dry, adiabatic, balanced dynamics**. +The missing half is diabatic: moisture converges into the collision zone +between air masses (the front), rises, condenses — releasing latent heat +that deepens the low — and **rains out irreversibly**: the water leaves the +column, the heat stays. That one-way flow is moist **entropy production**, +and treating the storm as a heat engine bounded by it is established +literature (Emanuel's potential-intensity Carnot frame; Pauluis' moist +entropy budgets). The state variable is equivalent potential temperature θe; +the sink is precipitation. + +Three things make this *tractable on this substrate, now*, rather than +aspirational: + +1. **The store has the variables** (verified in the `.zmetadata` earlier + this arc): `specific_humidity` (13 levels), `temperature` (13 levels) — + together θe; `total_column_water_vapour`; `total_precipitation_6hr`; + `vertical_velocity` (13 levels — the updraft itself). +2. **θe and TCWV are scalar fields** — the *same* ring/wn-1 decomposition + applies verbatim. The moisture spine costs nothing new. +3. **A diabatic-dominance moderator falls out for free:** the storms where + the dry spine's prediction failed worst (the July cases, wn1_frac + 0.19–0.36) are plausibly the diabatically-driven ones. Precip-per-disk / + TCWV-dipole-strength is a computable gate variable at intake. + +> **⚠ Scope of the moist vocabulary (CodeRabbit, PR #926, 2026-08-12).** +> `θe` is a **moist-adiabatic proxy**, not a complete entropy state for +> variable moisture composition, and precipitation is an **exported water and +> entropy flux**, not the full entropy-production term. Treat θe, TCWV, +> precipitation and vertical velocity here as **proxies**. Before the diabatic +> gate is used as a moderator in CT-M1..M3, the budget it stands for has to be +> written down explicitly — water-vapour, phase-change, latent-heat, +> precipitation and dissipation terms — rather than assumed from θe alone. +> The operator's "eine Art Entropie" framing is directionally right and is +> what makes this worth probing; it is not yet a defined budget. + +Named falsifiers, NOT run, `[S]` until probed: **CT-M1** — the TCWV/θe wn-1 +dipole leads the pressure dipole in bearing (moisture converges *ahead* of +the low, ≈90° from the left-of-motion low pole); **CT-M2** — 6h disk +precipitation is predicted by TCWV × mid-level ascent (`vertical_velocity` +at 700/500 hPa) — the rain-out entropy sink as a budget check; **CT-M3** — +adding the diabatic-dominance gate as a moderator cleans the directional +claim's residual where the displacement filter alone did not. + +### 9.4 The brutal step — learn the moderator matrix on the substrate's own machinery `[S]` + +Operator: *"du könntest sogar brutal sein und domino.rs / LSTM modellieren."* +The shapes already exist and are proven: + +- **The spine is a board state.** Per storm and timestep: ~16 spine values + (center, profile, dipole) + the moderator covariates (steering vector, f, + surface fraction, diabatic gate). A moderator set IS a weight matrix `W`, + and `domino.rs`'s symbiont step has the right SHAPE — `C[16,16] = + A[16,32]·W[32,16]`, a 16-board AMX BF16 tile-GEMM with requantise feedback. + The stencil-as-GEMM path is separately **byte-proven on real WB2 data** in + ndarray's `examples/geostrophic_stencil.rs` (4/4 pre-registered bars, + corr 0.9985). + + > **⚠ Corrected 2026-08-12 (CodeRabbit, PR #926; verified against + > `crates/symbiont/src/domino.rs`).** This bullet said domino.rs "executes + > exactly this". **It does not.** Its `W` is a *fixed* 32×16 BF16 kernel + > whose top 16×16 is a **tridiagonal smoothing kernel** (`domino.rs:113`); + > the sweep overwrites board lanes and updates `Energy`. There are **no + > learned weights, no gate matrices, no hidden state and no cell state**. + > What exists is the tile-GEMM *shape and substrate*, not the model. + > Weather tensor shapes, a training procedure, and the recurrent update + > are all still undefined — that is the work, not a wiring job. +- **The recurrence is an LSTM-shaped problem** `[S]`. Successive 6h spine + states are a short sequence, and the workspace carries byte-parity-proven + int8 LSTM machinery (`tesseract-recognizer`, `E-OCR-LSTM-1`) on the same + `ndarray` tile-GEMM. **That LSTM is an OCR model with its own trained + weights** — it is evidence the *primitives* exist, not that anything is + wired to weather. +- **The hybrid is the honest architecture:** explicit physics as the spine + (this report), learned weights as the moderators — the NeuralGCM-shaped + split, on a 512-byte-per-storm substrate encoding, with the training + discipline this arc has already built (pre-registration, held-out decades, + the audit gate). + +Gate, unchanged: train/test on disjoint decades, pre-registered bars, +adversarial audit (plan §8) before any of it is called more than a probe. diff --git a/probes/weather-p1/comet_tail_f10_f11.json b/probes/weather-p1/comet_tail_f10_f11.json new file mode 100644 index 00000000..a4e1a9bc --- /dev/null +++ b/probes/weather-p1/comet_tail_f10_f11.json @@ -0,0 +1,205 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "sample_start": "1980-02-10T12:00:00", + "stride_days": 411, + "n_candidates": 15, + "rows": [ + { + "date": "1980-02-10T12:00:00", + "t0": 30842, + "center_lat": 46.69223787871181, + "center_lon": 321.36432542323297, + "displacement_km": 135.63575106831914, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.628856457656897, + "R2_profile_wn1": 0.8545934458405746, + "error_deg": -92.97109364728313, + "sign_negative": true + }, + { + "date": "1981-03-27T12:00:00", + "t0": 32486, + "center_lat": 48.92486050236569, + "center_lon": 345.01823101795355, + "displacement_km": 222.0045582791306, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.9045030528074844, + "R2_profile_wn1": 0.9438245000977159, + "error_deg": -20.88097918876568, + "sign_negative": true + }, + { + "date": "1982-05-12T12:00:00", + "t0": 34130, + "center_lat": 47.97766935852807, + "center_lon": 153.37771288174426, + "displacement_km": 131.92171455168986, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5916269721362224, + "R2_profile_wn1": 0.9607952352984042, + "error_deg": -153.31708981303188, + "sign_negative": true + }, + { + "date": "1983-06-27T12:00:00", + "t0": 35774, + "center_lat": 28.9105918421428, + "center_lon": 67.83593520767262, + "displacement_km": 12.99254627435683, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1984-08-11T12:00:00", + "t0": 37418, + "center_lat": 63.0, + "center_lon": 315.9560617272064, + "displacement_km": 118.19750326644628, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8417670116416098, + "R2_profile_wn1": 0.9143783031954809, + "error_deg": -72.21558022018336, + "sign_negative": true + }, + { + "date": "1985-09-26T12:00:00", + "t0": 39062, + "center_lat": 29.997842013566018, + "center_lon": 284.6331381239523, + "displacement_km": 163.15273443482124, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8481833256690069, + "R2_profile_wn1": 0.966594833549946, + "error_deg": 18.124329929345407, + "sign_negative": false + }, + { + "date": "1986-11-11T12:00:00", + "t0": 40706, + "center_lat": 55.380629609588915, + "center_lon": 312.59062947910223, + "displacement_km": 277.5031914159023, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7596036253145539, + "R2_profile_wn1": 0.8539855095982779, + "error_deg": -41.91781393062573, + "sign_negative": true + }, + { + "date": "1987-12-27T12:00:00", + "t0": 42350, + "center_lat": 50.04223886424398, + "center_lon": 329.871606898506, + "displacement_km": 343.30238430474316, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.9222191170016778, + "R2_profile_wn1": 0.9436538419987199, + "error_deg": -17.356647012543647, + "sign_negative": true + }, + { + "date": "1989-02-10T12:00:00", + "t0": 43994, + "center_lat": 55.61303453356034, + "center_lon": 318.7445443332808, + "displacement_km": 438.4521705711391, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8695689494523423, + "R2_profile_wn1": 0.9364385109562684, + "error_deg": -46.04671589769973, + "sign_negative": true + }, + { + "date": "1990-03-28T12:00:00", + "t0": 45638, + "center_lat": 52.53971016691554, + "center_lon": 182.66171472593436, + "displacement_km": 192.03898301009167, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6890623749942085, + "R2_profile_wn1": 0.8866657330465724, + "error_deg": -40.64972313837734, + "sign_negative": true + }, + { + "date": "1991-05-13T12:00:00", + "t0": 47282, + "center_lat": 52.189441129488266, + "center_lon": 250.15073954493647, + "displacement_km": 134.3856569016244, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.13792706578736133, + "R2_profile_wn1": 0.7632855741063911, + "error_deg": 37.23294432626912, + "sign_negative": false + }, + { + "date": "1992-06-27T12:00:00", + "t0": 48926, + "center_lat": 30.962186946701625, + "center_lon": 85.7428667774291, + "displacement_km": 4.259276779660015, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1993-08-12T12:00:00", + "t0": 50570, + "center_lat": 32.66686855435837, + "center_lon": 83.84407774117935, + "displacement_km": 66.4636172410943, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1994-09-27T12:00:00", + "t0": 52214, + "center_lat": 26.30630015082043, + "center_lon": 132.4714451359455, + "displacement_km": 54.04994018738056, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1995-11-12T12:00:00", + "t0": 53858, + "center_lat": 53.73237073051221, + "center_lon": 160.71640607645267, + "displacement_km": 35.12238805382691, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + } + ], + "n_valid": 10, + "n_excluded": 5, + "CT_F13": { + "n": 10, + "sign_neg_frac": 0.8, + "median_abs_error_deg": 41.28376853450153, + "pass": true + }, + "CT_F10": { + "n": 3, + "verdict": "NO-VERDICT-INSUFFICIENT-N" + }, + "CT_F11": { + "n": 9, + "sign_neg_frac": 0.8888888888888888, + "median_abs_error_deg": 41.91781393062573, + "pass": true + }, + "CT_F12": { + "n": 3, + "verdict": "NO-VERDICT-INSUFFICIENT-N" + } +} \ No newline at end of file diff --git a/probes/weather-p1/comet_tail_f10_f11.py b/probes/weather-p1/comet_tail_f10_f11.py new file mode 100644 index 00000000..242847ec --- /dev/null +++ b/probes/weather-p1/comet_tail_f10_f11.py @@ -0,0 +1,311 @@ +"""EXPLORATORY — CT-F10 (displacement-filtered rerun) + CT-F11 (regime- +filtered rerun) + CT-F13 (raw replication check), all pre-registered. +Follow-up to comet_tail_f5_n10.py (ffa2e35d). NOT an EV; bars mine, unaudited. + +CT-N (previous probe) found sign-consistency of the left-of-motion offset +FAILED at n=10 (6/10=0.60) despite structure claims (wn1 dominance, R2) +replicating cleanly. Two post-hoc leads were named but explicitly NOT used +to override that FAIL: (a) restricting to storms with >=250 km/6h +displacement raised consistency to 3/4=0.75; (b) dropping the single most +extreme low-displacement/near-polar outlier raised it to 6/9=0.667. The +correct way to chase a post-hoc lead is a FRESH pre-registered sample with +the filter applied a priori (at least at the reporting stage) -- not a +re-scoring of the sample that already failed. This probe is that fresh +sample. + +SAMPLE INDEPENDENCE (load-bearing): dates are MECHANICALLY generated (fixed +start + fixed stride, no hand-picking) and land entirely in 1980-1997, +non-overlapping with the original 10-storm sample (2015-2021). A human did +not choose dates likely to produce a particular sign-consistency number. + +PRE-REGISTERED (computed on ONE blind draw of candidate dates; all storms +found are reported, none discarded except by the stated CT-E2 trackability +gate that was already standard before any filter is applied): + +CT-F13 RAW REPLICATION (no filter). Does the ~0.60 sign-consistency from + the first n=10 sample reproduce on an independent second blind + sample? No bar (this IS the replication test) -- report sign + fraction and median|error| for direct comparison against the first + sample's 0.60 / 40.2 deg [19.5,63.2]. + +CT-F10 DISPLACEMENT FILTER. Among valid (trackable) storms, restrict to + displacement >= 250 km/6h (the threshold that separated storms 1-2's + own regime in the post-hoc stratification). Bar: sign-consistency + >= 0.70. Minimum n=6 for a real verdict (below that: NO-VERDICT, + stated in advance -- a subset of 3-4 cannot meaningfully test a 0.70 + rate). + +CT-F11 REGIME FILTER. Among valid storms, restrict to wn1_frac >= 0.40 + (CT-E1's own dominance bar -- a storm without a dominant wn-1 mode + does not have a well-defined dipole bearing for the signed test to + apply to). Bar: sign-consistency >= 0.70, same n=6 minimum. + +CT-F12 INTERSECTION (both filters). No bar -- reported for completeness, + likely too small an n for a real verdict, stated as such. + +NOT tested here: this is n=~10-15 candidates yielding filtered subsets of +n=4-10 -- still far short of a properly powered sample for any of these +sub-questions. A PASS on CT-F10/F11 here is evidence the leads are worth a +dedicated large-n follow-up, not proof of a general rule. A FAIL closes the +lead, same as CT-N1 closed the unfiltered claim. +""" +import datetime +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +R_E = 6371.0 +R_DISK = 1200.0 +RING = 100.0 +EPOCH = datetime.datetime(1959, 1, 1) + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=600).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +def t_index(dt): + """WB2 time index for a datetime: 6-hourly steps since 1959-01-01.""" + return int(round((dt - EPOCH).total_seconds() / 3600 / 6)) + + +assert t_index(datetime.datetime(2021, 6, 15, 12)) == 91246, \ + "t-index formula does not reproduce the pinned arc anchor T0=91246" +print("t-index anchor guard: OK") + +_MSLP_SHAPE = meta["mean_sea_level_pressure/.zarray"]["shape"] +_MAX_T = _MSLP_SHAPE[0] - 1 +print(f"store bounds guard: max valid t={_MAX_T}") + +print("fetching static fields (latitude) ...", flush=True) +lat = fetch("latitude", "0").astype(np.float64).ravel() +NY = lat.size +NX = 1440 +phi = np.deg2rad(lat) +lon_deg = np.arange(NX) * 0.25 + +# ---- verbatim helpers (matches comet_tail_f5_n10.py) ----------------------- + + +def geom_ll(latc, lonc): + """dx, dy, r (km) and azimuth theta (rad, CCW from east) relative to a CONTINUOUS (lat, lon) centre.""" + phic = np.deg2rad(latc) + dlon = np.deg2rad((lon_deg[None, :] - lonc + 180) % 360 - 180) + dphi = phi[:, None] - phic + dx = R_E * np.cos(phic) * dlon * np.ones((NY, 1)) + dy = R_E * dphi * np.ones((1, NX)) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def find_center(field, near=None, radius_km=600.0, lat_lo=25.0, lat_hi=75.0): + """Deepest zonal-anomaly low; returns None when the (optionally `near`-limited) mask admits no finite candidate.""" + fa = field - field.mean(axis=1, keepdims=True) + mask = (lat[:, None] > lat_lo) & (lat[:, None] < lat_hi) + if near is not None: + _, _, r, _ = geom_ll(*near) + mask = mask & (r < radius_km) + masked = np.where(mask, fa, np.inf) + ci, cj = np.unravel_index(np.argmin(masked), field.shape) + # An empty mask makes `masked` all-inf and argmin returns index 0, i.e. the + # function would report grid cell (0,0) as a storm centre. A `near`-limited + # search CAN be fully masked, so this must be checked on every path + # (coderabbit on PR #926, 2026-08-11). + if not np.isfinite(masked[ci, cj]): + return None + return int(ci), int(cj) + + +def decompose_ll(field, latc, lonc): + """Ring-mean profile + per-ring wavenumber-1 fit about a continuous centre.""" + _, _, r, th = geom_ll(latc, lonc) + disk = r <= R_DISK + vals, rr, tt = field[disk], r[disk], th[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + prof = np.zeros(nb) + a1 = np.zeros(nb) + b1 = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + v, t = vals[m], tt[m] + prof[b] = v.mean() + a1[b] = 2 * ((v - prof[b]) * np.cos(t)).mean() + b1[b] = 2 * ((v - prof[b]) * np.sin(t)).mean() + resid0 = vals - prof[rings] + wn1 = a1[rings] * np.cos(tt) + b1[rings] * np.sin(tt) + amp = np.hypot(a1, b1) + w = amp * np.arange(nb) + ph = np.arctan2(np.sum(b1 * w), np.sum(a1 * w)) + return {"low_pole_rad": float((ph + np.pi) % (2 * np.pi)), + "wn1_frac": float(wn1.var() / max(resid0.var(), 1e-12)), + "R2_profile_wn1": float(1.0 - np.mean((resid0 - wn1) ** 2) / vals.var())} + + +def subgrid_min(field, ci, cj): + # Longitude WRAPS: a centre at cj == 0 or NX-1 would otherwise slice a 3x2 + # neighbourhood, `A` would have 9 rows against 6 values and lstsq would + # raise. Centres come from a global scan, so the seam at 0 deg is reachable + # (coderabbit on PR #926, 2026-08-11). Rows are clamped, not wrapped — the + # poles are not periodic. + """Sub-grid minimum by 2-D quadratic fit on the 3x3 neighbourhood; longitude wraps, latitude clamps.""" + ri = np.clip(np.array([ci - 1, ci, ci + 1]), 0, field.shape[0] - 1) + z = np.take(field[ri, :], [cj - 1, cj, cj + 1], axis=1, mode="wrap").ravel() + gy, gx = np.meshgrid([-1., 0., 1.], [-1., 0., 1.], indexing="ij") + A = np.column_stack([np.ones(9), gx.ravel(), gy.ravel(), + gx.ravel() ** 2, gy.ravel() ** 2, + (gx * gy).ravel()]) + c = np.linalg.lstsq(A, z, rcond=None)[0] + _, b, cc, d, e, g = c + H = np.array([[2 * d, g], [g, 2 * e]]) + try: + dj, di = np.linalg.solve(H, [-b, -cc]) + except np.linalg.LinAlgError: + dj = di = 0.0 + di, dj = float(np.clip(di, -1, 1)), float(np.clip(dj, -1, 1)) + lat_step = lat[ci + 1] - lat[ci] if ci + 1 < NY else lat[ci] - lat[ci - 1] + return lat[ci] + di * lat_step, lon_deg[cj] + dj * 0.25 + + +def wrap_deg(d): + """Wrap degrees into [-180, 180). + + The half-open end is LOW, not high: at exactly +180 this returns -180. + (Docstring said "(-180, 180]" until 2026-08-11 — CodeRabbit, PR #926 — + which was wrong about the code, not a bug in it.) The boundary is not + cosmetic: `stratum_verdict` scores offset > 0 as low-pole-left-of-motion, + so a value landing exactly on the boundary counts NEGATIVE. That is the + physically right call — +/-180 means the dipole points exactly OPPOSITE + the motion, which is not left-of-motion under either spelling — so the + convention is kept, now stated. Audited across every committed result + JSON: 283 angle-like values, 0 boundary hits, closest 0.91 deg. + """ + return (d + 180.0) % 360.0 - 180.0 + + +def err_deg(low_pole_rad, motion_rad): + """Signed alignment error, in degrees, of a low-pole bearing against the left-of-motion prediction.""" + return float(wrap_deg(np.rad2deg( + low_pole_rad - (motion_rad + np.pi / 2)))) + + +# ---- fresh, mechanically-generated candidate dates ------------------------- +START = datetime.datetime(1980, 2, 10, 12) +STRIDE_DAYS = 411 # non-round, decorrelates from the annual cycle; + # drifts ~1.5 months/sample -> natural season mix +N_CANDIDATES = 15 +DATES = [START + datetime.timedelta(days=STRIDE_DAYS * i) + for i in range(N_CANDIDATES)] +print(f"\ncandidate dates ({N_CANDIDATES}, stride={STRIDE_DAYS}d, " + f"mechanically generated, non-overlapping with the 2015-2021 sample):") +for d in DATES: + print(f" {d.date()}") + +rows = [] +for dt in DATES: + t0 = t_index(dt) + if t0 < 0 or t0 + 1 > _MAX_T: + rows.append({"date": dt.isoformat(), "status": "OUT-OF-STORE-BOUNDS"}) + print(f"\n{dt.date()}: out of store bounds -> excluded") + continue + p0 = fetch("mean_sea_level_pressure", f"{t0}.0.0")[0].astype(np.float64) + p1 = fetch("mean_sea_level_pressure", f"{t0+1}.0.0")[0].astype(np.float64) + + ci0, cj0 = find_center(p0) + la_a, lo_a = subgrid_min(p0, ci0, cj0) + ci1, cj1 = find_center(p1, near=(la_a, lo_a)) + dx, dy, _, _ = geom_ll(la_a, lo_a) + disp = (float(dx[ci1, cj1]), float(dy[ci1, cj1])) + dist = float(np.hypot(*disp)) + mth = float(np.arctan2(disp[1], disp[0])) + trackable = dist >= 100.0 + + row = {"date": dt.isoformat(), "t0": t0, + "center_lat": la_a, "center_lon": lo_a, + "displacement_km": dist, "CT_E2_trackable": bool(trackable)} + if not trackable: + row["status"] = "NOT-TRACKABLE" + rows.append(row) + print(f"\n{dt.date()}: center ({la_a:.2f}N,{lo_a:.2f}E) " + f"disp={dist:.0f}km -> not trackable, excluded") + continue + + d = decompose_ll(p0, la_a, lo_a) + err = err_deg(d["low_pole_rad"], mth) + row.update({"status": "OK", "wn1_frac": d["wn1_frac"], + "R2_profile_wn1": d["R2_profile_wn1"], "error_deg": err, + "sign_negative": bool(err < 0)}) + rows.append(row) + print(f"\n{dt.date()}: center ({la_a:.2f}N,{lo_a:.2f}E), " + f"disp={dist:.0f}km, wn1_frac={d['wn1_frac']:.3f}, " + f"R2={d['R2_profile_wn1']:.3f}, error={err:+.1f} deg " + f"({'neg' if err < 0 else 'pos'})") + +valid = [r for r in rows if r.get("status") == "OK"] +excluded = [r for r in rows if r.get("status") != "OK"] +print(f"\n{len(valid)}/{len(rows)} storms valid; {len(excluded)} excluded: " + + ", ".join(f"{r['date'][:10]}={r['status']}" for r in excluded)) + + +def stratum_verdict(name, subset, min_n=6, bar=0.70): + """Report a stratum. Below `min_n` this emits an explicit NO-VERDICT and + NEVER a `pass` field — an earlier version let CT-F12 (n=3) carry + `pass: true`, which a consumer could lift out of context as a valid + passing result (coderabbit on PR #926, 2026-08-11).""" + if len(subset) < min_n: + print(f"\n{name}: n={len(subset)} < min_n={min_n} -> NO-VERDICT " + "(pre-registered minimum, stated before results)") + return {"n": len(subset), "verdict": "NO-VERDICT-INSUFFICIENT-N"} + errs = np.array([r["error_deg"] for r in subset]) + neg = float((errs < 0).mean()) + med_abs = float(np.median(np.abs(errs))) + passed = neg >= bar + print(f"\n{name}: n={len(subset)}, sign_neg_frac={neg:.2f} " + f"({int((errs<0).sum())}/{len(subset)}), median|error|={med_abs:.1f} " + f"deg -> {'PASS' if passed else 'FAIL'} (bar >= {bar})") + return {"n": len(subset), "sign_neg_frac": neg, + "median_abs_error_deg": med_abs, "pass": bool(passed)} + + +out = {"store": B, "sample_start": START.isoformat(), + "stride_days": STRIDE_DAYS, "n_candidates": N_CANDIDATES, + "rows": rows, "n_valid": len(valid), "n_excluded": len(excluded)} + +print("\n=== CT-F13 raw replication (no filter) ===") +out["CT_F13"] = stratum_verdict( + "CT-F13 (all valid, unfiltered)", valid, min_n=1, bar=0.70) +if valid: + print(" (compare against the first sample: 0.60 sign_neg_frac, " + "median|error|=40.2 deg)") + +print("\n=== CT-F10 displacement filter (>=250 km/6h) ===") +disp_subset = [r for r in valid if r["displacement_km"] >= 250.0] +out["CT_F10"] = stratum_verdict("CT-F10 (disp>=250km)", disp_subset) + +print("\n=== CT-F11 regime filter (wn1_frac >= 0.40) ===") +regime_subset = [r for r in valid if r["wn1_frac"] >= 0.40] +out["CT_F11"] = stratum_verdict("CT-F11 (wn1_frac>=0.40)", regime_subset) + +print("\n=== CT-F12 intersection (both filters, exploratory, no bar) ===") +both_subset = [r for r in valid + if r["displacement_km"] >= 250.0 and r["wn1_frac"] >= 0.40] +# min_n stays at the real evaluable floor: CT-F12 is REPORTED for +# completeness, but it must not be able to emit a pass/fail verdict on n=3. +out["CT_F12"] = stratum_verdict("CT-F12 (both filters)", both_subset) + +with open(pathlib.Path(__file__).with_name("comet_tail_f10_f11.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote comet_tail_f10_f11.json") diff --git a/probes/weather-p1/comet_tail_f14.json b/probes/weather-p1/comet_tail_f14.json new file mode 100644 index 00000000..3b0093f5 --- /dev/null +++ b/probes/weather-p1/comet_tail_f14.json @@ -0,0 +1,1063 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "sample_start": "1996-01-15T12:00:00", + "stride_days": 61, + "n_candidates": 85, + "rows": [ + { + "date": "1996-01-15T12:00:00", + "t0": 54114, + "center_lat": 63.25982565609447, + "center_lon": 319.72181248801405, + "displacement_km": 143.29189743733238, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7034459124320716, + "R2_profile_wn1": 0.7677471033064385, + "error_deg": -25.25344533424891, + "sign_negative": true + }, + { + "date": "1996-03-16T12:00:00", + "t0": 54358, + "center_lat": 37.7606455990929, + "center_lon": 158.35550138007275, + "displacement_km": 394.9627752490917, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.9218634857535203, + "R2_profile_wn1": 0.9805501175374416, + "error_deg": -68.29092173098762, + "sign_negative": true + }, + { + "date": "1996-05-16T12:00:00", + "t0": 54602, + "center_lat": 54.897600066563484, + "center_lon": 214.70798071234748, + "displacement_km": 47.992408305643586, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1996-07-16T12:00:00", + "t0": 54846, + "center_lat": 40.09933139213595, + "center_lon": 90.77955055693623, + "displacement_km": 25.14492839088435, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1996-09-15T12:00:00", + "t0": 55090, + "center_lat": 56.55580974945308, + "center_lon": 326.0894682502776, + "displacement_km": 81.18684323003347, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1996-11-15T12:00:00", + "t0": 55334, + "center_lat": 64.59504763990147, + "center_lon": 168.28153548963402, + "displacement_km": 156.7998111565701, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.524392611674003, + "R2_profile_wn1": 0.8762281376394929, + "error_deg": -32.44412768042454, + "sign_negative": true + }, + { + "date": "1997-01-15T12:00:00", + "t0": 55578, + "center_lat": 49.30630498741868, + "center_lon": 329.96462355455805, + "displacement_km": 276.131907815995, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8194246098886457, + "R2_profile_wn1": 0.9285901826832574, + "error_deg": -37.85248283720318, + "sign_negative": true + }, + { + "date": "1997-03-17T12:00:00", + "t0": 55822, + "center_lat": 47.12116315432716, + "center_lon": 218.75905030400767, + "displacement_km": 328.84589939421153, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8664100164000111, + "R2_profile_wn1": 0.9275739829623769, + "error_deg": 9.241906494550676, + "sign_negative": false + }, + { + "date": "1997-05-17T12:00:00", + "t0": 56066, + "center_lat": 59.43981044064689, + "center_lon": 182.39408913900786, + "displacement_km": 92.30878917916243, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1997-07-17T12:00:00", + "t0": 56310, + "center_lat": 30.492912834704324, + "center_lon": 58.41880044695177, + "displacement_km": 49.26901469421065, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1997-09-16T12:00:00", + "t0": 56554, + "center_lat": 25.0, + "center_lon": 151.11076614387676, + "displacement_km": 164.69097546685467, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.903711657655023, + "R2_profile_wn1": 0.9771025530487909, + "error_deg": -4.117040155186743, + "sign_negative": true + }, + { + "date": "1997-11-16T12:00:00", + "t0": 56798, + "center_lat": 46.403879570799155, + "center_lon": 329.25, + "displacement_km": 213.5359983829942, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7314782724135089, + "R2_profile_wn1": 0.8842778518118193, + "error_deg": 16.941575393219324, + "sign_negative": false + }, + { + "date": "1998-01-16T12:00:00", + "t0": 57042, + "center_lat": 51.77403355932987, + "center_lon": 219.97633719709896, + "displacement_km": 168.02831813326404, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7720076498830907, + "R2_profile_wn1": 0.9164239025030998, + "error_deg": 7.720953773907667, + "sign_negative": false + }, + { + "date": "1998-03-18T12:00:00", + "t0": 57286, + "center_lat": 75.0, + "center_lon": 7.8398700973453215, + "displacement_km": 142.03529201897322, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7428388775809875, + "R2_profile_wn1": 0.9148459502265311, + "error_deg": 75.15217591923067, + "sign_negative": false + }, + { + "date": "1998-05-18T12:00:00", + "t0": 57530, + "center_lat": 58.90729227169889, + "center_lon": 180.080215988875, + "displacement_km": 100.99140102851234, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6842879621543636, + "R2_profile_wn1": 0.927927731596426, + "error_deg": -5.471571443434414, + "sign_negative": true + }, + { + "date": "1998-07-18T12:00:00", + "t0": 57774, + "center_lat": 38.60976995775677, + "center_lon": 149.05974642101765, + "displacement_km": 225.503583772933, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7661945719256544, + "R2_profile_wn1": 0.9120074282710237, + "error_deg": -0.7776508264390145, + "sign_negative": true + }, + { + "date": "1998-09-17T12:00:00", + "t0": 58018, + "center_lat": 56.5, + "center_lon": 331.25, + "displacement_km": 112.24849177871823, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.4888241658845042, + "R2_profile_wn1": 0.9064525601788888, + "error_deg": 57.08830208070586, + "sign_negative": false + }, + { + "date": "1998-11-17T12:00:00", + "t0": 58262, + "center_lat": 52.706425216656854, + "center_lon": 312.369926482629, + "displacement_km": 246.6937755585106, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.789681203573285, + "R2_profile_wn1": 0.9540027306207446, + "error_deg": -56.87084185571463, + "sign_negative": true + }, + { + "date": "1999-01-17T12:00:00", + "t0": 58506, + "center_lat": 63.13698760993007, + "center_lon": 354.61089053542, + "displacement_km": 171.82191999810522, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6519737084497358, + "R2_profile_wn1": 0.8072932867598355, + "error_deg": -25.826249572786423, + "sign_negative": true + }, + { + "date": "1999-03-19T12:00:00", + "t0": 58750, + "center_lat": 52.59343140125187, + "center_lon": 199.20157708107865, + "displacement_km": 145.56357973621667, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8876359605271936, + "R2_profile_wn1": 0.9630136389043775, + "error_deg": -31.452339541822454, + "sign_negative": true + }, + { + "date": "1999-05-19T12:00:00", + "t0": 58994, + "center_lat": 64.7313710897439, + "center_lon": 322.5890456810117, + "displacement_km": 90.72705186603977, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1999-07-19T12:00:00", + "t0": 59238, + "center_lat": 39.075813784861076, + "center_lon": 88.0, + "displacement_km": 28.9976163612832, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "1999-09-18T12:00:00", + "t0": 59482, + "center_lat": 51.364291408264435, + "center_lon": 345.8066333266287, + "displacement_km": 209.71808940426962, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7893569769617479, + "R2_profile_wn1": 0.9648667905738598, + "error_deg": -60.85194816568139, + "sign_negative": true + }, + { + "date": "1999-11-18T12:00:00", + "t0": 59726, + "center_lat": 56.26681556574107, + "center_lon": 178.97969837924182, + "displacement_km": 296.0538640285731, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8448856608604466, + "R2_profile_wn1": 0.9458882563630087, + "error_deg": -32.31823290971391, + "sign_negative": true + }, + { + "date": "2000-01-18T12:00:00", + "t0": 59970, + "center_lat": 53.0, + "center_lon": 33.59135745136407, + "displacement_km": 29.7568891111712, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2000-03-19T12:00:00", + "t0": 60214, + "center_lat": 51.33706443928758, + "center_lon": 183.08273576137316, + "displacement_km": 111.59605343012893, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.4025155903394447, + "R2_profile_wn1": 0.8572637636954391, + "error_deg": -150.90248838654327, + "sign_negative": true + }, + { + "date": "2000-05-19T12:00:00", + "t0": 60458, + "center_lat": 66.72159062092494, + "center_lon": 356.10222762770053, + "displacement_km": 50.53672783717243, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2000-07-19T12:00:00", + "t0": 60702, + "center_lat": 36.5, + "center_lon": 96.0, + "displacement_km": 210.53000954754404, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.33433757006691045, + "R2_profile_wn1": 0.3689236904901597, + "error_deg": -168.97040534886645, + "sign_negative": true + }, + { + "date": "2000-09-18T12:00:00", + "t0": 60946, + "center_lat": 62.25, + "center_lon": 282.43826615607304, + "displacement_km": 99.88316119975518, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2000-11-18T12:00:00", + "t0": 61190, + "center_lat": 59.69969082543526, + "center_lon": 188.8718145690208, + "displacement_km": 63.18353442924144, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2001-01-18T12:00:00", + "t0": 61434, + "center_lat": 53.53239744607066, + "center_lon": 195.8361021708986, + "displacement_km": 24.85645486522088, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2001-03-20T12:00:00", + "t0": 61678, + "center_lat": 48.20585250338296, + "center_lon": 333.6706671498179, + "displacement_km": 120.21625136033988, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6344691497988476, + "R2_profile_wn1": 0.8170266764245833, + "error_deg": 6.813786739258774, + "sign_negative": false + }, + { + "date": "2001-05-20T12:00:00", + "t0": 61922, + "center_lat": 54.798652738391525, + "center_lon": 319.86835786734275, + "displacement_km": 63.95142375810928, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2001-07-20T12:00:00", + "t0": 62166, + "center_lat": 42.70390885646676, + "center_lon": 92.29191516558832, + "displacement_km": 6.164216479185593, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2001-09-19T12:00:00", + "t0": 62410, + "center_lat": 45.102734243947886, + "center_lon": 307.4108806847002, + "displacement_km": 271.6480406924297, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.27514146987703963, + "R2_profile_wn1": 0.9107379114567985, + "error_deg": -135.8866370078875, + "sign_negative": true + }, + { + "date": "2001-11-19T12:00:00", + "t0": 62654, + "center_lat": 52.366450881222335, + "center_lon": 192.61053674574896, + "displacement_km": 126.9860733478576, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5020996874699949, + "R2_profile_wn1": 0.823804188316402, + "error_deg": 31.25054704858303, + "sign_negative": false + }, + { + "date": "2002-01-19T12:00:00", + "t0": 62898, + "center_lat": 52.23181291184118, + "center_lon": 328.6795165909927, + "displacement_km": 322.77297720503896, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8529649040849745, + "R2_profile_wn1": 0.8657847711935247, + "error_deg": -9.43809335671719, + "sign_negative": true + }, + { + "date": "2002-03-21T12:00:00", + "t0": 63142, + "center_lat": 38.751726980533526, + "center_lon": 323.6873562719305, + "displacement_km": 274.6911720596344, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5971704437111163, + "R2_profile_wn1": 0.9280315149581825, + "error_deg": -98.33221272712603, + "sign_negative": true + }, + { + "date": "2002-05-21T12:00:00", + "t0": 63386, + "center_lat": 52.56102565883963, + "center_lon": 343.79494038244593, + "displacement_km": 90.23309010100216, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2002-07-21T12:00:00", + "t0": 63630, + "center_lat": 28.845082353475743, + "center_lon": 67.81185755361025, + "displacement_km": 457.5499062282751, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.19397289953637967, + "R2_profile_wn1": 0.49205991444846264, + "error_deg": 49.13517569045763, + "sign_negative": false + }, + { + "date": "2002-09-20T12:00:00", + "t0": 63874, + "center_lat": 63.97971778034947, + "center_lon": 274.4204219297805, + "displacement_km": 208.68483995937757, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8965280593937306, + "R2_profile_wn1": 0.9260846119561289, + "error_deg": 42.386516847351004, + "sign_negative": false + }, + { + "date": "2002-11-20T12:00:00", + "t0": 64118, + "center_lat": 54.22082766713417, + "center_lon": 338.8655320094911, + "displacement_km": 166.74930895251876, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5676562389304307, + "R2_profile_wn1": 0.8708566720460047, + "error_deg": -22.14666309240542, + "sign_negative": true + }, + { + "date": "2003-01-20T12:00:00", + "t0": 64362, + "center_lat": 44.41552217189388, + "center_lon": 196.57743293394398, + "displacement_km": 100.28258487180135, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.12445026739016603, + "R2_profile_wn1": 0.7514059610710206, + "error_deg": 21.234564779699724, + "sign_negative": false + }, + { + "date": "2003-03-22T12:00:00", + "t0": 64606, + "center_lat": 75.0, + "center_lon": 43.059245403036854, + "displacement_km": 230.21440495669677, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.2336882653407327, + "R2_profile_wn1": 0.7836004784520969, + "error_deg": -46.59880232445519, + "sign_negative": true + }, + { + "date": "2003-05-22T12:00:00", + "t0": 64850, + "center_lat": 45.32920972627663, + "center_lon": 204.5, + "displacement_km": 200.95707646260365, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6860843730000245, + "R2_profile_wn1": 0.9044345651527659, + "error_deg": -49.470432741386674, + "sign_negative": true + }, + { + "date": "2003-07-22T12:00:00", + "t0": 65094, + "center_lat": 40.20503910539359, + "center_lon": 79.34242857084944, + "displacement_km": 591.8334818738821, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.35653286267264633, + "R2_profile_wn1": 0.40891542105004974, + "error_deg": -87.62762374253634, + "sign_negative": true + }, + { + "date": "2003-09-21T12:00:00", + "t0": 65338, + "center_lat": 32.102299615284466, + "center_lon": 138.94589585584745, + "displacement_km": 197.0830105460272, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.4313577275275672, + "R2_profile_wn1": 0.8619465012375966, + "error_deg": 154.33329538229998, + "sign_negative": false + }, + { + "date": "2003-11-21T12:00:00", + "t0": 65582, + "center_lat": 63.98450992701462, + "center_lon": 184.61926270085146, + "displacement_km": 128.3169992927387, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.4798554219333236, + "R2_profile_wn1": 0.7288681498577506, + "error_deg": -67.99793507448506, + "sign_negative": true + }, + { + "date": "2004-01-21T12:00:00", + "t0": 65826, + "center_lat": 73.02737088577726, + "center_lon": 311.4440303627265, + "displacement_km": 137.14269777852712, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.24568604639352612, + "R2_profile_wn1": 0.43497910980432, + "error_deg": 128.43408102089933, + "sign_negative": false + }, + { + "date": "2004-03-22T12:00:00", + "t0": 66070, + "center_lat": 45.74990265105207, + "center_lon": 210.74001005567348, + "displacement_km": 147.44726211090074, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8748561883554901, + "R2_profile_wn1": 0.9867541892996654, + "error_deg": -47.95993792791921, + "sign_negative": true + }, + { + "date": "2004-05-22T12:00:00", + "t0": 66314, + "center_lat": 43.474522622958894, + "center_lon": 187.67494728033174, + "displacement_km": 182.22240894076796, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.610251750105987, + "R2_profile_wn1": 0.8874600138334985, + "error_deg": -38.0203915736991, + "sign_negative": true + }, + { + "date": "2004-07-22T12:00:00", + "t0": 66558, + "center_lat": 54.20799369844886, + "center_lon": 130.25559414294094, + "displacement_km": 429.6171401779512, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.3261649646401837, + "R2_profile_wn1": 0.6482738493549359, + "error_deg": 126.4843089002764, + "sign_negative": false + }, + { + "date": "2004-09-21T12:00:00", + "t0": 66802, + "center_lat": 46.2872751765851, + "center_lon": 308.1552098771979, + "displacement_km": 137.42683531081013, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6317870111147476, + "R2_profile_wn1": 0.8813524053943474, + "error_deg": -160.78766673599588, + "sign_negative": true + }, + { + "date": "2004-11-21T12:00:00", + "t0": 67046, + "center_lat": 54.75, + "center_lon": 188.15043347333307, + "displacement_km": 306.60785355366534, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.9412133676528681, + "R2_profile_wn1": 0.9621657429278813, + "error_deg": -13.583816972829794, + "sign_negative": true + }, + { + "date": "2005-01-21T12:00:00", + "t0": 67290, + "center_lat": 51.724428116105315, + "center_lon": 211.96293583525767, + "displacement_km": 99.24068356815262, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2005-03-23T12:00:00", + "t0": 67534, + "center_lat": 60.94934701296311, + "center_lon": 338.84924928936954, + "displacement_km": 175.7455567964236, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6767214140445031, + "R2_profile_wn1": 0.766855397070139, + "error_deg": 3.347586660031766, + "sign_negative": false + }, + { + "date": "2005-05-23T12:00:00", + "t0": 67778, + "center_lat": 49.66383829643146, + "center_lon": 338.7627794330812, + "displacement_km": 307.28577630573585, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.3940654631918796, + "R2_profile_wn1": 0.6587755897671268, + "error_deg": -0.6118558893321051, + "sign_negative": true + }, + { + "date": "2005-07-23T12:00:00", + "t0": 68022, + "center_lat": 58.859046412630214, + "center_lon": 18.04617388881205, + "displacement_km": 127.30741310998, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6332482378766127, + "R2_profile_wn1": 0.8928203496295047, + "error_deg": -30.72669591825604, + "sign_negative": true + }, + { + "date": "2005-09-22T12:00:00", + "t0": 68266, + "center_lat": 25.2445767446035, + "center_lon": 271.62085553828723, + "displacement_km": 68.59851541724969, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2005-11-22T12:00:00", + "t0": 68510, + "center_lat": 46.252952173269, + "center_lon": 208.49998623458347, + "displacement_km": 225.92029658999758, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.27525031465262145, + "R2_profile_wn1": 0.7784413891426507, + "error_deg": 33.1314799989201, + "sign_negative": false + }, + { + "date": "2006-01-22T12:00:00", + "t0": 68754, + "center_lat": 41.63454690695439, + "center_lon": 158.0240357377511, + "displacement_km": 291.43078891941735, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5287657645924341, + "R2_profile_wn1": 0.8637769409048849, + "error_deg": 46.520425963822305, + "sign_negative": false + }, + { + "date": "2006-03-24T12:00:00", + "t0": 68998, + "center_lat": 40.081768429120515, + "center_lon": 316.97489409322253, + "displacement_km": 282.49422464834026, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7667653963904881, + "R2_profile_wn1": 0.9007108619579532, + "error_deg": -58.900962386334385, + "sign_negative": true + }, + { + "date": "2006-05-24T12:00:00", + "t0": 69242, + "center_lat": 31.602601848391366, + "center_lon": 70.75743365860119, + "displacement_km": 67.00996656060151, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2006-07-24T12:00:00", + "t0": 69486, + "center_lat": 31.494278222686688, + "center_lon": 62.41519015593225, + "displacement_km": 42.617992011605786, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2006-09-23T12:00:00", + "t0": 69730, + "center_lat": 36.65253482017127, + "center_lon": 312.4582302166105, + "displacement_km": 205.07582960076385, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5469621620984172, + "R2_profile_wn1": 0.9003659471137507, + "error_deg": -57.28938786863003, + "sign_negative": true + }, + { + "date": "2006-11-23T12:00:00", + "t0": 69974, + "center_lat": 55.741771104681035, + "center_lon": 352.21398518042116, + "displacement_km": 183.31670437410514, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.46915397428799316, + "R2_profile_wn1": 0.7046669408813198, + "error_deg": 19.881121208813937, + "sign_negative": false + }, + { + "date": "2007-01-23T12:00:00", + "t0": 70218, + "center_lat": 39.33113364502934, + "center_lon": 174.7526838816152, + "displacement_km": 317.67915242399596, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6656267643751554, + "R2_profile_wn1": 0.9289146945425433, + "error_deg": 108.08865494211045, + "sign_negative": false + }, + { + "date": "2007-03-25T12:00:00", + "t0": 70462, + "center_lat": 66.44401876532287, + "center_lon": 266.0184476446863, + "displacement_km": 126.06417229717535, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.49228042620710927, + "R2_profile_wn1": 0.7989870585344538, + "error_deg": 135.98711478712386, + "sign_negative": false + }, + { + "date": "2007-05-25T12:00:00", + "t0": 70706, + "center_lat": 41.990250690400565, + "center_lon": 134.35349734825567, + "displacement_km": 244.24100553586638, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.613718755258605, + "R2_profile_wn1": 0.7516332404696452, + "error_deg": 53.71512031682133, + "sign_negative": false + }, + { + "date": "2007-07-25T12:00:00", + "t0": 70950, + "center_lat": 36.76618471235129, + "center_lon": 96.17249268645661, + "displacement_km": 37.677201014839085, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2007-09-24T12:00:00", + "t0": 71194, + "center_lat": 59.69631488814643, + "center_lon": 183.509118148923, + "displacement_km": 105.24774845288307, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5871365986298074, + "R2_profile_wn1": 0.8849299246694261, + "error_deg": -2.212465859228871, + "sign_negative": true + }, + { + "date": "2007-11-24T12:00:00", + "t0": 71438, + "center_lat": 55.229505601945235, + "center_lon": 169.92053231351656, + "displacement_km": 74.2624359832062, + "CT_E2_trackable": false, + "status": "NOT-TRACKABLE" + }, + { + "date": "2008-01-24T12:00:00", + "t0": 71682, + "center_lat": 56.127472346306945, + "center_lon": 173.16491994869907, + "displacement_km": 127.51075300415441, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.28851992440842855, + "R2_profile_wn1": 0.8221809306100194, + "error_deg": 158.2208369158227, + "sign_negative": false + }, + { + "date": "2008-03-25T12:00:00", + "t0": 71926, + "center_lat": 52.95302344219514, + "center_lon": 30.90502116898233, + "displacement_km": 144.35750607678403, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5384726774578575, + "R2_profile_wn1": 0.8935969065345197, + "error_deg": 57.47145916488944, + "sign_negative": false + }, + { + "date": "2008-05-25T12:00:00", + "t0": 72170, + "center_lat": 57.097293225844446, + "center_lon": 308.38436394044123, + "displacement_km": 164.86759953878362, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7290751808591619, + "R2_profile_wn1": 0.8989490296494737, + "error_deg": -48.04552730472784, + "sign_negative": true + }, + { + "date": "2008-07-25T12:00:00", + "t0": 72414, + "center_lat": 31.900038974581634, + "center_lon": 63.05409675230088, + "displacement_km": 114.12091612889061, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6141772212888934, + "R2_profile_wn1": 0.642327611825033, + "error_deg": -156.31609111456203, + "sign_negative": true + }, + { + "date": "2008-09-24T12:00:00", + "t0": 72658, + "center_lat": 50.29407906924802, + "center_lon": 161.66609864388215, + "displacement_km": 134.7592934657519, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6842262474882242, + "R2_profile_wn1": 0.8536437505832669, + "error_deg": -24.841364650294935, + "sign_negative": true + }, + { + "date": "2008-11-24T12:00:00", + "t0": 72902, + "center_lat": 55.63059655563931, + "center_lon": 200.15611004795264, + "displacement_km": 213.88546971278637, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6902624708340415, + "R2_profile_wn1": 0.8551239580601979, + "error_deg": -18.96265701386858, + "sign_negative": true + }, + { + "date": "2009-01-24T12:00:00", + "t0": 73146, + "center_lat": 63.76633709633874, + "center_lon": 338.9549162858299, + "displacement_km": 243.54993083197846, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.7984554191055045, + "R2_profile_wn1": 0.8724220061337502, + "error_deg": 9.025548666417762, + "sign_negative": false + }, + { + "date": "2009-03-26T12:00:00", + "t0": 73390, + "center_lat": 59.458189862483295, + "center_lon": 354.749746897519, + "displacement_km": 141.35475016145975, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.6769636234075413, + "R2_profile_wn1": 0.9143044100189791, + "error_deg": -49.68728539987143, + "sign_negative": true + }, + { + "date": "2009-05-26T12:00:00", + "t0": 73634, + "center_lat": 32.081890067957204, + "center_lon": 85.29824266895127, + "displacement_km": 585.0246566127836, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.4499584064719253, + "R2_profile_wn1": 0.4952763158604363, + "error_deg": -53.08536899757155, + "sign_negative": true + }, + { + "date": "2009-07-26T12:00:00", + "t0": 73878, + "center_lat": 34.14437390359832, + "center_lon": 71.81849475304371, + "displacement_km": 363.2025023438297, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.4313284079650069, + "R2_profile_wn1": 0.44840644133553687, + "error_deg": -93.58088940292669, + "sign_negative": true + }, + { + "date": "2009-09-25T12:00:00", + "t0": 74122, + "center_lat": 69.01585282912403, + "center_lon": 353.4343585345505, + "displacement_km": 562.8503596905939, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.8679881135275417, + "R2_profile_wn1": 0.9189089794677993, + "error_deg": 0.13717943106883013, + "sign_negative": false + }, + { + "date": "2009-11-25T12:00:00", + "t0": 74366, + "center_lat": 59.52619677699913, + "center_lon": 352.03934730709716, + "displacement_km": 270.81659311746546, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.5955321552576727, + "R2_profile_wn1": 0.8724852150614523, + "error_deg": -30.594845686320213, + "sign_negative": true + }, + { + "date": "2010-01-25T12:00:00", + "t0": 74610, + "center_lat": 66.49068385848419, + "center_lon": 328.610334955992, + "displacement_km": 141.0921969569801, + "CT_E2_trackable": true, + "status": "OK", + "wn1_frac": 0.3735504876612974, + "R2_profile_wn1": 0.8651860376528754, + "error_deg": 3.7479716406901105, + "sign_negative": false + } + ], + "n_valid": 64, + "n_qualifying": 19, + "n_excluded": 21, + "exclusion_breakdown": { + "NOT-TRACKABLE": 21 + }, + "CT_F14": { + "n": 19, + "verdict": "NO-VERDICT-INSUFFICIENT-N" + }, + "CT_F14_combined_3sample": { + "n": 26, + "n_negative": 19, + "sign_neg_frac": 0.7307692307692307, + "one_sided_p": 0.014479637145996094, + "applied_verdict": "NOT PROMOTED -- the largest component (CT-F14, n=19) failed its own pre-registered n>=20 floor. Pooling cannot rescue a component that did not qualify, and the pre-registration had no contingency for this case. Directional claim remains SUGGESTIVE (report SS5.11).", + "interpretation_preregistered_rule": "ESTABLISHED at this n, displacement-filtered regime -> ready for audit-gate queue as [H]-graded, scope-limited claim", + "subset_below_min_n": true, + "largest_component_min_n": 20, + "components": { + "sample1_disp250": { + "n": 4, + "neg": 3 + }, + "sample2_disp250": { + "n": 3, + "neg": 3 + }, + "CT_F14": { + "n": 19, + "neg": 13 + } + } + } +} \ No newline at end of file diff --git a/probes/weather-p1/comet_tail_f14.py b/probes/weather-p1/comet_tail_f14.py new file mode 100644 index 00000000..1269b088 --- /dev/null +++ b/probes/weather-p1/comet_tail_f14.py @@ -0,0 +1,376 @@ +"""EXPLORATORY — CT-F14, the correctly-scoped next step named in §5.10: a +SINGLE, properly powered, displacement-filtered-ONLY pre-registered sample. +Follow-up to comet_tail_f10_f11.py (e24ecf3d). NOT an EV; bars mine, +unaudited. + +WHY THIS IS THE RIGHT TEST NOW (not a third exploratory rerun). Two prior +independent 10-storm samples gave a combined 14/20 unfiltered (one-sided +p~=0.058, noise floor) but a combined 6/7 among storms with >=250 km/6h +displacement (one-sided p~=0.0625) -- suggestive but n=7 is too small to +call, and that pooling was NOT itself pre-registered before either sample +ran. The regime-contamination explanation was checked directly against +sample 1's own data and did NOT survive (dropped sample 1 to exactly +chance). The apparatus explanation (motion-bearing noise on slow-moving +storms) is the one with surviving support. CT-F14 is the single test that +moves that number from "suggestive, n=7" to a real verdict: ONE +pre-registered displacement-filtered-only sample, sized for n>=20-30 +qualifying storms. + +SAMPLE INDEPENDENCE (load-bearing, third time in this chain): dates are +MECHANICALLY generated (fixed start + fixed stride, chosen before this file +was written and never adjusted after seeing output) and land in 1996-2014 -- +zero overlap with sample 1 (2015-2021, hand-picked) or sample 2 +(1980-1995, stride=411d from 1980-02-10). Expected qualifying rate is +estimated from the TWO PRIOR samples' combined empirical rate (7 qualifying +storms out of 25 total candidates tried = 0.28/candidate) to size N_CANDIDATES +for the sample -- an estimate, not a guarantee; actual attrition is reported +however it lands, never resampled to hit a target. + +PRE-REGISTERED, single test: + +CT-F14 Among valid (CT-E2 trackable) storms in this fresh sample with + displacement >= 250 km/6h, sign-consistency (fraction with error<0, + the same direction as storms 1-2 and both prior samples' filtered + subsets) >= 0.70. Minimum n=20 for a real verdict (below that: + NO-VERDICT, stated in advance -- consistent with the n=6 floor used + for smaller strata in the prior probe, scaled up for a properly + powered single test). + +REPORTED ALONGSIDE (not additional bars, just always-computed descriptive +stats + the decision rule pre-committed for how this updates the arc's +overall verdict): + + - exact one-sided binomial p-value (H0: p=0.5) for CT-F14's own subset + - THE COMBINED three-independent-sample figure: CT-F14's qualifying + storms + the two prior samples' displacement>=250 subsets (n=4+3=7), + with its own one-sided binomial p-value + - median wn1_frac / R2_profile_wn1 among the qualifying subset, for + comparison against N3/N4's already-established generalization + - PRE-COMMITTED interpretation of the combined 3-sample p-value (decided + now, before running, so the read cannot be tuned to the result): + p < 0.05 -> established at this n, displacement-filtered + regime; ready for the CT-F10..F15 chain to close + and the finding to enter the audit-gate queue + as an [H]-graded, scope-limited claim + 0.05 <= p < 0.10 -> still suggestive; needs a further doubling of n + before promotion, not yet audit-gate-ready + p >= 0.10 -> NOT established; the apparatus explanation is + itself now in question and the offset's + directional claim should be retired to [S] + pending a fundamentally different design (e.g. + modeling motion-bearing uncertainty explicitly + rather than a hard displacement cutoff) + +NOT tested here: this remains a MSLP-only, NH-only, one-season-mix test; no +claim about SH, no claim about tropical cyclones, no claim beyond the +extratropical-low steering-flow framing §2 laid out from the start. +""" +import datetime +import json +import pathlib +import urllib.request +from math import comb + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +R_E = 6371.0 +R_DISK = 1200.0 +RING = 100.0 +EPOCH = datetime.datetime(1959, 1, 1) + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=600).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +def t_index(dt): + """WB2 time index for a datetime: 6-hourly steps since 1959-01-01.""" + return int(round((dt - EPOCH).total_seconds() / 3600 / 6)) + + +assert t_index(datetime.datetime(2021, 6, 15, 12)) == 91246, \ + "t-index formula does not reproduce the pinned arc anchor T0=91246" +print("t-index anchor guard: OK", flush=True) + +_MSLP_SHAPE = meta["mean_sea_level_pressure/.zarray"]["shape"] +_MAX_T = _MSLP_SHAPE[0] - 1 +print(f"store bounds guard: max valid t={_MAX_T}", flush=True) + +print("fetching static fields (latitude) ...", flush=True) +lat = fetch("latitude", "0").astype(np.float64).ravel() +NY = lat.size +NX = 1440 +phi = np.deg2rad(lat) +lon_deg = np.arange(NX) * 0.25 + +# ---- verbatim helpers (matches comet_tail_f10_f11.py / f5_n10.py) --------- + + +def geom_ll(latc, lonc): + """dx, dy, r (km) and azimuth theta (rad, CCW from east) relative to a CONTINUOUS (lat, lon) centre.""" + phic = np.deg2rad(latc) + dlon = np.deg2rad((lon_deg[None, :] - lonc + 180) % 360 - 180) + dphi = phi[:, None] - phic + dx = R_E * np.cos(phic) * dlon * np.ones((NY, 1)) + dy = R_E * dphi * np.ones((1, NX)) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def find_center(field, near=None, radius_km=600.0, lat_lo=25.0, lat_hi=75.0): + """Deepest zonal-anomaly low; returns None when the (optionally `near`-limited) mask admits no finite candidate.""" + fa = field - field.mean(axis=1, keepdims=True) + mask = (lat[:, None] > lat_lo) & (lat[:, None] < lat_hi) + if near is not None: + _, _, r, _ = geom_ll(*near) + mask = mask & (r < radius_km) + masked = np.where(mask, fa, np.inf) + ci, cj = np.unravel_index(np.argmin(masked), field.shape) + # An empty mask makes `masked` all-inf and argmin returns index 0, i.e. the + # function would report grid cell (0,0) as a storm centre. A `near`-limited + # search CAN be fully masked, so this must be checked on every path + # (coderabbit on PR #926, 2026-08-11). + if not np.isfinite(masked[ci, cj]): + return None + return int(ci), int(cj) + + +def decompose_ll(field, latc, lonc): + """Ring-mean profile + per-ring wavenumber-1 fit about a continuous centre.""" + _, _, r, th = geom_ll(latc, lonc) + disk = r <= R_DISK + vals, rr, tt = field[disk], r[disk], th[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + prof = np.zeros(nb) + a1 = np.zeros(nb) + b1 = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + v, t = vals[m], tt[m] + prof[b] = v.mean() + a1[b] = 2 * ((v - prof[b]) * np.cos(t)).mean() + b1[b] = 2 * ((v - prof[b]) * np.sin(t)).mean() + resid0 = vals - prof[rings] + wn1 = a1[rings] * np.cos(tt) + b1[rings] * np.sin(tt) + amp = np.hypot(a1, b1) + w = amp * np.arange(nb) + ph = np.arctan2(np.sum(b1 * w), np.sum(a1 * w)) + return {"low_pole_rad": float((ph + np.pi) % (2 * np.pi)), + "wn1_frac": float(wn1.var() / max(resid0.var(), 1e-12)), + "R2_profile_wn1": float(1.0 - np.mean((resid0 - wn1) ** 2) / vals.var())} + + +def subgrid_min(field, ci, cj): + # Longitude WRAPS: a centre at cj == 0 or NX-1 would otherwise slice a 3x2 + # neighbourhood, `A` would have 9 rows against 6 values and lstsq would + # raise. Centres come from a global scan, so the seam at 0 deg is reachable + # (coderabbit on PR #926, 2026-08-11). Rows are clamped, not wrapped — the + # poles are not periodic. + """Sub-grid minimum by 2-D quadratic fit on the 3x3 neighbourhood; longitude wraps, latitude clamps.""" + ri = np.clip(np.array([ci - 1, ci, ci + 1]), 0, field.shape[0] - 1) + z = np.take(field[ri, :], [cj - 1, cj, cj + 1], axis=1, mode="wrap").ravel() + gy, gx = np.meshgrid([-1., 0., 1.], [-1., 0., 1.], indexing="ij") + A = np.column_stack([np.ones(9), gx.ravel(), gy.ravel(), + gx.ravel() ** 2, gy.ravel() ** 2, + (gx * gy).ravel()]) + c = np.linalg.lstsq(A, z, rcond=None)[0] + _, b, cc, d, e, g = c + H = np.array([[2 * d, g], [g, 2 * e]]) + try: + dj, di = np.linalg.solve(H, [-b, -cc]) + except np.linalg.LinAlgError: + dj = di = 0.0 + di, dj = float(np.clip(di, -1, 1)), float(np.clip(dj, -1, 1)) + lat_step = lat[ci + 1] - lat[ci] if ci + 1 < NY else lat[ci] - lat[ci - 1] + return lat[ci] + di * lat_step, lon_deg[cj] + dj * 0.25 + + +def wrap_deg(d): + """Wrap degrees into (-180, 180].""" + return (d + 180.0) % 360.0 - 180.0 + + +def err_deg(low_pole_rad, motion_rad): + """Signed alignment error, in degrees, of a low-pole bearing against the left-of-motion prediction.""" + return float(wrap_deg(np.rad2deg( + low_pole_rad - (motion_rad + np.pi / 2)))) + + +def binom_sf_ge(k, n, p=0.5): + """Exact one-sided binomial tail P(X >= k) for n trials at probability p.""" + return sum(comb(n, i) * (p ** i) * ((1 - p) ** (n - i)) + for i in range(k, n + 1)) + + +# ---- fresh, mechanically-generated candidate dates ------------------------- +START = datetime.datetime(1996, 1, 15, 12) +STRIDE_DAYS = 61 # arbitrary, fixed before writing this loop, + # never adjusted after seeing output +N_CANDIDATES = 85 # sized from the two prior samples' combined + # empirical rate (7/25 candidates ~= 0.28) to + # target n>=20-30 qualifying storms +DATES = [START + datetime.timedelta(days=STRIDE_DAYS * i) + for i in range(N_CANDIDATES)] +print(f"\nCT-F14: {N_CANDIDATES} candidate dates, stride={STRIDE_DAYS}d, " + f"{DATES[0].date()} .. {DATES[-1].date()} " + "(1996-2014, zero overlap with prior samples)", flush=True) + +rows = [] +for idx, dt in enumerate(DATES): + t0 = t_index(dt) + if t0 < 0 or t0 + 1 > _MAX_T: + rows.append({"date": dt.isoformat(), "status": "OUT-OF-STORE-BOUNDS"}) + continue + try: + p0 = fetch("mean_sea_level_pressure", f"{t0}.0.0")[0].astype(np.float64) + p1 = fetch("mean_sea_level_pressure", f"{t0+1}.0.0")[0].astype(np.float64) + except Exception as e: + rows.append({"date": dt.isoformat(), "status": f"FETCH-ERROR: {e}"}) + print(f"[{idx+1}/{N_CANDIDATES}] {dt.date()}: fetch error {e}", + flush=True) + continue + + ci0, cj0 = find_center(p0) + la_a, lo_a = subgrid_min(p0, ci0, cj0) + ci1, cj1 = find_center(p1, near=(la_a, lo_a)) + dx, dy, _, _ = geom_ll(la_a, lo_a) + disp = (float(dx[ci1, cj1]), float(dy[ci1, cj1])) + dist = float(np.hypot(*disp)) + mth = float(np.arctan2(disp[1], disp[0])) + trackable = dist >= 100.0 + + row = {"date": dt.isoformat(), "t0": t0, + "center_lat": la_a, "center_lon": lo_a, + "displacement_km": dist, "CT_E2_trackable": bool(trackable)} + if not trackable: + row["status"] = "NOT-TRACKABLE" + rows.append(row) + if (idx + 1) % 10 == 0 or idx == 0: + print(f"[{idx+1}/{N_CANDIDATES}] {dt.date()}: " + f"disp={dist:.0f}km not trackable", flush=True) + continue + + d = decompose_ll(p0, la_a, lo_a) + err = err_deg(d["low_pole_rad"], mth) + row.update({"status": "OK", "wn1_frac": d["wn1_frac"], + "R2_profile_wn1": d["R2_profile_wn1"], "error_deg": err, + "sign_negative": bool(err < 0)}) + rows.append(row) + tag = " ***DISP>=250***" if dist >= 250 else "" + print(f"[{idx+1}/{N_CANDIDATES}] {dt.date()}: disp={dist:.0f}km " + f"err={err:+.1f}deg wn1={d['wn1_frac']:.2f}{tag}", flush=True) + +valid = [r for r in rows if r.get("status") == "OK"] +excluded = [r for r in rows if r.get("status") != "OK"] +qualifying = [r for r in valid if r["displacement_km"] >= 250.0] + +print(f"\n{len(valid)}/{len(rows)} valid (trackable); " + f"{len(qualifying)} qualify (disp>=250km); " + f"{len(excluded)} excluded", flush=True) +exclusion_counts = {} +for r in excluded: + exclusion_counts[r["status"]] = exclusion_counts.get(r["status"], 0) + 1 +print("exclusion breakdown:", exclusion_counts, flush=True) + +out = {"store": B, "sample_start": START.isoformat(), + "stride_days": STRIDE_DAYS, "n_candidates": N_CANDIDATES, + "rows": rows, "n_valid": len(valid), "n_qualifying": len(qualifying), + "n_excluded": len(excluded), "exclusion_breakdown": exclusion_counts} + +print("\n=== CT-F14 verdict ===", flush=True) +MIN_N = 20 +if len(qualifying) < MIN_N: + print(f"n={len(qualifying)} < MIN_N={MIN_N} -> NO-VERDICT " + "(pre-registered minimum, stated before results)") + out["CT_F14"] = {"n": len(qualifying), "verdict": "NO-VERDICT-INSUFFICIENT-N"} +else: + errs = np.array([r["error_deg"] for r in qualifying]) + neg = int((errs < 0).sum()) + frac = neg / len(qualifying) + p_own = binom_sf_ge(neg, len(qualifying)) + passed = frac >= 0.70 + med_wn1 = float(np.median([r["wn1_frac"] for r in qualifying])) + med_r2 = float(np.median([r["R2_profile_wn1"] for r in qualifying])) + print(f"CT-F14: n={len(qualifying)}, neg={neg}/{len(qualifying)} = " + f"{frac:.3f}, one-sided p={p_own:.4f} -> " + f"{'PASS' if passed else 'FAIL'} (bar >= 0.70)") + print(f" median wn1_frac={med_wn1:.3f}, median R2={med_r2:.3f} " + "(compare against N3/N4's 0.723/0.900)") + out["CT_F14"] = {"n": len(qualifying), "n_negative": neg, + "sign_neg_frac": frac, "one_sided_p": p_own, + "pass": bool(passed), "median_wn1_frac": med_wn1, + "median_R2_profile_wn1": med_r2} + +# combined with the two prior independent samples' disp>=250 subsets +PRIOR_S1 = {"n": 4, "neg": 3} # comet_tail_f5_n10.json CT_N disp>=250 +PRIOR_S2 = {"n": 3, "neg": 3} # comet_tail_f10_f11.json CT_F10 disp>=250 +n_qual = len(qualifying) +neg_qual = int(sum(1 for r in qualifying if r["error_deg"] < 0)) if n_qual else 0 +n_combined = PRIOR_S1["n"] + PRIOR_S2["n"] + n_qual +neg_combined = PRIOR_S1["neg"] + PRIOR_S2["neg"] + neg_qual +frac_combined = neg_combined / n_combined if n_combined else float("nan") +p_combined = binom_sf_ge(neg_combined, n_combined) if n_combined else float("nan") + +if p_combined < 0.05: + interp = "ESTABLISHED at this n, displacement-filtered regime -> ready for audit-gate queue as [H]-graded, scope-limited claim" +elif p_combined < 0.10: + interp = "SUGGESTIVE, needs further n before promotion, not audit-gate-ready" +else: + interp = "NOT ESTABLISHED -> apparatus explanation itself in question; retire directional claim to [S] pending a fundamentally different design" + +# THE POOLING RULE HAD A GAP, and the artifact must not hide it. +# The pre-registered rule above reads ONLY p_combined, so when CT-F14 -- the +# largest and most carefully powered component -- fails its OWN n>=20 floor, +# this block still emitted "ESTABLISHED ... ready for audit-gate queue" +# unconditionally. A result that failed its own gate then carried a promotion +# recommendation in the MACHINE-READABLE artifact, contradicting the report's +# own section 5.11 conclusion ("graded down to still suggestive"). The prose was +# corrected; this file was not. (coderabbit on PR #926, 2026-08-11.) +# +# The rule's mechanical output is KEPT -- deleting it would hide what the +# pre-registration actually said -- but renamed to +# `interpretation_preregistered_rule`, and is no longer the field a consumer +# reads as the verdict. `applied_verdict` is, and it is gated. +subset_below_min_n = n_qual < MIN_N +if subset_below_min_n: + applied = (f"NOT PROMOTED -- the largest component (CT-F14, n={n_qual}) " + f"failed its own pre-registered n>={MIN_N} floor. Pooling cannot " + "rescue a component that did not qualify, and the " + "pre-registration had no contingency for this case. Directional " + "claim remains SUGGESTIVE (report section 5.11).") +else: + applied = interp + +print(f"\n=== COMBINED across THREE independent samples " + f"(sample1 n=4 + sample2 n=3 + CT-F14 n={n_qual}) ===") +print(f"n={n_combined}, neg={neg_combined}/{n_combined} = {frac_combined:.3f}, " + f"one-sided p={p_combined:.4f}") +print(f"PRE-REGISTERED RULE would say: {interp}") +if subset_below_min_n: + print(f"GATED -> {applied}") + +out["CT_F14_combined_3sample"] = { + "n": n_combined, "n_negative": neg_combined, + "sign_neg_frac": frac_combined, "one_sided_p": p_combined, + "applied_verdict": applied, + "interpretation_preregistered_rule": interp, + "subset_below_min_n": subset_below_min_n, + "largest_component_min_n": MIN_N, + "components": {"sample1_disp250": PRIOR_S1, "sample2_disp250": PRIOR_S2, + "CT_F14": {"n": n_qual, "neg": neg_qual}}} + +with open(pathlib.Path(__file__).with_name("comet_tail_f14.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote comet_tail_f14.json") diff --git a/probes/weather-p1/comet_tail_f4_f7.json b/probes/weather-p1/comet_tail_f4_f7.json new file mode 100644 index 00000000..e4d96a4c --- /dev/null +++ b/probes/weather-p1/comet_tail_f4_f7.json @@ -0,0 +1,143 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "t0": 91246, + "t1": 91247, + "R_disk_km": 1200.0, + "CT_F4": { + "storm1": { + "definitions": { + "A_mslp_min_subgrid": { + "lat": 55.80687987858285, + "lon": 334.4939777023923, + "error_deg": -42.5420559346087, + "wn1_frac": 0.9202092475432342 + }, + "B_lap_p_centroid": { + "lat": 55.67248760175915, + "lon": 334.5258307823925, + "error_deg": -41.03082259793251, + "wn1_frac": 0.9274214198031719 + }, + "C_vort10m_centroid": { + "lat": 55.74488562168902, + "lon": 334.50400941714094, + "error_deg": -41.84890947295523, + "wn1_frac": 0.9247416946213036 + }, + "D_z850_min_subgrid": { + "lat": 55.84988514619299, + "lon": 334.45235986651124, + "error_deg": -43.358858835550905, + "wn1_frac": 0.9190636009723636 + } + }, + "pairwise_km": { + "A_mslp_min_subgrid|B_lap_p_centroid": 15.076176192114817, + "A_mslp_min_subgrid|C_vort10m_centroid": 6.921937117137708, + "A_mslp_min_subgrid|D_z850_min_subgrid": 5.442731637081937, + "B_lap_p_centroid|C_vort10m_centroid": 8.165539470959404, + "B_lap_p_centroid|D_z850_min_subgrid": 20.254183727029453, + "C_vort10m_centroid|D_z850_min_subgrid": 12.1135289910227 + }, + "max_separation_km": 20.254183727029453, + "grid_diagonal_km": 31.923560193171888, + "F4c_degenerate": true, + "F4a_spread_deg": 2.3280362376183916, + "F4a_pass": false, + "F4b_curve_deg": { + "25": 6.498108774570255, + "50": 13.815323316109641, + "100": 30.92174366688357, + "200": 77.1715527314677 + }, + "F4b_monotone": true, + "verdict": "NO-VERDICT (F4c degenerate: definitions coincide)" + }, + "storm2": { + "definitions": { + "A_mslp_min_subgrid": { + "lat": 67.06990712939054, + "lon": 28.07950007516945, + "error_deg": -39.76952133194351, + "wn1_frac": 0.8923855403580097 + }, + "B_lap_p_centroid": { + "lat": 67.33249409332545, + "lon": 28.227820437093918, + "error_deg": -38.55815599153158, + "wn1_frac": 0.8854887175188012 + }, + "C_vort10m_centroid": { + "lat": 67.6571433498891, + "lon": 28.839775385651926, + "error_deg": -35.08918494397858, + "wn1_frac": 0.8696967018113038 + }, + "D_z850_min_subgrid": { + "lat": 67.25, + "lon": 27.617800693082515, + "error_deg": -41.634684257149104, + "wn1_frac": 0.8949910410039148 + } + }, + "pairwise_km": { + "A_mslp_min_subgrid|B_lap_p_centroid": 29.8895451740939, + "A_mslp_min_subgrid|C_vort10m_centroid": 72.95532070228214, + "A_mslp_min_subgrid|D_z850_min_subgrid": 28.251126528244853, + "B_lap_p_centroid|C_vort10m_centroid": 44.51460985013863, + "B_lap_p_centroid|D_z850_min_subgrid": 27.746146990061987, + "C_vort10m_centroid|D_z850_min_subgrid": 69.02145235651011 + }, + "max_separation_km": 72.95532070228214, + "grid_diagonal_km": 29.865768531992796, + "F4c_degenerate": false, + "F4a_spread_deg": 6.545499313170524, + "F4a_pass": true, + "F4b_curve_deg": { + "25": 4.950760999825064, + "50": 9.843879800240217, + "100": 19.277048241942765, + "200": 35.84663238862359 + }, + "F4b_monotone": true, + "verdict": "PASS \u2014 center choice does not dominate" + } + }, + "CT_F7": { + "center": { + "lat": 28.5, + "lon": 67.5 + }, + "land_fraction_ring": 0.8026728020331112, + "candidates_scanned": [ + { + "lat": 55.75, + "lon": 334.5, + "anomaly_Pa": -2954.4862955729186, + "land_fraction": 0.006198612930567842 + }, + { + "lat": 28.5, + "lon": 67.5, + "anomaly_Pa": -2608.9383192274254, + "land_fraction": 0.8026728020331112 + } + ], + "n_land_unguarded": 2642, + "n_ocean_unguarded": 743, + "n_land_oro_guarded": 1816, + "n_ocean_oro_guarded": 743, + "median_land_unguarded_deg": 43.073134580981545, + "median_land_oro_guarded_deg": 34.19596200888765, + "median_ocean_oro_guarded_deg": 20.527235763376723, + "iqr_land_deg": [ + -11.555514115716903, + 73.97162827559512 + ], + "ocean_reference_deg": 14.7, + "F7a_across_storm_pass": true, + "F7b_within_storm_paired_pass": true, + "F7b_evaluable": true, + "F7d_friction_alone_could_own_40deg_over_land": false + } +} \ No newline at end of file diff --git a/probes/weather-p1/comet_tail_f4_f7.py b/probes/weather-p1/comet_tail_f4_f7.py new file mode 100644 index 00000000..bef0d68b --- /dev/null +++ b/probes/weather-p1/comet_tail_f4_f7.py @@ -0,0 +1,460 @@ +"""EXPLORATORY — CT-F4 (sub-grid center, the blocking item) + CT-F7 (friction +over LAND, operator-requested replication). Follow-up to comet_tail_followup.py +(f6310b0e). NOT an EV; bars mine, unaudited. + +WHY F4 IS SHAPED THIS WAY. CT-F3 failed its gate: a +/-100 km center jitter +moved the alignment error by up to 29.4 deg. But +/-100 km was an amplitude I +CHOSE, not one I measured — so "the apparatus dominates" was demonstrated at an +arbitrary scale. Re-running the same jitter at a smaller amplitude would be +goalpost-moving. The non-circular question is: + + how far apart do INDEPENDENT center definitions actually land? + +That disagreement IS the center uncertainty. So F4's primary test needs no +jitter at all: compute the low-pole error from four center definitions built +from three different physical fields, and look at the spread of the ANSWER. +The jitter sweep is retained only as a supporting sensitivity CURVE (a knob +sweep, reported at four amplitudes, not a single chosen point). + +Four center definitions (deliberately not variations of one idea): + A sub-grid MSLP minimum — 2D quadratic fit on the 3x3, removes grid snap + (0.25 deg = 15.6 km zonal at 56N, 27.8 km merid.) + B grad^2 p centroid — pressure curvature, i.e. geostrophic vorticity + C 10m relative-vorticity centroid — WIND field, independent of pressure + D 850 hPa geopotential minimum, sub-grid — a different field AND a + different altitude + +PRE-REGISTERED: + +CT-F4a PRIMARY. Spread of the alignment error across definitions A-D <= 10 deg + => center choice does not dominate, and an offset becomes measurable. + > 10 deg => the offset remains unmeasurable and no constant may be fitted; + CT-E3's magnitude re-grade stands permanently, not provisionally. +CT-F4b SUPPORTING. Sensitivity curve: spread of the error over 4-direction + jitters at amplitudes 25 / 50 / 100 / 200 km. Expect MONOTONE increase; + non-monotone => the apparatus is worse than F3 already showed. +CT-F4c ANTI-VACUITY GUARD (checked BEFORE F4a is read). The four definitions + must actually DISAGREE: max pairwise separation >= one grid diagonal + (~32 km at 56N). If all four collapse to the same point, F4a passes + trivially and proves nothing about method sensitivity -> NO-VERDICT. + (This arc has shipped vacuous falsifiers repeatedly; this is the guard.) + +CT-F7 FRICTION OVER LAND (operator: "Bodenreibung bei einem anderen Sturm auf + dem Land"). Textbook: cross-isobar inflow ~10-30 deg over ocean, + ~25-45 deg over land (roughness). CT-F2 measured +14.7 / +13.0 deg over + ocean. If the apparatus is real it must resolve the land contrast. + Storm selection is BLIND to the answer: deepest NH zonal-anomaly low + whose 300-1000 km ring is >= 70% land, no inspection of its inflow. + F7a ACROSS-STORM: median alpha(land storm, land pts) >= median alpha(CT-F2 + ocean storm) + 8 deg, and lands in [20, 50] deg. + F7b WITHIN-STORM PAIRED (the stronger half — controls for depth, latitude, + curvature, which differ between storms): inside the SAME disk, + median alpha(land pts) > median alpha(ocean pts). Needs >= 500 points + of each class, else NO-VERDICT on this half. + F7c OROGRAPHY GUARD. MSLP over high terrain is an extrapolated fiction and + its gradient is unreliable, which would corrupt alpha through the + geostrophic reference rather than through friction. Points with surface + elevation > 1000 m are excluded; medians are reported BOTH ways so the + guard's effect is visible rather than assumed. (Storms 1-2 were ocean, + so CT-F2 is unaffected either way.) + F7d CONSEQUENCE, stated in advance so it cannot be spun afterwards: if the + land median reaches ~40 deg, then over land friction ALONE could own an + offset of that size — which would NOT overturn the storm-1/2 verdict + (both ~99% and ocean-only) but WOULD mean the friction bound is + surface-type dependent and cannot be applied globally. + +Same store / timestep as the whole arc. geom_ll is a strict generalisation of +comet_tail_probe.py's geom to a continuous center; the identity +geom_ll(lat[ci], lon[cj]) == geom(ci, cj) is ASSERTED numerically below so the +refactor cannot silently change the baseline. +""" +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T0, T1 = 91246, 91247 +R_E = 6371.0 +R_DISK = 1200.0 +RING = 100.0 +G0 = 9.80665 + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=600).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +def static_key(var): + """Chunk key for a static (time-independent) variable: all-zero index of the right arity.""" + return ".".join("0" * len(meta[f"{var}/.zarray"]["chunks"])) + + +print("fetching MSLP, winds, masks ...", flush=True) +lat = fetch("latitude", "0").astype(np.float64).ravel() +p0 = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0].astype(np.float64) +p1 = fetch("mean_sea_level_pressure", f"{T1}.0.0")[0].astype(np.float64) +NY, NX = p0.shape +phi = np.deg2rad(lat) +lon_deg = np.arange(NX) * 0.25 +u10 = fetch("10m_u_component_of_wind", f"{T0}.0.0")[0].astype(np.float64) +v10 = fetch("10m_v_component_of_wind", f"{T0}.0.0")[0].astype(np.float64) +lsm = fetch("land_sea_mask", static_key("land_sea_mask")).astype(np.float64) +lsm = lsm.reshape(NY, NX) if lsm.size == NY * NX else lsm[0] +zs = fetch("geopotential_at_surface", + static_key("geopotential_at_surface")).astype(np.float64) +zs = zs.reshape(NY, NX) if zs.size == NY * NX else zs[0] +elev_m = zs / G0 + +# -------------------------------------------------------------------------- + + +def geom_ll(latc, lonc): + """Continuous-center generalisation of comet_tail_probe.py's geom().""" + phic = np.deg2rad(latc) + dlon = np.deg2rad((lon_deg[None, :] - lonc + 180) % 360 - 180) + dphi = phi[:, None] - phic + dx = R_E * np.cos(phic) * dlon * np.ones((NY, 1)) + dy = R_E * dphi * np.ones((1, NX)) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def geom(ci, cj): + """dx, dy, r (km) and azimuth theta (rad, CCW from east) relative to a grid-point centre.""" + return geom_ll(lat[ci], lon_deg[cj]) + + +def find_center(field, near=None, radius_km=600.0): + """Deepest zonal-anomaly low; returns None when the (optionally `near`-limited) mask admits no finite candidate.""" + fa = field - field.mean(axis=1, keepdims=True) + mask = lat[:, None] > 15 + if near is not None: + _, _, r, _ = geom_ll(*near) + mask = mask & (r < radius_km) + masked = np.where(mask, fa, np.inf) + ci, cj = np.unravel_index(np.argmin(masked), field.shape) + # An empty mask makes `masked` all-inf and argmin returns index 0, i.e. the + # function would report grid cell (0,0) as a storm centre. A `near`-limited + # search CAN be fully masked, so this must be checked on every path + # (coderabbit on PR #926, 2026-08-11). + if not np.isfinite(masked[ci, cj]): + return None + return int(ci), int(cj) + + +def decompose_ll(field, latc, lonc): + """Verbatim decompose(), continuous center.""" + _, _, r, th = geom_ll(latc, lonc) + disk = r <= R_DISK + vals, rr, tt = field[disk], r[disk], th[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + prof = np.zeros(nb) + a1 = np.zeros(nb) + b1 = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + v, t = vals[m], tt[m] + prof[b] = v.mean() + a1[b] = 2 * ((v - prof[b]) * np.cos(t)).mean() + b1[b] = 2 * ((v - prof[b]) * np.sin(t)).mean() + resid0 = vals - prof[rings] + wn1 = a1[rings] * np.cos(tt) + b1[rings] * np.sin(tt) + amp = np.hypot(a1, b1) + w = amp * np.arange(nb) + ph = np.arctan2(np.sum(b1 * w), np.sum(a1 * w)) + return {"low_pole_rad": float((ph + np.pi) % (2 * np.pi)), + "wn1_frac": float(wn1.var() / resid0.var()), + "R2_profile_wn1": float(1.0 - np.mean((resid0 - wn1) ** 2) / vals.var())} + + +def wrap_deg(d): + """Wrap degrees into [-180, 180). + + The half-open end is LOW, not high: at exactly +180 this returns -180. + (Docstring said "(-180, 180]" until 2026-08-11 — CodeRabbit, PR #926 — + which was wrong about the code, not a bug in it.) The boundary is not + cosmetic: `stratum_verdict` scores offset > 0 as low-pole-left-of-motion, + so a value landing exactly on the boundary counts NEGATIVE. That is the + physically right call — +/-180 means the dipole points exactly OPPOSITE + the motion, which is not left-of-motion under either spelling — so the + convention is kept, now stated. Audited across every committed result + JSON: 283 angle-like values, 0 boundary hits, closest 0.91 deg. + """ + return (d + 180.0) % 360.0 - 180.0 + + +def err_deg(low_pole_rad, motion_rad): + """Signed alignment error, in degrees, of a low-pole bearing against the left-of-motion prediction.""" + return float(wrap_deg(np.rad2deg( + low_pole_rad - (motion_rad + np.pi / 2)))) + + +def subgrid_min(field, ci, cj): + """2D quadratic LS fit on the 3x3; stationary point -> (lat, lon).""" + # Longitude WRAPS: a centre at cj == 0 or NX-1 would otherwise slice a 3x2 + # neighbourhood, `A` would have 9 rows against 6 values and lstsq would + # raise. Centres come from a global scan, so the seam at 0 deg is reachable + # (coderabbit on PR #926, 2026-08-11). Rows are clamped, not wrapped — the + # poles are not periodic. + ri = np.clip(np.array([ci - 1, ci, ci + 1]), 0, field.shape[0] - 1) + z = np.take(field[ri, :], [cj - 1, cj, cj + 1], axis=1, mode="wrap").ravel() + gy, gx = np.meshgrid([-1., 0., 1.], [-1., 0., 1.], indexing="ij") + A = np.column_stack([np.ones(9), gx.ravel(), gy.ravel(), + gx.ravel() ** 2, gy.ravel() ** 2, + (gx * gy).ravel()]) + c = np.linalg.lstsq(A, z, rcond=None)[0] + _, b, cc, d, e, g = c + H = np.array([[2 * d, g], [g, 2 * e]]) + try: + dj, di = np.linalg.solve(H, [-b, -cc]) + except np.linalg.LinAlgError: + dj = di = 0.0 + di, dj = float(np.clip(di, -1, 1)), float(np.clip(dj, -1, 1)) + return (lat[ci] + di * (lat[ci + 1] - lat[ci]), + lon_deg[cj] + dj * 0.25) + + +def centroid_ll(weight, ci, cj, radius_km=300.0): + """Half-max-weighted centroid of `weight` near (ci,cj) -> (lat, lon).""" + dx, dy, r, _ = geom(ci, cj) + m = (r <= radius_km) & (weight > 0) + if not m.any(): + return lat[ci], lon_deg[cj] + w = np.clip(weight[m] - 0.5 * weight[m].max(), 0, None) + if w.sum() <= 0: + return lat[ci], lon_deg[cj] + cx = float((dx[m] * w).sum() / w.sum()) + cy = float((dy[m] * w).sum() / w.sum()) + latc = lat[ci] + np.rad2deg(cy / R_E) + lonc = lon_deg[cj] + np.rad2deg(cx / (R_E * np.cos(phi[ci]))) + return latc, lonc + + +def d_dx(f): + """Zonal derivative in per-km units (centred differences, cos(lat) metric). + + Longitude WRAPS (np.roll), it is not zero-padded. The grid is global — + 1440 x 0.25 deg — so column 0 and column 1439 are neighbours, and the + earlier `o[:, 1:-1] = ...` left both seam columns at exactly zero: a + fabricated zero gradient on a real meridian. Fixed 2026-08-11 (CodeRabbit, + PR #926). Impact on this arc's published numbers is nil-by-geometry + (nearest storm disk edge stops ~2 columns short of the seam) but the + defect was real and would bite any storm crossing 0 deg E. + """ + dxk = R_E * np.cos(phi)[:, None] * np.deg2rad(0.25) + return (np.roll(f, -1, axis=1) - np.roll(f, 1, axis=1)) / (2 * dxk) + + +def d_dy(f): + """Meridional derivative in per-km units; the row index grows southward, so the sign is flipped.""" + dyk = R_E * np.deg2rad(0.25) + o = np.zeros_like(f) + o[1:-1, :] = -(f[2:, :] - f[:-2, :]) / (2 * dyk) # index grows southward + return o + + +def sep_km(a, b): + """Great-circle-ish separation between two (lat, lon) points, in km.""" + la, lo = a + lb, lob = b + dlon = np.deg2rad((lo - lob + 180) % 360 - 180) + return float(np.hypot(R_E * np.cos(np.deg2rad((la + lb) / 2)) * dlon, + R_E * np.deg2rad(la - lb))) + + +# --- refactor guard: geom_ll must reproduce geom exactly ------------------- +_a = geom_ll(lat[200], lon_deg[500])[2] +_b = geom(200, 500)[2] +assert np.abs(_a - _b).max() == 0.0, "geom_ll is NOT identical to geom" +print(f"geom_ll identity guard: max|diff| = {np.abs(_a - _b).max():.1e} OK") + +lap_p = np.zeros_like(p0) +lap_p[1:-1, :] += p0[2:, :] + p0[:-2, :] - 2 * p0[1:-1, :] +lap_p[:, 1:-1] += p0[:, 2:] + p0[:, :-2] - 2 * p0[:, 1:-1] +zeta10 = d_dx(v10) - d_dy(u10) +gx, gy = d_dx(p0), d_dy(p0) + +print("fetching geopotential t0 (13 levels) ...", flush=True) +levels = fetch("level", "0").astype(int).ravel() +z0 = fetch("geopotential", f"{T0}.0.0.0")[0].astype(np.float64) +z850 = z0[int(np.where(levels == 850)[0][0])] + +out = {"store": B, "t0": T0, "t1": T1, "R_disk_km": R_DISK} + +# ===================== CT-F4 : SUB-GRID CENTER ============================= +print("\n=== CT-F4 independent center definitions (the blocking item) ===") +STORMS = [] +for nm, hint in (("storm1", None), ("storm2", (67.0, 28.0))): + ci0, cj0 = find_center(p0, near=hint) + ci1, cj1 = find_center(p1, near=(lat[ci0], lon_deg[cj0])) + dx, dy, _, _ = geom(ci0, cj0) + mv = (float(dx[ci1, cj1]), float(dy[ci1, cj1])) + STORMS.append({"name": nm, "ij": (ci0, cj0), + "motion_rad": float(np.arctan2(mv[1], mv[0]))}) + +f4 = {} +for st in STORMS: + ci0, cj0 = st["ij"] + mth = st["motion_rad"] + zi, zj = find_center(z850, near=(lat[ci0], lon_deg[cj0]), radius_km=400.0) + defs = { + "A_mslp_min_subgrid": subgrid_min(p0, ci0, cj0), + "B_lap_p_centroid": centroid_ll(lap_p, ci0, cj0), + "C_vort10m_centroid": centroid_ll(zeta10, ci0, cj0), + "D_z850_min_subgrid": subgrid_min(z850, zi, zj), + } + rows = {} + for k, (la, lo) in defs.items(): + d = decompose_ll(p0, la, lo) + rows[k] = {"lat": la, "lon": lo, + "error_deg": err_deg(d["low_pole_rad"], mth), + "wn1_frac": d["wn1_frac"]} + keys = list(defs) + pair = {f"{a}|{b}": sep_km(defs[a], defs[b]) + for i, a in enumerate(keys) for b in keys[i + 1:]} + max_sep = max(pair.values()) + grid_diag = sep_km((lat[ci0], lon_deg[cj0]), + (lat[ci0 + 1], lon_deg[cj0] + 0.25)) + errs = np.array([r["error_deg"] for r in rows.values()]) + rel = wrap_deg(errs - errs[0]) + spread = float(rel.max() - rel.min()) + degenerate = max_sep < grid_diag + + curve = {} + for amp in (25.0, 50.0, 100.0, 200.0): + es = [] + for ang in (0, 90, 180, 270): + a = np.deg2rad(ang) + mth + la = lat[ci0] + np.rad2deg(amp * np.sin(a) / R_E) + lo = lon_deg[cj0] + np.rad2deg( + amp * np.cos(a) / (R_E * np.cos(phi[ci0]))) + es.append(err_deg(decompose_ll(p0, la, lo)["low_pole_rad"], mth)) + rl = wrap_deg(np.array(es) - errs[0]) + curve[int(amp)] = float(rl.max() - rl.min()) + amps = sorted(curve) + monotone = all(curve[amps[i]] <= curve[amps[i + 1]] + 1e-9 + for i in range(len(amps) - 1)) + + verdict = ("NO-VERDICT (F4c degenerate: definitions coincide)" if degenerate + else "PASS — center choice does not dominate" if spread <= 10 + else "FAIL — offset remains unmeasurable") + f4[st["name"]] = {"definitions": rows, "pairwise_km": pair, + "max_separation_km": max_sep, + "grid_diagonal_km": grid_diag, + "F4c_degenerate": bool(degenerate), + "F4a_spread_deg": spread, + "F4a_pass": bool((not degenerate) and spread <= 10), + "F4b_curve_deg": curve, "F4b_monotone": bool(monotone), + "verdict": verdict} + print(f"\n{st['name']}:") + for k, r in rows.items(): + print(f" {k:22s} ({r['lat']:7.3f}N,{r['lon']:8.3f}E) " + f"error {r['error_deg']:+7.1f} wn1 {r['wn1_frac']:.3f}") + print(f" max pairwise separation {max_sep:6.1f} km " + f"(grid diagonal {grid_diag:.1f} km) -> " + f"F4c degenerate: {degenerate}") + print(f" F4a spread across A-D = {spread:.1f} deg (bar <= 10) -> {verdict}") + print(f" F4b sensitivity curve (km -> deg): " + + ", ".join(f"{a}:{curve[a]:.1f}" for a in amps) + + f" monotone: {monotone}") +out["CT_F4"] = f4 + +# =============== CT-F7 : FRICTION OVER LAND (blind selection) ============== +print("\n=== CT-F7 friction over LAND (storm chosen blind to its inflow) ===") + + +def inflow(mask): + """Signed 10m cross-isobar inflow angle in degrees; positive = turned toward the low (NH friction sign).""" + bg = np.arctan2(gx[mask], -gy[mask]) + ba = np.arctan2(v10[mask], u10[mask]) + return wrap_deg(np.rad2deg(ba - bg)) + + +fa = p0 - p0.mean(axis=1, keepdims=True) +cand = np.where((lat[:, None] > 25) & (lat[:, None] < 70), fa, np.inf) +picked = None +tried = [] +work = cand.copy() +for _ in range(40): + ci, cj = np.unravel_index(np.argmin(work), work.shape) + if not np.isfinite(work[ci, cj]): + break + _, _, r, _ = geom(ci, cj) + band = (r >= 300.0) & (r <= 1000.0) + lf = float(lsm[band].mean()) + tried.append({"lat": float(lat[ci]), "lon": float(lon_deg[cj]), + "anomaly_Pa": float(fa[ci, cj]), "land_fraction": lf}) + if lf >= 0.70 and picked is None: + picked = (ci, cj, lf) + break + work[r < 800.0] = np.inf + +if picked is None: + print(" no NH low with >=70% land in its 300-1000 km ring — NO-VERDICT") + out["CT_F7"] = {"verdict": "NO-VERDICT (no qualifying land storm)", + "candidates_scanned": tried} +else: + ci, cj, lf = picked + _, _, r, _ = geom(ci, cj) + spd = np.hypot(u10, v10) + base = (r >= 300.0) & (r <= 1000.0) & (spd > 3.0) + lo_oro = base & (elev_m <= 1000.0) + land_a = inflow(base & (lsm >= 0.5)) + ocean_a = inflow(base & (lsm < 0.5)) + land_g = inflow(lo_oro & (lsm >= 0.5)) + ocean_g = inflow(lo_oro & (lsm < 0.5)) + med = lambda a: float(np.median(a)) if a.size else None # noqa: E731 + OCEAN_REF = 14.7 # CT-F2 storm1, 99% ocean + ml, mo = med(land_g), med(ocean_g) + f7a = bool(ml is not None and ml >= OCEAN_REF + 8.0 and 20.0 <= ml <= 50.0) + paired_ok = land_g.size >= 500 and ocean_g.size >= 500 + f7b = bool(paired_ok and ml is not None and mo is not None and ml > mo) + out["CT_F7"] = { + "center": {"lat": float(lat[ci]), "lon": float(lon_deg[cj])}, + "land_fraction_ring": lf, "candidates_scanned": tried, + "n_land_unguarded": int(land_a.size), "n_ocean_unguarded": int(ocean_a.size), + "n_land_oro_guarded": int(land_g.size), + "n_ocean_oro_guarded": int(ocean_g.size), + "median_land_unguarded_deg": med(land_a), + "median_land_oro_guarded_deg": ml, + "median_ocean_oro_guarded_deg": mo, + "iqr_land_deg": [float(np.percentile(land_g, 25)), + float(np.percentile(land_g, 75))] if land_g.size else None, + "ocean_reference_deg": OCEAN_REF, + "F7a_across_storm_pass": f7a, + "F7b_within_storm_paired_pass": f7b, + "F7b_evaluable": bool(paired_ok), + # The key names 40 deg and the docstring pre-registers 40 deg, so the + # test uses 40 deg. It read >= 35.0, which would have reported true for + # a 36 deg median under a key claiming 40 (coderabbit on PR #926). + "F7d_friction_alone_could_own_40deg_over_land": + bool(ml is not None and ml >= 40.0), + } + print(f" storm chosen: ({lat[ci]:.2f}N, {lon_deg[cj]:.2f}E), " + f"ring land fraction {lf:.2f}, anomaly {fa[ci, cj]:.0f} Pa") + print(f" land n={land_g.size:5d} (oro-guarded, was {land_a.size}) " + f"median {ml if ml is None else round(ml, 1)} deg") + print(f" ocean n={ocean_g.size:5d} median " + f"{mo if mo is None else round(mo, 1)} deg") + print(f" F7a across-storm (>= {OCEAN_REF}+8 and in [20,50]): {f7a}") + print(f" F7b within-storm paired (land > ocean, n>=500 each): {f7b} " + f"(evaluable: {paired_ok})") + print(f" F7d friction alone could own 40 deg OVER LAND: " + f"{out['CT_F7']['F7d_friction_alone_could_own_40deg_over_land']}") + +with open(pathlib.Path(__file__).with_name("comet_tail_f4_f7.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote comet_tail_f4_f7.json") diff --git a/probes/weather-p1/comet_tail_f5_n10.json b/probes/weather-p1/comet_tail_f5_n10.json new file mode 100644 index 00000000..4edb240c --- /dev/null +++ b/probes/weather-p1/comet_tail_f5_n10.json @@ -0,0 +1,461 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "CT_F5": { + "storm1": { + "walk": [ + { + "level_hPa": 1000, + "found": true, + "center_lat": 55.75, + "center_lon": 334.5, + "step_km": 0.0, + "error_deg": -40.46281630581089, + "wn1_frac": 0.9239743148665512 + }, + { + "level_hPa": 925, + "found": true, + "center_lat": 55.75, + "center_lon": 334.5, + "step_km": 0.0, + "error_deg": -32.49848062992805, + "wn1_frac": 0.9291294230426438 + }, + { + "level_hPa": 850, + "found": true, + "center_lat": 55.75, + "center_lon": 334.5, + "step_km": 0.0, + "error_deg": -23.84972881632487, + "wn1_frac": 0.9313941524789896 + }, + { + "level_hPa": 700, + "found": true, + "center_lat": 55.75, + "center_lon": 334.5, + "step_km": 0.0, + "error_deg": -8.135863343362843, + "wn1_frac": 0.9383064883729374 + }, + { + "level_hPa": 600, + "found": true, + "center_lat": 56.0, + "center_lon": 334.25, + "step_km": 31.874386008833685, + "error_deg": -2.0835685390135836, + "wn1_frac": 0.9407521073422084 + }, + { + "level_hPa": 500, + "found": true, + "center_lat": 56.5, + "center_lon": 334.0, + "step_km": 57.70268303955165, + "error_deg": 2.929258589915861, + "wn1_frac": 0.9397828007331679 + }, + { + "level_hPa": 400, + "found": true, + "center_lat": 57.0, + "center_lon": 334.25, + "step_km": 57.64886914018281, + "error_deg": 8.693329705454858, + "wn1_frac": 0.9351995900375635 + }, + { + "level_hPa": 300, + "found": true, + "center_lat": 58.0, + "center_lon": 332.25, + "step_km": 163.22428522804827, + "error_deg": 7.799928849981455, + "wn1_frac": 0.91698885957228 + }, + { + "level_hPa": 250, + "found": true, + "center_lat": 58.25, + "center_lon": 331.25, + "step_km": 64.96635838297796, + "error_deg": 8.693684473120186, + "wn1_frac": 0.9301703153070721 + }, + { + "level_hPa": 200, + "found": true, + "center_lat": 58.5, + "center_lon": 329.5, + "step_km": 105.75431188308447, + "error_deg": 6.3267716796407, + "wn1_frac": 0.9518008032307684 + }, + { + "level_hPa": 150, + "found": true, + "center_lat": 59.75, + "center_lon": 326.0, + "step_km": 243.3217640666972, + "error_deg": 1.4392427165792299, + "wn1_frac": 0.9728261172739403 + }, + { + "level_hPa": 100, + "found": true, + "center_lat": 60.25, + "center_lon": 321.75, + "step_km": 242.7419885394034, + "error_deg": 0.34572713491658646, + "wn1_frac": 0.9777388272593495 + }, + { + "level_hPa": 50, + "found": true, + "center_lat": 61.75, + "center_lon": 318.5, + "step_km": 241.89980011103893, + "error_deg": 101.22296579577375, + "wn1_frac": 0.784430214085918 + } + ], + "max_step_km": 243.3217640666972, + "CT_F5b_no_saturation": true, + "best_level_in_400_850": 600, + "best_error_in_400_850_deg": -2.0835685390135836, + "CT_F5a_pass": true, + "surface_error_deg": -40.46281630581089 + }, + "storm2": { + "walk": [ + { + "level_hPa": 1000, + "found": true, + "center_lat": 67.0, + "center_lon": 28.0, + "step_km": 0.0, + "error_deg": -39.746495421917615, + "wn1_frac": 0.9013658742632329 + }, + { + "level_hPa": 925, + "found": true, + "center_lat": 67.0, + "center_lon": 27.75, + "step_km": 10.861829792527612, + "error_deg": -30.747604434365485, + "wn1_frac": 0.9315082183168824 + }, + { + "level_hPa": 850, + "found": true, + "center_lat": 67.0, + "center_lon": 27.5, + "step_km": 10.861829792527612, + "error_deg": -23.661401835548673, + "wn1_frac": 0.9453390686389784 + }, + { + "level_hPa": 700, + "found": true, + "center_lat": 67.0, + "center_lon": 27.0, + "step_km": 21.723659585055223, + "error_deg": -14.448233406134591, + "wn1_frac": 0.96208166112389 + }, + { + "level_hPa": 600, + "found": true, + "center_lat": 66.75, + "center_lon": 26.75, + "step_km": 29.865768531992796, + "error_deg": -9.453202593238132, + "wn1_frac": 0.9732871055886206 + }, + { + "level_hPa": 500, + "found": true, + "center_lat": 66.75, + "center_lon": 26.75, + "step_km": 0.0, + "error_deg": -4.535940468762789, + "wn1_frac": 0.979546746038315 + }, + { + "level_hPa": 400, + "found": true, + "center_lat": 65.75, + "center_lon": 21.75, + "step_km": 250.0062182281284, + "error_deg": -9.967555334404125, + "wn1_frac": 0.9753646009204503 + }, + { + "level_hPa": 300, + "found": true, + "center_lat": 66.75, + "center_lon": 17.0, + "step_km": 240.03036558362672, + "error_deg": -15.518762931104476, + "wn1_frac": 0.9532764219257279 + }, + { + "level_hPa": 250, + "found": true, + "center_lat": 68.25, + "center_lon": 13.0, + "step_km": 238.30880343700352, + "error_deg": -18.21717334583633, + "wn1_frac": 0.9113507159715785 + }, + { + "level_hPa": 200, + "found": true, + "center_lat": 69.75, + "center_lon": 8.5, + "step_km": 244.8980511482822, + "error_deg": -15.029233239201972, + "wn1_frac": 0.8615742247192899 + }, + { + "level_hPa": 150, + "found": true, + "center_lat": 71.5, + "center_lon": 4.5, + "step_km": 244.20958114913347, + "error_deg": -3.7980101602379364, + "wn1_frac": 0.8203884791212692 + }, + { + "level_hPa": 100, + "found": true, + "center_lat": 71.75, + "center_lon": 357.5, + "step_km": 246.93741081543493, + "error_deg": 9.629025896242183, + "wn1_frac": 0.7662752070006318 + }, + { + "level_hPa": 50, + "found": true, + "center_lat": 72.75, + "center_lon": 351.25, + "step_km": 239.27733129052552, + "error_deg": 91.60679688095513, + "wn1_frac": 0.7830473042485049 + } + ], + "max_step_km": 250.0062182281284, + "CT_F5b_no_saturation": false, + "best_level_in_400_850": 500, + "best_error_in_400_850_deg": -4.535940468762789, + "CT_F5a_pass": true, + "surface_error_deg": -39.746495421917615 + } + }, + "CT_N": { + "n_dates_scanned": 10, + "n_valid": 10, + "n_excluded": 0, + "excluded": [], + "rows": [ + { + "date": "2021-06-15T12:00:00", + "t0": 91246, + "center_lat": 55.80687987858285, + "center_lon": 334.4939777023923, + "anomaly_Pa": -2954.4862955729186, + "status": "OK", + "displacement_km": 276.8999854203758, + "motion_bearing_deg": 16.160961598455202, + "CT_E1_wn1_frac": 0.9202092475432342, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": -41.29391345304771, + "CT_E4_R2_profile_wn1": 0.9717218278560205, + "F8_error_C_vort_deg": -40.60076699139424, + "F8_shrinks_at_vort_center": true, + "F9_land_dipole_amp_mean": 0.032741717222016825, + "F9_unexplained_residual": 0.02827817214397954 + }, + { + "date": "2020-01-10T12:00:00", + "t0": 89158, + "center_lat": 61.76332083974577, + "center_lon": 336.76458162678887, + "anomaly_Pa": -6435.60147569445, + "status": "OK", + "displacement_km": 250.01498052561794, + "motion_bearing_deg": 84.13750576837133, + "CT_E1_wn1_frac": 0.7501674763254162, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": 4.97393064812556, + "CT_E4_R2_profile_wn1": 0.9060564303343459, + "F8_error_C_vort_deg": 4.518224943103888, + "F8_shrinks_at_vort_center": true, + "F9_land_dipole_amp_mean": 0.18841767640411589, + "F9_unexplained_residual": 0.09394356966565409 + }, + { + "date": "2020-07-20T12:00:00", + "t0": 89926, + "center_lat": 32.00448422802389, + "center_lon": 84.55110630844109, + "anomaly_Pa": -2761.386664496531, + "status": "OK", + "displacement_km": 406.3264370931077, + "motion_bearing_deg": 160.07153054906823, + "CT_E1_wn1_frac": 0.23031439133632856, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": -39.03893530327841, + "CT_E4_R2_profile_wn1": 0.32318999155113404, + "F8_error_C_vort_deg": -36.79985002873886, + "F8_shrinks_at_vort_center": true, + "F9_land_dipole_amp_mean": 0.015877602961425315, + "F9_unexplained_residual": 0.676810008448866 + }, + { + "date": "2019-03-05T12:00:00", + "t0": 87914, + "center_lat": 40.994608546917, + "center_lon": 164.57271453394864, + "anomaly_Pa": -4613.096446397569, + "status": "OK", + "displacement_km": 454.89233163829124, + "motion_bearing_deg": 47.27720997508244, + "CT_E1_wn1_frac": 0.7195133651262611, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": -67.84110258667374, + "CT_E4_R2_profile_wn1": 0.9300416121473195, + "F8_error_C_vort_deg": -109.59291315167077, + "F8_shrinks_at_vort_center": false, + "F9_land_dipole_amp_mean": 8.392273127481189e-07, + "F9_unexplained_residual": 0.06995838785268049 + }, + { + "date": "2019-10-25T12:00:00", + "t0": 88850, + "center_lat": 48.232948733448765, + "center_lon": 181.3731924348638, + "anomaly_Pa": -5508.608865017362, + "status": "OK", + "displacement_km": 185.38754145594066, + "motion_bearing_deg": 49.46188739415042, + "CT_E1_wn1_frac": 0.3826733777910667, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": -107.56851600599256, + "CT_E4_R2_profile_wn1": 0.8944783799153628, + "F8_error_C_vort_deg": -103.34756787847593, + "F8_shrinks_at_vort_center": true, + "F9_land_dipole_amp_mean": 0.0042608176446558, + "F9_unexplained_residual": 0.10552162008463717 + }, + { + "date": "2014-09-12T12:00:00", + "t0": 81374, + "center_lat": 57.477701985466574, + "center_lon": 189.60581664149174, + "anomaly_Pa": -3804.4426595052064, + "status": "OK", + "displacement_km": 155.79931373309316, + "motion_bearing_deg": 179.0881433023282, + "CT_E1_wn1_frac": 0.49333651672920176, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": 3.1864161905094193, + "CT_E4_R2_profile_wn1": 0.8711762185147772, + "F8_error_C_vort_deg": -12.231585723069259, + "F8_shrinks_at_vort_center": false, + "F9_land_dipole_amp_mean": 0.19965406402569774, + "F9_unexplained_residual": 0.12882378148522278 + }, + { + "date": "2018-08-08T12:00:00", + "t0": 87078, + "center_lat": 34.34816558383418, + "center_lon": 141.0810553314141, + "anomaly_Pa": -2506.3990071614535, + "status": "OK", + "displacement_km": 128.29413009287907, + "motion_bearing_deg": 93.3251303108303, + "CT_E1_wn1_frac": 0.8690169152909185, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": 19.419896683788465, + "CT_E4_R2_profile_wn1": 0.9185974636741742, + "F8_error_C_vort_deg": 18.378751022945096, + "F8_shrinks_at_vort_center": true, + "F9_land_dipole_amp_mean": 0.16900163752766473, + "F9_unexplained_residual": 0.08140253632582584 + }, + { + "date": "2017-11-30T12:00:00", + "t0": 86074, + "center_lat": 53.05490702247342, + "center_lon": 161.89484928373727, + "anomaly_Pa": -4879.580777994794, + "status": "OK", + "displacement_km": 157.52167018912363, + "motion_bearing_deg": -2.2212834840136217, + "CT_E1_wn1_frac": 0.5336721341668013, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": -49.09763383875617, + "CT_E4_R2_profile_wn1": 0.9084832303074415, + "F8_error_C_vort_deg": -101.23788670442838, + "F8_shrinks_at_vort_center": false, + "F9_land_dipole_amp_mean": 0.2735047290202235, + "F9_unexplained_residual": 0.09151676969255851 + }, + { + "date": "2016-04-18T12:00:00", + "t0": 83710, + "center_lat": 74.99235874483051, + "center_lon": 34.17923473226179, + "anomaly_Pa": -2846.3334255642403, + "status": "OK", + "displacement_km": 113.26656979918488, + "motion_bearing_deg": -13.764163971903077, + "CT_E1_wn1_frac": 0.7262198061965601, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": 165.74524346486282, + "CT_E4_R2_profile_wn1": 0.8297778267876115, + "F8_error_C_vort_deg": 166.56350091696947, + "F8_shrinks_at_vort_center": false, + "F9_land_dipole_amp_mean": 0.1523592531762006, + "F9_unexplained_residual": 0.17022217321238853 + }, + { + "date": "2015-12-25T12:00:00", + "t0": 83250, + "center_lat": 53.595933945152815, + "center_lon": 198.4544267473596, + "anomaly_Pa": -5083.723589409725, + "status": "OK", + "displacement_km": 131.78366990126747, + "motion_bearing_deg": 103.15329163600231, + "CT_E1_wn1_frac": 0.7577768563687628, + "CT_E2_trackable": true, + "CT_E3_error_A_mslp_deg": -19.6650235396622, + "CT_E4_R2_profile_wn1": 0.8874979838957339, + "F8_error_C_vort_deg": -14.95115801528786, + "F8_shrinks_at_vort_center": true, + "F9_land_dipole_amp_mean": 0.1739603509911075, + "F9_unexplained_residual": 0.11250201610426613 + } + ], + "CT_N1_sign_frac_negative": 0.6, + "CT_N1_pass": false, + "CT_N2_median_abs_error_deg": 40.16642437816306, + "CT_N2_iqr_deg": [ + 19.4811783977569, + 63.15523539969435 + ], + "CT_N3_median_wn1_frac": 0.7228665856614106, + "CT_N3_pass": true, + "CT_N4_median_R2": 0.9002674051248544, + "CT_N4_pass": true, + "CT_N5_F8_shrink_frac": 0.6, + "CT_N5_pass": false, + "CT_F9_corr_land_dipole_vs_residual": -0.2947669853570691 + } +} \ No newline at end of file diff --git a/probes/weather-p1/comet_tail_f5_n10.py b/probes/weather-p1/comet_tail_f5_n10.py new file mode 100644 index 00000000..85d3f7e1 --- /dev/null +++ b/probes/weather-p1/comet_tail_f5_n10.py @@ -0,0 +1,525 @@ +"""EXPLORATORY — CT-F5 (fix F1's saturation defect) + CT-N (n=10 blind storm +sample: the binding constraint after CT-F4 cleared the apparatus). Follow-up +to comet_tail_f4_f7.py (432bcab2). NOT an EV; bars mine, unaudited. + +============================== CT-F5 ======================================== +CT-F1 (comet_tail_followup.py) swept 13 pressure levels searching for each +level's own low CENTER within a FIXED 600 km radius of the SURFACE center. +Storm 2's search saturated at exactly that radius for 5 of 13 levels +(586-599 km) -- it never found a co-located upper center and locked onto a +different system, producing a physically-absurd "best level = 100 hPa". + +FIX: track the center level-by-level, searching near the PREVIOUS level's +found center (radius 250 km per step) instead of always near the surface. +This lets the center walk continuously along the tilt axis with height, +rather than being asked to jump the whole tilt in one 600 km hop from the +surface. Levels are walked from 1000 hPa UPWARD (surface-anchored, since +that is the level both storms' baseline tracking used). + +PRE-REGISTERED: + CT-F5a storm 2's walking-center path clears CT-F1's original bar (best + level in 400-850 hPa AND |error| <= 20 deg there) -- the walking + fix is credited only if it turns a NO-VERDICT into a real pass, + not merely a different number. + CT-F5b no level's walking step exceeds 250 km (the search radius) -- + if the walk itself saturates, the fix has the same disease at a + different radius and must be reported as such, not silently capped. + CT-F5c storm 1 (which already passed) must reproduce its original + 400-850 hPa crossing within 10 deg -- the fix must not be a free + rewrite that also perturbs the case that was already correct. + +============================== CT-N ========================================= +CT-F4 showed the apparatus (center choice) is NOT what makes the -40 deg +offset unreliable; CT-F7 showed friction is bounded and mis-scoped as the +mechanism. What remains untested at n=2 is whether -40 deg is a real +central tendency across INDEPENDENT synoptic times, or an n=2 coincidence. + +Ten dates, BLIND selection (no hint, no inspection before recording): NH, +25-75 lat, deepest zonal-anomaly MSLP low at 12Z, one per date, spanning +~6 years and all four seasons so storms are independent systems, not the +same event re-sampled. The anchor date (2021-06-15, storm 1) is INCLUDED +and its t-index is asserted against the arc's pinned T0=91246 so the whole +chain stays anchored to previously-published numbers. + +PRE-REGISTERED (storms failing CT-E2 trackability are EXCLUDED from N1-N5, +their exclusion reported, not silently dropped): + CT-N1 SIGN consistency: among valid storms, count negative alignment + error (same rotational sense as storms 1-2) / total. + Bar: >= 0.70 -- majority-same-sign is the minimum for "systematic", + not requiring unanimity (n=2 was already not unanimous-required, + it was 2/2 by chance of only having 2). + CT-N2 MAGNITUDE: median |error| and IQR reported (no bar -- this is the + number that tells us if -40 is a central tendency or storms 1-2 + were the extreme tail of a wide distribution). + CT-N3 wn1 DOMINANCE at scale: median wn1_frac >= 0.40 (replicates E1). + CT-N4 R2 at scale: median R2_profile_wn1 >= 0.80 (replicates E4). + CT-N5 = CT-F8: vorticity-centroid (wind) center vs sub-grid MSLP center, + PAIRED sign test per storm: does |error| shrink at the wind center + more often than not? Bar: >= 0.70 of valid storms show shrinkage + (storm 2 showed this in F4; testing if it generalizes). + CT-F9 Land-fraction ASYMMETRY vs pressure residual: fit the SAME + ring/wn-1 decomposition to land_sea_mask around each center, + producing a land-dipole magnitude and bearing per storm. Under the + Ekman-pumping candidate, storms with a stronger land dipole should + show a LARGER unexplained pressure residual (1 - R2_profile_wn1). + Bar: corr(land_dipole_amplitude, 1-R2_profile_wn1) reported with a + sign call -- POSITIVE supports candidate 2 residually mattering, + near-zero/negative does not. No pre-set threshold (this is n=10, + exploratory-of-exploratory; correlation SIGN is the only claim). + +NOT tested here: n=10 is not n>=30; no offset CONSTANT is fitted from CT-N2, +only its distribution is reported. CT-F5's fix does not retroactively change +any bar already scored in COMET_TAIL_REPORT.md sec 5.2 -- it is reported as +a correction of that section's storm-2 verdict, dated and cited. + +RUN LOG (transparency): run 1 crashed on a 404 for 2022-02-14 -- the store's +own filename claims "1959-2022" coverage but its actual last timestep is +2021-12-31 18Z (six months short), diagnosed against the .zarray shape before +any code changed. Added a bounds guard (report+exclude, never crash) and +swapped that one date for 2014-09-12 (in-bounds, autumn, different decade) to +keep n=10 candidate dates. No bar above was touched. +""" +import datetime +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +R_E = 6371.0 +R_DISK = 1200.0 +RING = 100.0 +EPOCH = datetime.datetime(1959, 1, 1) + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=600).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +def static_key(var): + """Chunk key for a static (time-independent) variable: all-zero index of the right arity.""" + return ".".join("0" * len(meta[f"{var}/.zarray"]["chunks"])) + + +def t_index(dt): + """WB2 time index for a datetime: 6-hourly steps since 1959-01-01.""" + return int(round((dt - EPOCH).total_seconds() / 3600 / 6)) + + +assert t_index(datetime.datetime(2021, 6, 15, 12)) == 91246, \ + "t-index formula does not reproduce the pinned arc anchor T0=91246" +print("t-index anchor guard: OK (2021-06-15 12Z -> 91246)") + +_MSLP_SHAPE = meta["mean_sea_level_pressure/.zarray"]["shape"] +_MAX_T = _MSLP_SHAPE[0] - 1 +_last_valid = EPOCH + datetime.timedelta(hours=_MAX_T * 6) +print(f"store bounds guard: max valid t={_MAX_T} " + f"(last timestep {_last_valid.isoformat()}Z) — " + f"filename says '1959-2022' but coverage actually ends here") + +print("fetching static fields (latitude, land_sea_mask) ...", flush=True) +lat = fetch("latitude", "0").astype(np.float64).ravel() +NY = lat.size +NX = 1440 +phi = np.deg2rad(lat) +lon_deg = np.arange(NX) * 0.25 +lsm = fetch("land_sea_mask", static_key("land_sea_mask")).astype(np.float64) +lsm = lsm.reshape(NY, NX) if lsm.size == NY * NX else lsm[0] + +# ---- verbatim / near-verbatim helpers, consistent with comet_tail_f4_f7.py - + + +def geom_ll(latc, lonc): + """dx, dy, r (km) and azimuth theta (rad, CCW from east) relative to a CONTINUOUS (lat, lon) centre.""" + phic = np.deg2rad(latc) + dlon = np.deg2rad((lon_deg[None, :] - lonc + 180) % 360 - 180) + dphi = phi[:, None] - phic + dx = R_E * np.cos(phic) * dlon * np.ones((NY, 1)) + dy = R_E * dphi * np.ones((1, NX)) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def find_center(field, near=None, radius_km=600.0, lat_lo=25.0, lat_hi=75.0): + """Deepest zonal-anomaly low; returns None when the (optionally `near`-limited) mask admits no finite candidate.""" + fa = field - field.mean(axis=1, keepdims=True) + mask = (lat[:, None] > lat_lo) & (lat[:, None] < lat_hi) + if near is not None: + _, _, r, _ = geom_ll(*near) + mask = mask & (r < radius_km) + masked = np.where(mask, fa, np.inf) + ci, cj = np.unravel_index(np.argmin(masked), field.shape) + # An empty mask makes `masked` all-inf and argmin returns index 0, i.e. the + # function would report grid cell (0,0) as a storm centre. A `near`-limited + # search CAN be fully masked, so this must be checked on every path + # (coderabbit on PR #926, 2026-08-11). + if not np.isfinite(masked[ci, cj]): + return None + return int(ci), int(cj) + + +def decompose_ll(field, latc, lonc): + """Ring-mean profile + per-ring wavenumber-1 fit about a continuous centre.""" + _, _, r, th = geom_ll(latc, lonc) + disk = r <= R_DISK + vals, rr, tt = field[disk], r[disk], th[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + prof = np.zeros(nb) + a1 = np.zeros(nb) + b1 = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + v, t = vals[m], tt[m] + prof[b] = v.mean() + a1[b] = 2 * ((v - prof[b]) * np.cos(t)).mean() + b1[b] = 2 * ((v - prof[b]) * np.sin(t)).mean() + resid0 = vals - prof[rings] + wn1 = a1[rings] * np.cos(tt) + b1[rings] * np.sin(tt) + amp = np.hypot(a1, b1) + w = amp * np.arange(nb) + ph = np.arctan2(np.sum(b1 * w), np.sum(a1 * w)) + return {"low_pole_rad": float((ph + np.pi) % (2 * np.pi)), + "wn1_frac": float(wn1.var() / max(resid0.var(), 1e-12)), + "R2_profile_wn1": float(1.0 - np.mean((resid0 - wn1) ** 2) / vals.var()), + "amp_by_ring": amp, "ring_mid_km": (np.arange(nb) + 0.5) * RING} + + +def subgrid_min(field, ci, cj): + # Longitude WRAPS: a centre at cj == 0 or NX-1 would otherwise slice a 3x2 + # neighbourhood, `A` would have 9 rows against 6 values and lstsq would + # raise. Centres come from a global scan, so the seam at 0 deg is reachable + # (coderabbit on PR #926, 2026-08-11). Rows are clamped, not wrapped — the + # poles are not periodic. + """Sub-grid minimum by 2-D quadratic fit on the 3x3 neighbourhood; longitude wraps, latitude clamps.""" + ri = np.clip(np.array([ci - 1, ci, ci + 1]), 0, field.shape[0] - 1) + z = np.take(field[ri, :], [cj - 1, cj, cj + 1], axis=1, mode="wrap").ravel() + gy, gx = np.meshgrid([-1., 0., 1.], [-1., 0., 1.], indexing="ij") + A = np.column_stack([np.ones(9), gx.ravel(), gy.ravel(), + gx.ravel() ** 2, gy.ravel() ** 2, + (gx * gy).ravel()]) + c = np.linalg.lstsq(A, z, rcond=None)[0] + _, b, cc, d, e, g = c + H = np.array([[2 * d, g], [g, 2 * e]]) + try: + dj, di = np.linalg.solve(H, [-b, -cc]) + except np.linalg.LinAlgError: + dj = di = 0.0 + di, dj = float(np.clip(di, -1, 1)), float(np.clip(dj, -1, 1)) + lat_step = lat[ci + 1] - lat[ci] if ci + 1 < NY else lat[ci] - lat[ci - 1] + return lat[ci] + di * lat_step, lon_deg[cj] + dj * 0.25 + + +def centroid_ll(weight, ci, cj, radius_km=300.0): + """Half-max-weighted centroid of `weight` near a centre, as continuous (lat, lon).""" + dx, dy, r, _ = geom_ll(lat[ci], lon_deg[cj]) + m = (r <= radius_km) & (weight > 0) + if not m.any(): + return lat[ci], lon_deg[cj] + w = np.clip(weight[m] - 0.5 * weight[m].max(), 0, None) + if w.sum() <= 0: + return lat[ci], lon_deg[cj] + cx = float((dx[m] * w).sum() / w.sum()) + cy = float((dy[m] * w).sum() / w.sum()) + latc = lat[ci] + np.rad2deg(cy / R_E) + lonc = lon_deg[cj] + np.rad2deg(cx / (R_E * np.cos(phi[ci]))) + return latc, lonc + + +def d_dx(f): + """Zonal derivative in per-km units (centred differences, cos(lat) metric). + + Longitude WRAPS (np.roll), it is not zero-padded. The grid is global — + 1440 x 0.25 deg — so column 0 and column 1439 are neighbours, and the + earlier `o[:, 1:-1] = ...` left both seam columns at exactly zero: a + fabricated zero gradient on a real meridian. Fixed 2026-08-11 (CodeRabbit, + PR #926). Impact on this arc's published numbers is nil-by-geometry + (nearest storm disk edge stops ~2 columns short of the seam) but the + defect was real and would bite any storm crossing 0 deg E. + """ + dxk = R_E * np.cos(phi)[:, None] * np.deg2rad(0.25) + return (np.roll(f, -1, axis=1) - np.roll(f, 1, axis=1)) / (2 * dxk) + + +def d_dy(f): + """Meridional derivative in per-km units; the row index grows southward, so the sign is flipped.""" + dyk = R_E * np.deg2rad(0.25) + o = np.zeros_like(f) + o[1:-1, :] = -(f[2:, :] - f[:-2, :]) / (2 * dyk) + return o + + +def wrap_deg(d): + """Wrap degrees into [-180, 180). + + The half-open end is LOW, not high: at exactly +180 this returns -180. + (Docstring said "(-180, 180]" until 2026-08-11 — CodeRabbit, PR #926 — + which was wrong about the code, not a bug in it.) The boundary is not + cosmetic: `stratum_verdict` scores offset > 0 as low-pole-left-of-motion, + so a value landing exactly on the boundary counts NEGATIVE. That is the + physically right call — +/-180 means the dipole points exactly OPPOSITE + the motion, which is not left-of-motion under either spelling — so the + convention is kept, now stated. Audited across every committed result + JSON: 283 angle-like values, 0 boundary hits, closest 0.91 deg. + """ + return (d + 180.0) % 360.0 - 180.0 + + +def err_deg(low_pole_rad, motion_rad): + """Signed alignment error, in degrees, of a low-pole bearing against the left-of-motion prediction.""" + return float(wrap_deg(np.rad2deg( + low_pole_rad - (motion_rad + np.pi / 2)))) + + +def sep_km(a, b): + """Great-circle-ish separation between two (lat, lon) points, in km.""" + la, lo = a + lb, lob = b + dlon = np.deg2rad((lo - lob + 180) % 360 - 180) + return float(np.hypot(R_E * np.cos(np.deg2rad((la + lb) / 2)) * dlon, + R_E * np.deg2rad(la - lb))) + + +out = {"store": B} + +# ============================ CT-F5 ========================================= +print("\n=== CT-F5 walking-center geopotential sweep (fixes F1 saturation) ===") +T0 = 91246 +lat0_p = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0].astype(np.float64) +lat1_p = fetch("mean_sea_level_pressure", f"{T0+1}.0.0")[0].astype(np.float64) +levels = fetch("level", "0").astype(int).ravel() +print("fetching geopotential t0 (13 levels) ...", flush=True) +z0 = fetch("geopotential", f"{T0}.0.0.0")[0].astype(np.float64) +order = np.argsort(-levels) # 1000 hPa first (surface-anchored walk) + +STORMS0 = [] +for nm, hint in (("storm1", None), ("storm2", (67.0, 28.0))): + ci0, cj0 = find_center(lat0_p, near=hint) + ci1, cj1 = find_center(lat1_p, near=(lat[ci0], lon_deg[cj0])) + dx, dy, _, _ = geom_ll(lat[ci0], lon_deg[cj0]) + mv = (float(dx[ci1, cj1]), float(dy[ci1, cj1])) + STORMS0.append({"name": nm, "lat0": lat[ci0], "lon0": lon_deg[cj0], + "motion_rad": float(np.arctan2(mv[1], mv[0]))}) + +f5 = {} +for st in STORMS0: + mth = st["motion_rad"] + cur = (st["lat0"], st["lon0"]) + walk = [] + max_step = 0.0 + for li in order: + lev = int(levels[li]) + found = find_center(z0[li], near=cur, radius_km=250.0) + if found is None: + walk.append({"level_hPa": lev, "found": False}) + continue + fi, fj = found + step = sep_km(cur, (lat[fi], lon_deg[fj])) + max_step = max(max_step, step) + cur = (lat[fi], lon_deg[fj]) + d = decompose_ll(z0[li], cur[0], cur[1]) + walk.append({"level_hPa": lev, "found": True, + "center_lat": cur[0], "center_lon": cur[1], + "step_km": step, + "error_deg": err_deg(d["low_pole_rad"], mth), + "wn1_frac": d["wn1_frac"]}) + valid = [w for w in walk if w["found"]] + errs = {w["level_hPa"]: w["error_deg"] for w in valid} + in_band = {h: e for h, e in errs.items() if 400 <= h <= 850} + if in_band: + best_h = min(in_band, key=lambda h: abs(in_band[h])) + f5a = abs(in_band[best_h]) <= 20.0 + else: + best_h, f5a = None, False + f5b = max_step <= 250.0 + sfc_err = errs.get(1000) + f5[st["name"]] = {"walk": sorted(walk, key=lambda w: -w["level_hPa"]), + "max_step_km": max_step, "CT_F5b_no_saturation": bool(f5b), + "best_level_in_400_850": best_h, + "best_error_in_400_850_deg": in_band.get(best_h) if best_h else None, + "CT_F5a_pass": bool(f5a), "surface_error_deg": sfc_err} + print(f"\n{st['name']}: surface error {sfc_err:+.1f} deg") + for w in sorted(valid, key=lambda w: -w["level_hPa"]): + print(f" {w['level_hPa']:4d} hPa err {w['error_deg']:+7.1f} " + f"step {w['step_km']:5.1f} km wn1 {w['wn1_frac']:.3f}") + unfound = [w["level_hPa"] for w in walk if not w["found"]] + if unfound: + print(f" (no center found at: {unfound} hPa)") + print(f" max single-step jump: {max_step:.1f} km -> " + f"F5b (<=250km, no saturation): {f5b}") + if best_h: + print(f" best |error| in 400-850 band: {best_h} hPa " + f"({in_band[best_h]:+.1f} deg) -> F5a pass: {f5a}") + else: + print(" NO level found in 400-850 band -> F5a: NO-VERDICT") + +out["CT_F5"] = f5 +print("\nCT-F5 net: storm2 own-center path was NO-VERDICT/dead-absurd in the " + "original sweep (saturated 586-599km at 5/13 levels); walking fix " + f"result: F5a={f5['storm2']['CT_F5a_pass']}, " + f"F5b={f5['storm2']['CT_F5b_no_saturation']}") + +# ============================== CT-N ========================================= +print("\n=== CT-N n=10 blind storm sample (binding constraint after F4/F7) ===") +DATES = [ + datetime.datetime(2021, 6, 15, 12), # storm1 anchor (arc-pinned) + datetime.datetime(2020, 1, 10, 12), + datetime.datetime(2020, 7, 20, 12), + datetime.datetime(2019, 3, 5, 12), + datetime.datetime(2019, 10, 25, 12), + datetime.datetime(2014, 9, 12, 12), # was 2022-02-14: out of store bounds + # (see store-bounds guard above) + datetime.datetime(2018, 8, 8, 12), + datetime.datetime(2017, 11, 30, 12), + datetime.datetime(2016, 4, 18, 12), + datetime.datetime(2015, 12, 25, 12), +] + +n_rows = [] +for dt in DATES: + t0 = t_index(dt) + if t0 < 0 or t0 + 1 > _MAX_T: + print(f"\n{dt.date()}: t0={t0} outside store bounds [0,{_MAX_T}] " + "-> excluded (data unavailable)") + n_rows.append({"date": dt.isoformat(), "t0": t0, + "status": "OUT-OF-STORE-BOUNDS"}) + continue + p0 = fetch("mean_sea_level_pressure", f"{t0}.0.0")[0].astype(np.float64) + p1 = fetch("mean_sea_level_pressure", f"{t0+1}.0.0")[0].astype(np.float64) + u10 = fetch("10m_u_component_of_wind", f"{t0}.0.0")[0].astype(np.float64) + v10 = fetch("10m_v_component_of_wind", f"{t0}.0.0")[0].astype(np.float64) + zeta10 = d_dx(v10) - d_dy(u10) + + found0 = find_center(p0) + row = {"date": dt.isoformat(), "t0": t0} + if found0 is None: + row["status"] = "NO-CENTER-FOUND" + n_rows.append(row) + print(f"\n{dt.date()}: no NH low found -> excluded") + continue + ci0, cj0 = found0 + la_a, lo_a = subgrid_min(p0, ci0, cj0) + found1 = find_center(p1, near=(la_a, lo_a)) + row.update({"center_lat": la_a, "center_lon": lo_a, + "anomaly_Pa": float(p0[ci0, cj0] - p0[ci0].mean())}) + if found1 is None: + row["status"] = "NOT-TRACKABLE" + n_rows.append(row) + print(f"\n{dt.date()}: center ({la_a:.2f}N,{lo_a:.2f}E) " + "not trackable at t+6h -> excluded from N1-N5") + continue + ci1, cj1 = found1 + dx, dy, _, _ = geom_ll(la_a, lo_a) + disp = (float(dx[ci1, cj1]), float(dy[ci1, cj1])) + dist = float(np.hypot(*disp)) + mth = float(np.arctan2(disp[1], disp[0])) + trackable = dist >= 100.0 + + dA = decompose_ll(p0, la_a, lo_a) + la_c, lo_c = centroid_ll(zeta10, ci0, cj0) + dC = decompose_ll(p0, la_c, lo_c) + + errA = err_deg(dA["low_pole_rad"], mth) + errC = err_deg(dC["low_pole_rad"], mth) + row.update({ + "status": "OK" if trackable else "SUBTHRESHOLD-DISPLACEMENT", + "displacement_km": dist, "motion_bearing_deg": float(np.rad2deg(mth)), + "CT_E1_wn1_frac": dA["wn1_frac"], "CT_E2_trackable": bool(trackable), + "CT_E3_error_A_mslp_deg": errA, "CT_E4_R2_profile_wn1": dA["R2_profile_wn1"], + "F8_error_C_vort_deg": errC, + "F8_shrinks_at_vort_center": bool(abs(errC) < abs(errA)), + }) + _, _, r_, th_ = geom_ll(la_a, lo_a) + disk = r_ <= R_DISK + lv, rr, tt = lsm[disk].astype(np.float64), r_[disk], th_[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + a1l = np.zeros(nb) + b1l = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + vv, t = lv[m], tt[m] + pr = vv.mean() + a1l[b] = 2 * ((vv - pr) * np.cos(t)).mean() + b1l[b] = 2 * ((vv - pr) * np.sin(t)).mean() + land_dipole_amp = float(np.hypot(a1l, b1l).mean()) + row["F9_land_dipole_amp_mean"] = land_dipole_amp + row["F9_unexplained_residual"] = 1.0 - dA["R2_profile_wn1"] + + n_rows.append(row) + print(f"\n{dt.date()}: center ({la_a:.2f}N,{lo_a:.2f}E), " + f"disp {dist:.0f} km/6h, trackable={trackable}") + print(f" E1 wn1_frac={dA['wn1_frac']:.3f} E4 R2={dA['R2_profile_wn1']:.3f} " + f"E3 error(A)={errA:+.1f} deg F8 error(C)={errC:+.1f} deg " + f"({'shrinks' if abs(errC) < abs(errA) else 'grows'})") + print(f" F9 land_dipole_amp={land_dipole_amp:.4f} " + f"unexplained_residual={row['F9_unexplained_residual']:.3f}") + +valid = [r for r in n_rows if r.get("status") == "OK"] +excluded = [r for r in n_rows if r.get("status") != "OK"] +print(f"\n{len(valid)}/{len(n_rows)} storms valid (CT-E2 trackable); " + f"{len(excluded)} excluded: " + + ", ".join(f"{r['date'][:10]}={r['status']}" for r in excluded)) + +if valid: + errs = np.array([r["CT_E3_error_A_mslp_deg"] for r in valid]) + neg_frac = float((errs < 0).mean()) + n1_pass = neg_frac >= 0.70 + med_abs_err = float(np.median(np.abs(errs))) + iqr = [float(np.percentile(np.abs(errs), 25)), + float(np.percentile(np.abs(errs), 75))] + wn1s = np.array([r["CT_E1_wn1_frac"] for r in valid]) + n3_pass = bool(np.median(wn1s) >= 0.40) + r2s = np.array([r["CT_E4_R2_profile_wn1"] for r in valid]) + n4_pass = bool(np.median(r2s) >= 0.80) + shrink = np.array([r["F8_shrinks_at_vort_center"] for r in valid]) + n5_pass = bool(shrink.mean() >= 0.70) + lda = np.array([r["F9_land_dipole_amp_mean"] for r in valid]) + resid = np.array([r["F9_unexplained_residual"] for r in valid]) + f9_corr = float(np.corrcoef(lda, resid)[0, 1]) if len(valid) >= 3 else None + + print(f"\nCT-N1 sign consistency: {neg_frac:.2f} negative " + f"({int((errs<0).sum())}/{len(valid)}) -> " + f"{'PASS' if n1_pass else 'FAIL'} (bar >= 0.70)") + print(f"CT-N2 magnitude: median|error|={med_abs_err:.1f} deg, " + f"IQR=[{iqr[0]:.1f},{iqr[1]:.1f}] (no bar, observation)") + print(f"CT-N3 wn1 dominance at scale: median={np.median(wn1s):.3f} -> " + f"{'PASS' if n3_pass else 'FAIL'} (bar >= 0.40)") + print(f"CT-N4 R2 at scale: median={np.median(r2s):.3f} -> " + f"{'PASS' if n4_pass else 'FAIL'} (bar >= 0.80)") + print(f"CT-N5/F8 vort-center shrinks error: {shrink.mean():.2f} " + f"({int(shrink.sum())}/{len(valid)}) -> " + f"{'PASS' if n5_pass else 'FAIL'} (bar >= 0.70)") + print(f"CT-F9 corr(land_dipole_amp, unexplained_residual) = " + f"{f9_corr if f9_corr is not None else 'n/a (n<3)'} " + f"-> {'supports candidate 2 residually' if (f9_corr or 0) > 0.2 else 'does not support / inconclusive'}") + + out["CT_N"] = {"n_dates_scanned": len(DATES), "n_valid": len(valid), + "n_excluded": len(excluded), "excluded": excluded, + "rows": valid, + "CT_N1_sign_frac_negative": neg_frac, "CT_N1_pass": n1_pass, + "CT_N2_median_abs_error_deg": med_abs_err, "CT_N2_iqr_deg": iqr, + "CT_N3_median_wn1_frac": float(np.median(wn1s)), "CT_N3_pass": n3_pass, + "CT_N4_median_R2": float(np.median(r2s)), "CT_N4_pass": n4_pass, + "CT_N5_F8_shrink_frac": float(shrink.mean()), "CT_N5_pass": n5_pass, + "CT_F9_corr_land_dipole_vs_residual": f9_corr} +else: + out["CT_N"] = {"n_dates_scanned": len(DATES), "n_valid": 0, + "excluded": excluded, "verdict": "NO-VERDICT: zero valid storms"} + +with open(pathlib.Path(__file__).with_name("comet_tail_f5_n10.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote comet_tail_f5_n10.json") diff --git a/probes/weather-p1/comet_tail_followup.json b/probes/weather-p1/comet_tail_followup.json new file mode 100644 index 00000000..b11b5b7a --- /dev/null +++ b/probes/weather-p1/comet_tail_followup.json @@ -0,0 +1,447 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "t0": 91246, + "t1": 91247, + "R_disk_km": 1200.0, + "storms": {}, + "CT_F3": { + "per_storm": { + "storm1": { + "variants": { + "mslp_min_baseline": { + "lat": 55.75, + "lon": 334.5, + "error_deg": -41.96709521646497, + "wn1_frac": 0.9243451823302548, + "R2_profile_wn1": 0.9722877428704574 + }, + "laplacian_centroid": { + "lat": 55.75, + "lon": 334.5, + "error_deg": -41.96709521646497, + "wn1_frac": 0.9243451823302548, + "R2_profile_wn1": 0.9722877428704574 + }, + "jitter_along_+100km": { + "lat": 56.0, + "lon": 336.0, + "error_deg": -25.364593956108138, + "wn1_frac": 0.9163364290730204, + "R2_profile_wn1": 0.9749750702491462 + }, + "jitter_along_-100km": { + "lat": 55.5, + "lon": 333.0, + "error_deg": -51.689873398133045, + "wn1_frac": 0.9341109139751019, + "R2_profile_wn1": 0.9691441022398409 + }, + "jitter_across_+100km": { + "lat": 56.5, + "lon": 334.0, + "error_deg": -54.783536772466476, + "wn1_frac": 0.8750989688058749, + "R2_profile_wn1": 0.9649272116342785 + }, + "jitter_across_-100km": { + "lat": 55.0, + "lon": 335.0, + "error_deg": -31.916114494021485, + "wn1_frac": 0.9535333835128866, + "R2_profile_wn1": 0.9769806541442484 + } + }, + "spread_deg": 29.41894281635834, + "verdict": "APPARATUS-DOMINATED", + "lap_center_moves_error_toward_zero": false, + "baseline_error_deg": -41.96709521646497, + "lap_error_deg": -41.96709521646497 + }, + "storm2": { + "variants": { + "mslp_min_baseline": { + "lat": 67.0, + "lon": 28.0, + "error_deg": -40.20945429994683, + "wn1_frac": 0.8948035500704673, + "R2_profile_wn1": 0.9259330249471591 + }, + "laplacian_centroid": { + "lat": 67.25, + "lon": 28.25, + "error_deg": -38.637967525479326, + "wn1_frac": 0.8871352743191343, + "R2_profile_wn1": 0.9215053413918612 + }, + "jitter_along_+100km": { + "lat": 67.5, + "lon": 30.0, + "error_deg": -29.80966056672463, + "wn1_frac": 0.8703477881346473, + "R2_profile_wn1": 0.9110418799998037 + }, + "jitter_along_-100km": { + "lat": 66.5, + "lon": 26.0, + "error_deg": -49.25253403863928, + "wn1_frac": 0.9215899818964381, + "R2_profile_wn1": 0.9380158764036869 + }, + "jitter_across_+100km": { + "lat": 67.75, + "lon": 26.75, + "error_deg": -45.37560695885452, + "wn1_frac": 0.8937571510598974, + "R2_profile_wn1": 0.9212396751703932 + }, + "jitter_across_-100km": { + "lat": 66.25, + "lon": 29.25, + "error_deg": -35.55708055436722, + "wn1_frac": 0.9045025287039017, + "R2_profile_wn1": 0.9301238831576134 + } + }, + "spread_deg": 19.442873471914652, + "verdict": "SURVIVES-WITH-UNCERTAINTY", + "lap_center_moves_error_toward_zero": true, + "baseline_error_deg": -40.20945429994683, + "lap_error_deg": -38.637967525479326 + } + }, + "worst_spread_deg": 29.41894281635834, + "gate_passed": false + }, + "CT_F1": { + "storm1": { + "per_level": [ + { + "level_hPa": 50, + "center_lat": 59.0, + "center_lon": 327.0, + "center_offset_km": 592.3638037016171, + "center_search_saturated": true, + "error_own_center_deg": 61.35605283041153, + "error_sfc_center_deg": 58.19216107152542, + "wn1_frac": 0.9670323651903189, + "R2_profile_wn1": 0.9704188919492365 + }, + { + "level_hPa": 100, + "center_lat": 59.25, + "center_lon": 327.25, + "center_offset_km": 597.760797177986, + "center_search_saturated": true, + "error_own_center_deg": 11.948719885232663, + "error_sfc_center_deg": 16.45329587607327, + "wn1_frac": 0.9772680948447687, + "R2_profile_wn1": 0.9770600020662775 + }, + { + "level_hPa": 150, + "center_lat": 59.25, + "center_lon": 327.25, + "center_offset_km": 597.760797177986, + "center_search_saturated": true, + "error_own_center_deg": 4.298302249996766, + "error_sfc_center_deg": 13.841128668636912, + "wn1_frac": 0.973382087136172, + "R2_profile_wn1": 0.9735298068816332 + }, + { + "level_hPa": 200, + "center_lat": 58.5, + "center_lon": 329.5, + "center_offset_km": 437.5097839917024, + "center_search_saturated": false, + "error_own_center_deg": 6.3267716796407, + "error_sfc_center_deg": 14.459485000911172, + "wn1_frac": 0.9518008032307684, + "R2_profile_wn1": 0.9553512076203367 + }, + { + "level_hPa": 250, + "center_lat": 58.25, + "center_lon": 331.25, + "center_offset_km": 344.4470903343073, + "center_search_saturated": false, + "error_own_center_deg": 8.693684473120186, + "error_sfc_center_deg": 13.4202348071978, + "wn1_frac": 0.9301703153070721, + "R2_profile_wn1": 0.9381962415986352 + }, + { + "level_hPa": 300, + "center_lat": 58.0, + "center_lon": 332.25, + "center_offset_km": 287.09065302253333, + "center_search_saturated": false, + "error_own_center_deg": 7.799928849981455, + "error_sfc_center_deg": 10.855158555384946, + "wn1_frac": 0.91698885957228, + "R2_profile_wn1": 0.9324410594134489 + }, + { + "level_hPa": 400, + "center_lat": 57.0, + "center_lon": 334.25, + "center_offset_km": 139.8714099043376, + "center_search_saturated": false, + "error_own_center_deg": 8.693329705454858, + "error_sfc_center_deg": 7.367701240628207, + "wn1_frac": 0.9351995900375635, + "R2_profile_wn1": 0.9509392805209759 + }, + { + "level_hPa": 500, + "center_lat": 56.5, + "center_lon": 334.0, + "center_offset_km": 89.07312936804145, + "center_search_saturated": false, + "error_own_center_deg": 2.929258589915861, + "error_sfc_center_deg": 3.660354230928192, + "wn1_frac": 0.9397828007331679, + "R2_profile_wn1": 0.9576952295272626 + }, + { + "level_hPa": 600, + "center_lat": 56.0, + "center_lon": 334.25, + "center_offset_km": 31.898961446948423, + "center_search_saturated": false, + "error_own_center_deg": -2.0835685390135836, + "error_sfc_center_deg": -1.2238884564239925, + "wn1_frac": 0.9407521073422084, + "R2_profile_wn1": 0.9622091362585159 + }, + { + "level_hPa": 700, + "center_lat": 55.75, + "center_lon": 334.5, + "center_offset_km": 0.0, + "center_search_saturated": false, + "error_own_center_deg": -8.135863343362843, + "error_sfc_center_deg": -8.135863343362843, + "wn1_frac": 0.9383064883729374, + "R2_profile_wn1": 0.9636452679906369 + }, + { + "level_hPa": 850, + "center_lat": 55.75, + "center_lon": 334.5, + "center_offset_km": 0.0, + "center_search_saturated": false, + "error_own_center_deg": -23.84972881632487, + "error_sfc_center_deg": -23.84972881632487, + "wn1_frac": 0.9313941524789896, + "R2_profile_wn1": 0.9693469709841633 + }, + { + "level_hPa": 925, + "center_lat": 55.75, + "center_lon": 334.5, + "center_offset_km": 0.0, + "center_search_saturated": false, + "error_own_center_deg": -32.49848062992805, + "error_sfc_center_deg": -32.49848062992805, + "wn1_frac": 0.9291294230426438, + "R2_profile_wn1": 0.9715600799103316 + }, + { + "level_hPa": 1000, + "center_lat": 55.75, + "center_lon": 334.5, + "center_offset_km": 0.0, + "center_search_saturated": false, + "error_own_center_deg": -40.46281630581089, + "error_sfc_center_deg": -40.46281630581089, + "wn1_frac": 0.9239743148665512, + "R2_profile_wn1": 0.9719752417055713 + } + ], + "best_level_hPa": 600, + "best_error_deg": -2.0835685390135836, + "sfc_error_deg": -41.96709521646497, + "error_spread_across_levels_deg": 101.81886913622242, + "CT_F1_pass": true, + "CT_F1_dead_flat": false, + "CT_F1_dead_absurd_level": false + }, + "storm2": { + "per_level": [ + { + "level_hPa": 50, + "center_lat": 71.0, + "center_lon": 18.75, + "center_offset_km": 599.4520103083312, + "center_search_saturated": true, + "error_own_center_deg": 51.82054658169051, + "error_sfc_center_deg": 34.67018188925604, + "wn1_frac": 0.901432885819133, + "R2_profile_wn1": 0.9034844192118838 + }, + { + "level_hPa": 100, + "center_lat": 71.0, + "center_lon": 18.75, + "center_offset_km": 599.4520103083312, + "center_search_saturated": true, + "error_own_center_deg": 2.0164030755331908, + "error_sfc_center_deg": -5.240993564940538, + "wn1_frac": 0.9531737013560987, + "R2_profile_wn1": 0.9545025593312747 + }, + { + "level_hPa": 150, + "center_lat": 69.25, + "center_lon": 15.5, + "center_offset_km": 597.948738725999, + "center_search_saturated": true, + "error_own_center_deg": -7.972893067162033, + "error_sfc_center_deg": -4.654494247868968, + "wn1_frac": 0.9584459294666916, + "R2_profile_wn1": 0.9594776301341776 + }, + { + "level_hPa": 200, + "center_lat": 69.25, + "center_lon": 15.5, + "center_offset_km": 597.948738725999, + "center_search_saturated": true, + "error_own_center_deg": -10.674219833183258, + "error_sfc_center_deg": -3.239404372201534, + "wn1_frac": 0.9374814196459246, + "R2_profile_wn1": 0.9421606351558359 + }, + { + "level_hPa": 250, + "center_lat": 68.5, + "center_lon": 14.75, + "center_offset_km": 599.3527221144318, + "center_search_saturated": true, + "error_own_center_deg": -14.807646339441447, + "error_sfc_center_deg": 0.09672334053246345, + "wn1_frac": 0.9215579703496299, + "R2_profile_wn1": 0.9301151613267495 + }, + { + "level_hPa": 300, + "center_lat": 68.0, + "center_lon": 14.5, + "center_offset_km": 596.9858339490942, + "center_search_saturated": true, + "error_own_center_deg": -17.28080857434108, + "error_sfc_center_deg": 2.041052679989008, + "wn1_frac": 0.9097645829979871, + "R2_profile_wn1": 0.9213069581688746 + }, + { + "level_hPa": 400, + "center_lat": 68.0, + "center_lon": 14.75, + "center_offset_km": 586.3175725378001, + "center_search_saturated": true, + "error_own_center_deg": -19.27607397055641, + "error_sfc_center_deg": 1.0003071905722436, + "wn1_frac": 0.9134458751317969, + "R2_profile_wn1": 0.92330971883983 + }, + { + "level_hPa": 500, + "center_lat": 66.75, + "center_lon": 26.75, + "center_offset_km": 61.01027079938362, + "center_search_saturated": false, + "error_own_center_deg": -4.535940468762789, + "error_sfc_center_deg": -2.8101913403015146, + "wn1_frac": 0.979546746038315, + "R2_profile_wn1": 0.9796149846637433 + }, + { + "level_hPa": 600, + "center_lat": 66.75, + "center_lon": 26.75, + "center_offset_km": 61.01027079938362, + "center_search_saturated": false, + "error_own_center_deg": -9.453202593238132, + "error_sfc_center_deg": -7.692808120736544, + "wn1_frac": 0.9732871055886206, + "R2_profile_wn1": 0.9736253816050656 + }, + { + "level_hPa": 700, + "center_lat": 67.0, + "center_lon": 27.0, + "center_offset_km": 43.44731917011045, + "center_search_saturated": false, + "error_own_center_deg": -14.448233406134591, + "error_sfc_center_deg": -12.920053691199456, + "wn1_frac": 0.96208166112389, + "R2_profile_wn1": 0.9653193475155147 + }, + { + "level_hPa": 850, + "center_lat": 67.0, + "center_lon": 27.5, + "center_offset_km": 21.723659585055223, + "center_search_saturated": false, + "error_own_center_deg": -23.661401835548673, + "error_sfc_center_deg": -22.325237272421816, + "wn1_frac": 0.9453390686389784, + "R2_profile_wn1": 0.9538800816334196 + }, + { + "level_hPa": 925, + "center_lat": 67.0, + "center_lon": 27.75, + "center_offset_km": 10.861829792527612, + "center_search_saturated": false, + "error_own_center_deg": -30.747604434365485, + "error_sfc_center_deg": -29.861737215801043, + "wn1_frac": 0.9315082183168824, + "R2_profile_wn1": 0.9464905885127584 + }, + { + "level_hPa": 1000, + "center_lat": 67.0, + "center_lon": 28.0, + "center_offset_km": 0.0, + "center_search_saturated": false, + "error_own_center_deg": -39.746495421917615, + "error_sfc_center_deg": -39.746495421917615, + "wn1_frac": 0.9013658742632329, + "R2_profile_wn1": 0.9312405064761293 + } + ], + "best_level_hPa": 100, + "best_error_deg": 2.0164030755331908, + "sfc_error_deg": -40.20945429994683, + "error_spread_across_levels_deg": 91.56704200360812, + "CT_F1_pass": false, + "CT_F1_dead_flat": false, + "CT_F1_dead_absurd_level": true + } + }, + "CT_F2": { + "storm1": { + "n_points": 6552, + "land_fraction": 0.006215642086970501, + "median_inflow_deg": 14.727268148170722, + "q25_inflow_deg": 10.10705375815943, + "q75_inflow_deg": 18.248227278092536, + "median_inflow_ocean_only_deg": 14.714974202408655, + "n_ocean_points": 6517, + "friction_alone_could_own_40deg": false + }, + "storm2": { + "n_points": 8960, + "land_fraction": 0.4578373873273709, + "median_inflow_deg": 22.032197742463666, + "q25_inflow_deg": 10.51176091663094, + "q75_inflow_deg": 34.7427460800546, + "median_inflow_ocean_only_deg": 12.990653593650848, + "n_ocean_points": 4674, + "friction_alone_could_own_40deg": false + } + } +} \ No newline at end of file diff --git a/probes/weather-p1/comet_tail_followup.py b/probes/weather-p1/comet_tail_followup.py new file mode 100644 index 00000000..e85109b2 --- /dev/null +++ b/probes/weather-p1/comet_tail_followup.py @@ -0,0 +1,431 @@ +"""EXPLORATORY follow-up to comet_tail_probe.py (db57aac0) — CT-F1/F2/F3 as +pre-registered in COMET_TAIL_REPORT.md sec.5. NOT an EV; bars mine, unaudited. + +THE QUESTION. The comet-tail probe found the wn-1 low pole left-of-motion on +2/2 storms, but with a COMMON offset of -42 deg / -40 deg from the naive +geostrophic prediction. Same magnitude, same sense, two storms => a systematic +mechanism, not noise. Three ranked candidates were named; this probe tests all +three, APPARATUS FIRST (a systematic number is a claim about the measurement +apparatus until proven otherwise -- the arc's standing rule). + +geom/find_center/decompose are COPIED VERBATIM from comet_tail_probe.py so +every number below is comparable to the committed baseline. Baseline to beat: +storm1 error -41.97 deg, storm2 error -40.21 deg. + +PRE-REGISTERED, in run order: + +CT-F3 APPARATUS (gates the other two). Recompute the low-pole bearing from 6 + center choices per storm: the baseline MSLP minimum; a Laplacian- + centroid center (grad^2 p is proportional to geostrophic vorticity, so + its centroid is a circulation-center proxy); and +/-100 km jitters + along- and across-track. A 100 km miscentering of a monopole INJECTS a + wn-1 by construction, so this measures how much of the -40 deg is a + choice I made with ~100 km of arbitrariness. + spread <= 10 deg -> ROBUST (offset is not center choice) + 10 < spread <= 20 -> SURVIVES-WITH-UNCERTAINTY + spread > 20 deg -> APPARATUS-DOMINATED: CT-E3's verdict must be + re-graded and F1/F2 are moot. + Directional sub-test: if the Laplacian-centroid center moves the error + TOWARD zero, that is positive evidence the offset was center bias. + +CT-F1 STEERING LEVEL (candidate 1: baroclinic tilt). The store ships all 13 + pressure levels in ONE chunk, so the yes/no becomes a sweep: decompose + geopotential at every level about that level's own low center, and + score each against the SAME surface-measured motion bearing. + PASS the |error|-minimising level lies in 400-850 hPa (the textbook + extratropical steering layer) AND |error| there <= 20 deg. + DEAD the error is FLAT across levels (max-min < 15 deg) -- level + structure then explains nothing -- or the minimising level sits + at 50-100 hPa (physically absurd => apparatus). + Reported twice: about each level's own center, and about the SURFACE + center, to separate "the field changes with height" from "the center + moves with height". + +CT-F2 FRICTION (candidate 2: Ekman turning). Measure the actual 10m cross- + isobar inflow angle: alpha = bearing(v10m) - bearing(v_geostrophic), + positive = CCW = turned toward the low (the friction sign in NH). + Sampled on rings 300-1000 km (excluding the core, where gradient-wind + curvature is largest, and the outer edge), |v10m| > 3 m/s. + Expect median alpha in [5, 40] deg over open ocean (textbook 10-30). + The bar that matters: if median alpha << 40 deg, friction CANNOT own + the -40 deg offset alone and is at most a contributor. If median + alpha >= 40 deg, friction alone remains sufficient and this probe + does NOT separate it from candidate 1. + Land fraction from land_sea_mask is reported so "over ocean" is a + measurement, not an assumption. + +NOT tested here (stated, not hidden): n is still 2 storms at one synoptic +time. No offset constant may be baked into any predictor on this evidence; +the report's n>=10 rule stands regardless of how these three come out. + +RUN LOG (transparency — the bars above are VERBATIM as first run, unedited): + run 1 CT-F3 and CT-F1 completed; CT-F2 crashed on a shape bug in grad_p + (dxk was (NY,1) and was being column-sliced). Fixed. Also ADDED a + DIAGNOSTIC field `center_search_saturated` to CT-F1 after run 1 + exposed that find_center can pin at exactly the 600 km search radius + — i.e. it found no co-located upper center and locked onto a + different system. That is a diagnostic, not a bar: no pass/fail + criterion above was added, removed, or loosened. +""" +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T0, T1 = 91246, 91247 +R_E = 6371.0 +R_DISK = 1200.0 +RING = 100.0 + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=600).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +print("fetching MSLP t0,t1 ...", flush=True) +lat = fetch("latitude", "0").astype(np.float64).ravel() +p0 = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0].astype(np.float64) +p1 = fetch("mean_sea_level_pressure", f"{T1}.0.0")[0].astype(np.float64) +NY, NX = p0.shape +phi = np.deg2rad(lat) +lon_deg = np.arange(NX) * 0.25 + +# ---- verbatim from comet_tail_probe.py ------------------------------------- + + +def geom(ci, cj): + """dx,dy,r (km) and azimuth theta (rad, CCW from east) rel. to center.""" + dlon = np.deg2rad((lon_deg[None, :] - lon_deg[cj] + 180) % 360 - 180) + dphi = phi[:, None] - phi[ci] + dx = R_E * np.cos(phi[ci]) * dlon * np.ones((NY, 1)) + dy = R_E * dphi * np.ones((1, NX)) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def find_center(field, near=None, radius_km=600.0): + """Deepest zonal-anomaly low, globally NH or within radius of `near`.""" + fa = field - field.mean(axis=1, keepdims=True) + mask = lat[:, None] > 15 + if near is not None: + _, _, r, _ = geom(*near) + mask = mask & (r < radius_km) + masked = np.where(mask, fa, np.inf) + ci, cj = np.unravel_index(np.argmin(masked), field.shape) + # An empty mask makes `masked` all-inf and argmin returns index 0, i.e. the + # function would report grid cell (0,0) as a storm centre. A `near`-limited + # search CAN be fully masked, so this must be checked on every path + # (coderabbit on PR #926, 2026-08-11). + if not np.isfinite(masked[ci, cj]): + return None + return int(ci), int(cj) + + +def decompose(field, ci, cj): + """Ring-mean + per-ring wn-1 fit; returns metrics dict.""" + dx, dy, r, th = geom(ci, cj) + disk = r <= R_DISK + vals, rr, tt = field[disk], r[disk], th[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + + prof = np.zeros(nb) + a1 = np.zeros(nb) + b1 = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + v, t = vals[m], tt[m] + prof[b] = v.mean() + c, s = np.cos(t), np.sin(t) + a1[b] = 2 * ((v - prof[b]) * c).mean() + b1[b] = 2 * ((v - prof[b]) * s).mean() + + resid0 = vals - prof[rings] + wn1 = a1[rings] * np.cos(tt) + b1[rings] * np.sin(tt) + resid1 = resid0 - wn1 + + var_t = vals.var() + e1 = 1.0 - np.mean(resid0 ** 2) / var_t + e2 = 1.0 - np.mean(resid1 ** 2) / var_t + wn1_frac = wn1.var() / resid0.var() + + amp = np.hypot(a1, b1) + w = amp * np.arange(nb) + ph = np.arctan2(np.sum(b1 * w), np.sum(a1 * w)) + low_pole = (ph + np.pi) % (2 * np.pi) + r_mid = (np.arange(nb) + 0.5) * RING + a_corr = float(np.corrcoef(amp[1:], r_mid[1:])[0, 1]) + return {"R2_profile": float(e1), "R2_profile_wn1": float(e2), + "wn1_frac_of_resid": float(wn1_frac), + "low_pole_bearing_rad": float(low_pole), + "amp_vs_r_corr": a_corr} + +# ---- helpers new to this probe --------------------------------------------- + + +def wrap_deg(d): + """Wrap degrees into [-180, 180). + + The half-open end is LOW, not high: at exactly +180 this returns -180. + (Docstring said "(-180, 180]" until 2026-08-11 — CodeRabbit, PR #926 — + which was wrong about the code, not a bug in it.) The boundary is not + cosmetic: `stratum_verdict` scores offset > 0 as low-pole-left-of-motion, + so a value landing exactly on the boundary counts NEGATIVE. That is the + physically right call — +/-180 means the dipole points exactly OPPOSITE + the motion, which is not left-of-motion under either spelling — so the + convention is kept, now stated. Audited across every committed result + JSON: 283 angle-like values, 0 boundary hits, closest 0.91 deg. + """ + return (d + 180.0) % 360.0 - 180.0 + + +def err_deg(low_pole_rad, motion_rad): + """Signed alignment error, in degrees, of a low-pole bearing against the left-of-motion prediction.""" + pred = (motion_rad + np.pi / 2) % (2 * np.pi) + return float(wrap_deg(np.rad2deg(low_pole_rad - pred))) + + +def offset_center(ci, cj, dx_km, dy_km): + """Grid point nearest the (dx,dy) km offset from (ci,cj).""" + dx, dy, _, _ = geom(ci, cj) + d2 = (dx - dx_km) ** 2 + (dy - dy_km) ** 2 + i, j = np.unravel_index(np.argmin(d2), d2.shape) + return int(i), int(j) + + +def lap_centroid(field, ci, cj, radius_km=300.0): + """Centroid of grad^2 p (proportional to geostrophic vorticity) near the + center — a circulation-center proxy independent of the depth minimum.""" + lap = np.zeros_like(field) + lap[1:-1, :] += field[2:, :] + field[:-2, :] - 2 * field[1:-1, :] + lap[:, 1:-1] += field[:, 2:] + field[:, :-2] - 2 * field[:, 1:-1] + dx, dy, r, _ = geom(ci, cj) + m = (r <= radius_km) & (lap > 0) + if not m.any(): + return ci, cj + w = lap[m] - 0.5 * lap[m].max() + w = np.where(w > 0, w, 0.0) + if w.sum() <= 0: + return ci, cj + return offset_center(ci, cj, + float((dx[m] * w).sum() / w.sum()), + float((dy[m] * w).sum() / w.sum())) + + +def grad_p(field): + """d p/dx, d p/dy in Pa/km on the sphere (centered differences).""" + dy_km = R_E * np.deg2rad(0.25) + gy = np.zeros_like(field) + gy[1:-1, :] = (field[2:, :] - field[:-2, :]) / (2 * dy_km) + gy = -gy # index increases southward => flip sign + gx = np.zeros_like(field) + dxk = R_E * np.cos(phi)[:, None] * np.deg2rad(0.25) # (NY,1), broadcasts + gx[:, 1:-1] = (field[:, 2:] - field[:, :-2]) / (2 * dxk) + return gx, gy + + +def static_key(var): + """Chunk key for a variable, all-zero index of the right arity.""" + return ".".join("0" * len(meta[f"{var}/.zarray"]["chunks"])) + + +# ---- storms: reproduce the baseline track ---------------------------------- +STORMS = [] +for name, hint in (("storm1", None), ("storm2", (int(round((90 - 67.0) / 0.25)), + int(round(28.0 / 0.25))))): + ci0, cj0 = find_center(p0, near=hint) + ci1, cj1 = find_center(p1, near=(ci0, cj0)) + dx, dy, _, _ = geom(ci0, cj0) + mv = np.array([dx[ci1, cj1], dy[ci1, cj1]]) + STORMS.append({"name": name, "c0": (ci0, cj0), + "motion_rad": float(np.arctan2(mv[1], mv[0])), + "disp_km": float(np.hypot(*mv))}) + +# STORMS carries the centre, motion bearing and displacement each per-storm +# CT-F1/F2/F3 record is computed from; writing `{}` dropped that provenance +# (coderabbit on PR #926, 2026-08-11). +out = { + "store": B, "t0": T0, "t1": T1, "R_disk_km": R_DISK, + "storms": { + st["name"]: { + "center_t0": {"lat": float(lat[st["c0"][0]]), + "lon": float(lon_deg[st["c0"][1]])}, + "motion_bearing_deg": float(np.rad2deg(st["motion_rad"])), + "displacement_km": st["disp_km"], + } + for st in STORMS + }, +} + +# ============================ CT-F3 : APPARATUS ============================= +print("\n=== CT-F3 APPARATUS (center sensitivity) — runs first, gates F1/F2 ===") +f3_all = {} +for st in STORMS: + ci0, cj0 = st["c0"] + mth = st["motion_rad"] + al = np.array([np.cos(mth), np.sin(mth)]) * 100.0 # along-track 100 km + ac = np.array([-np.sin(mth), np.cos(mth)]) * 100.0 # across-track 100 km + variants = { + "mslp_min_baseline": (ci0, cj0), + "laplacian_centroid": lap_centroid(p0, ci0, cj0), + "jitter_along_+100km": offset_center(ci0, cj0, *al), + "jitter_along_-100km": offset_center(ci0, cj0, *(-al)), + "jitter_across_+100km": offset_center(ci0, cj0, *ac), + "jitter_across_-100km": offset_center(ci0, cj0, *(-ac)), + } + rows = {} + for k, (i, j) in variants.items(): + d = decompose(p0, i, j) + rows[k] = {"lat": float(lat[i]), "lon": float(lon_deg[j]), + "error_deg": err_deg(d["low_pole_bearing_rad"], mth), + "wn1_frac": d["wn1_frac_of_resid"], + "R2_profile_wn1": d["R2_profile_wn1"]} + errs = np.array([r["error_deg"] for r in rows.values()]) + # circular spread about the baseline + rel = wrap_deg(errs - rows["mslp_min_baseline"]["error_deg"]) + spread = float(rel.max() - rel.min()) + verdict = ("ROBUST" if spread <= 10 else + "SURVIVES-WITH-UNCERTAINTY" if spread <= 20 else + "APPARATUS-DOMINATED") + base_e = rows["mslp_min_baseline"]["error_deg"] + lap_e = rows["laplacian_centroid"]["error_deg"] + toward_zero = abs(lap_e) < abs(base_e) + f3_all[st["name"]] = {"variants": rows, "spread_deg": spread, + "verdict": verdict, + "lap_center_moves_error_toward_zero": bool(toward_zero), + "baseline_error_deg": base_e, + "lap_error_deg": lap_e} + print(f"\n{st['name']}: baseline error {base_e:+.1f} deg") + for k, r in rows.items(): + print(f" {k:22s} ({r['lat']:.2f}N,{r['lon']:.2f}E) " + f"error {r['error_deg']:+7.1f} wn1_frac {r['wn1_frac']:.3f}") + print(f" spread across 6 centers = {spread:.1f} deg -> {verdict}") + print(f" Laplacian-centroid moves error toward zero: {toward_zero}") + +f3_worst = max(v["spread_deg"] for v in f3_all.values()) +f3_gate = f3_worst <= 20.0 +out["CT_F3"] = {"per_storm": f3_all, "worst_spread_deg": f3_worst, + "gate_passed": bool(f3_gate)} +print(f"\nCT-F3 GATE: worst spread {f3_worst:.1f} deg -> " + f"{'PASS (F1/F2 interpretable)' if f3_gate else 'FAIL (F1/F2 moot; CT-E3 must be re-graded)'}") + +# ========================= CT-F1 : STEERING LEVEL =========================== +print("\n=== CT-F1 STEERING LEVEL (geopotential sweep, 13 levels) ===") +levels = fetch("level", "0").astype(int).ravel() +print(f"fetching geopotential t0 (13 levels, one chunk) ...", flush=True) +z0 = fetch("geopotential", f"{T0}.0.0.0")[0].astype(np.float64) # (13,ny,nx) + +f1_all = {} +for st in STORMS: + ci0, cj0 = st["c0"] + mth = st["motion_rad"] + per_level = [] + for li, lev in enumerate(levels): + fld = z0[li] + zi, zj = find_center(fld, near=(ci0, cj0)) + d_own = decompose(fld, zi, zj) + d_sfc = decompose(fld, ci0, cj0) + _, _, r_, _ = geom(ci0, cj0) + per_level.append({ + "level_hPa": int(lev), + "center_lat": float(lat[zi]), "center_lon": float(lon_deg[zj]), + "center_offset_km": float(r_[zi, zj]), + # DIAGNOSTIC (added after run 1, not a bar): the finder pinned at + # the 600 km search radius => no co-located upper center found. + "center_search_saturated": bool(r_[zi, zj] > 580.0), + "error_own_center_deg": err_deg(d_own["low_pole_bearing_rad"], mth), + "error_sfc_center_deg": err_deg(d_sfc["low_pole_bearing_rad"], mth), + "wn1_frac": d_own["wn1_frac_of_resid"], + "R2_profile_wn1": d_own["R2_profile_wn1"], + }) + errs = np.array([p["error_own_center_deg"] for p in per_level]) + best = int(np.argmin(np.abs(errs))) + flat = float(errs.max() - errs.min()) + best_lev = int(levels[best]) + passed = (400 <= best_lev <= 850) and abs(errs[best]) <= 20.0 + dead_flat = flat < 15.0 + dead_absurd = best_lev <= 100 + f1_all[st["name"]] = { + "per_level": per_level, "best_level_hPa": best_lev, + "best_error_deg": float(errs[best]), + "sfc_error_deg": float(err_deg( + decompose(p0, ci0, cj0)["low_pole_bearing_rad"], mth)), + "error_spread_across_levels_deg": flat, + "CT_F1_pass": bool(passed), + "CT_F1_dead_flat": bool(dead_flat), + "CT_F1_dead_absurd_level": bool(dead_absurd), + } + print(f"\n{st['name']}: (surface MSLP error " + f"{f1_all[st['name']]['sfc_error_deg']:+.1f} deg)") + for p in per_level: + print(f" {p['level_hPa']:4d} hPa err(own ctr) {p['error_own_center_deg']:+7.1f} " + f"err(sfc ctr) {p['error_sfc_center_deg']:+7.1f} " + f"ctr offset {p['center_offset_km']:5.0f} km " + f"wn1 {p['wn1_frac']:.3f}") + print(f" |error| minimised at {best_lev} hPa ({errs[best]:+.1f} deg); " + f"spread across levels {flat:.1f} deg") + print(f" CT-F1 pass(400-850 hPa & |err|<=20): {passed} | " + f"dead-flat(<15 deg spread): {dead_flat} | " + f"dead-absurd(<=100 hPa): {dead_absurd}") +out["CT_F1"] = f1_all + +# ============================ CT-F2 : FRICTION ============================== +print("\n=== CT-F2 FRICTION (10m cross-isobar inflow angle) ===") +u10 = fetch("10m_u_component_of_wind", f"{T0}.0.0")[0].astype(np.float64) +v10 = fetch("10m_v_component_of_wind", f"{T0}.0.0")[0].astype(np.float64) +lsm = fetch("land_sea_mask", static_key("land_sea_mask")).astype(np.float64) +lsm = lsm.reshape(NY, NX) if lsm.size == NY * NX else lsm[0] +gx, gy = grad_p(p0) + +f2_all = {} +for st in STORMS: + ci0, cj0 = st["c0"] + _, _, r, _ = geom(ci0, cj0) + band = (r >= 300.0) & (r <= 1000.0) + spd = np.hypot(u10, v10) + m = band & (spd > 3.0) + # geostrophic direction: v_g ∝ (-dp/dy, dp/dx) + bg = np.arctan2(gx[m], -gy[m]) + ba = np.arctan2(v10[m], u10[m]) + alpha = wrap_deg(np.rad2deg(ba - bg)) # + = CCW = turned toward the low + land = float(lsm[m].mean()) + ocean = alpha[lsm[m] < 0.5] + med = float(np.median(alpha)) + f2_all[st["name"]] = { + "n_points": int(m.sum()), "land_fraction": land, + "median_inflow_deg": med, + "q25_inflow_deg": float(np.percentile(alpha, 25)), + "q75_inflow_deg": float(np.percentile(alpha, 75)), + "median_inflow_ocean_only_deg": + float(np.median(ocean)) if ocean.size else None, + "n_ocean_points": int(ocean.size), + "friction_alone_could_own_40deg": bool(abs(med) >= 40.0), + } + o = f2_all[st['name']]['median_inflow_ocean_only_deg'] + print(f"\n{st['name']}: n={int(m.sum())} pts in 300-1000 km, " + f"land fraction {land:.2f}") + print(f" inflow angle median {med:+.1f} deg " + f"(IQR {f2_all[st['name']]['q25_inflow_deg']:+.1f} .. " + f"{f2_all[st['name']]['q75_inflow_deg']:+.1f})") + print(f" ocean-only median: " + f"{('%+.1f deg' % o) if o is not None else 'n/a'} " + f"(n={f2_all[st['name']]['n_ocean_points']})") + print(f" friction alone could own the -40 deg offset: " + f"{f2_all[st['name']]['friction_alone_could_own_40deg']}") +out["CT_F2"] = f2_all + +with open(pathlib.Path(__file__).with_name("comet_tail_followup.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote comet_tail_followup.json") diff --git a/probes/weather-p1/comet_tail_probe.json b/probes/weather-p1/comet_tail_probe.json new file mode 100644 index 00000000..fb58d3cf --- /dev/null +++ b/probes/weather-p1/comet_tail_probe.json @@ -0,0 +1,67 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "t0": 91246, + "t1": 91247, + "R_disk_km": 1200.0, + "storm1": { + "center_t0": { + "lat": 55.75, + "lon": 334.5 + }, + "center_t1": { + "lat": 56.5, + "lon": 338.75 + }, + "displacement_km": 278.73766575720873, + "motion_bearing_deg": 17.40910408001618, + "low_pole_bearing_deg": 65.44200886355121, + "predicted_low_pole_deg": 107.40910408001618, + "alignment_error_deg": -41.96709521646496, + "R2_profile": 0.6348125006744301, + "R2_profile_wn1": 0.9722877428704574, + "R2_profile_wn1_constrained_2param": 0.9434382813469203, + "n_params_profile": 12, + "n_params_profile_wn1_perring": 36, + "n_params_profile_wn1_constrained": 14, + "constrained_slope_pa_per_km": 0.9408726353949788, + "constrained_bearing_deg": -116.09993446761843, + "wn1_frac_of_resid": 0.9243451823302548, + "low_pole_bearing_rad": 1.142178523788281, + "amp_vs_r_corr": 0.8002995480605919, + "CT_E1_wn1_frac": 0.9243451823302548, + "CT_E2_trackable": true, + "CT_E3_left_of_motion_within_45": true, + "CT_E4_R2_profile_wn1": 0.9722877428704574 + }, + "storm2": { + "center_t0": { + "lat": 67.0, + "lon": 28.0 + }, + "center_t1": { + "lat": 69.0, + "lon": 36.75 + }, + "displacement_km": 440.43381595559754, + "motion_bearing_deg": 30.32694785348875, + "low_pole_bearing_deg": 80.11749355354193, + "predicted_low_pole_deg": 120.32694785348876, + "alignment_error_deg": -40.209454299946835, + "R2_profile": 0.29427821878043336, + "R2_profile_wn1": 0.9259330249471591, + "R2_profile_wn1_constrained_2param": 0.909049958041129, + "n_params_profile": 12, + "n_params_profile_wn1_perring": 36, + "n_params_profile_wn1_constrained": 14, + "constrained_slope_pa_per_km": 1.0442717052955404, + "constrained_bearing_deg": -99.32825625591818, + "wn1_frac_of_resid": 0.8948035500704673, + "low_pole_bearing_rad": 1.3983140509546386, + "amp_vs_r_corr": 0.9977355074833578, + "CT_E1_wn1_frac": 0.8948035500704673, + "CT_E2_trackable": true, + "CT_E3_left_of_motion_within_45": true, + "CT_E4_R2_profile_wn1": 0.9259330249471591 + }, + "CT_E3_joint_hits": 2 +} \ No newline at end of file diff --git a/probes/weather-p1/comet_tail_probe.py b/probes/weather-p1/comet_tail_probe.py new file mode 100644 index 00000000..003cdecb --- /dev/null +++ b/probes/weather-p1/comet_tail_probe.py @@ -0,0 +1,213 @@ +"""EXPLORATORY — 'die fehlende Achsensymmetrie mit Flugzeug-Formel x Traegheit +berechnen, wie beim Kometen dessen Schweif nach hinten drueckt' (operator, +2026-08-11). NOT an EV; bars mine, unaudited. + +The physics is textbook, not invented: a translating vortex = a vortex in a +steering flow (the airplane's relative wind). Geostrophy makes it SIGNED and +falsifiable: if the storm moves WITH the geostrophic steering flow, the +background pressure gradient is PERPENDICULAR to the motion with the LOW pole +to the LEFT of motion (NH). A linear background gradient has zero ring-mean, +so it survives ring-profile removal ENTIRELY as a wavenumber-1 residual with +amplitude growing ~linearly in r. The comet tail IS wavenumber-1. + +PRE-REGISTERED (per storm; two storms = replication, n=2 smallness stated): + CT-E1 wavenumber-1 carries >= 0.40 of the azimuthal-residual variance in + the disk (R=1200 km) — the tail is the DOMINANT asymmetry mode. + CT-E2 motion trackable: displacement t -> t+6h >= 100 km within a 600 km + search radius, else NO-VERDICT (direction undefined below that). + CT-E3 THE SIGNED TEST: bearing(wn-1 low pole) = bearing(motion) + 90 deg + (CCW, x=east/y=north) within +/-45 deg. Null probability 0.25 per + storm; 2/2 hits = 0.0625 under null — reported as exactly that. + CT-E4 ring-profile + per-ring wn-1 explains >= 0.80 of in-disk variance + (single-storm baseline was 0.639 profile-only). + CT-E5 bonus observation (no bar): wn-1 amplitude a1(r) approx linear in r + (the linear-background signature); report corr(a1, r). + +Same store as the arc (WB2 ERA5 6h). t=91246 -> 91247 (2021-06-15 12Z->18Z). +Storm 1: the arc's storm (55.75N 334.5E). Storm 2: the #3 center from +go_territory_probe (67.0N 28.0E) as replication. +""" +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T0, T1 = 91246, 91247 +R_E = 6371.0 +R_DISK = 1200.0 +RING = 100.0 + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=180).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +lat = fetch("latitude", "0").astype(np.float64).ravel() +p0 = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0].astype(np.float64) +p1 = fetch("mean_sea_level_pressure", f"{T1}.0.0")[0].astype(np.float64) +phi = np.deg2rad(lat) +lon_deg = np.arange(p0.shape[1]) * 0.25 + + +def geom(ci, cj): + """dx,dy,r (km) and azimuth theta (rad, CCW from east) rel. to center.""" + dlon = np.deg2rad((lon_deg[None, :] - lon_deg[cj] + 180) % 360 - 180) + dphi = phi[:, None] - phi[ci] + dx = R_E * np.cos(phi[ci]) * dlon * np.ones((p0.shape[0], 1)) + dy = R_E * dphi * np.ones((1, p0.shape[1])) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def find_center(field, near=None, radius_km=600.0): + """Deepest zonal-anomaly low, globally NH or within radius of `near`.""" + fa = field - field.mean(axis=1, keepdims=True) + mask = lat[:, None] > 15 + if near is not None: + _, _, r, _ = geom(*near) + mask = mask & (r < radius_km) + masked = np.where(mask, fa, np.inf) + ci, cj = np.unravel_index(np.argmin(masked), field.shape) + # An empty mask makes `masked` all-inf and argmin returns index 0, i.e. the + # function would report grid cell (0,0) as a storm centre. A `near`-limited + # search CAN be fully masked, so this must be checked on every path + # (coderabbit on PR #926, 2026-08-11). + if not np.isfinite(masked[ci, cj]): + return None + return int(ci), int(cj) + + +def decompose(field, ci, cj): + """Ring-mean + per-ring wn-1 fit; returns metrics dict.""" + dx, dy, r, th = geom(ci, cj) + disk = r <= R_DISK + vals, rr, tt = field[disk], r[disk], th[disk] + rings = np.clip((rr / RING).astype(int), 0, int(R_DISK / RING) - 1) + nb = int(R_DISK / RING) + + prof = np.zeros(nb) + a1 = np.zeros(nb) + b1 = np.zeros(nb) + for b in range(nb): + m = rings == b + if not m.any(): + continue + v, t = vals[m], tt[m] + prof[b] = v.mean() + c, s = np.cos(t), np.sin(t) + # least-squares wn-1 on the ring (cos/sin nearly orthogonal on a ring) + a1[b] = 2 * ((v - prof[b]) * c).mean() + b1[b] = 2 * ((v - prof[b]) * s).mean() + + resid0 = vals - prof[rings] # after profile + wn1 = a1[rings] * np.cos(tt) + b1[rings] * np.sin(tt) + resid1 = resid0 - wn1 # after profile + wn-1 + + # CONSTRAINED dipole — the model the report's storage claim actually + # describes: ONE amplitude slope + ONE bearing, i.e. a1(r) = s*r*cos(t0), + # b1(r) = s*r*sin(t0), which is the linear-background signature from §2. + # The per-ring fit above has 2*nb = 24 free parameters and is NOT a + # 2-value representation; conflating the two overstated the compression + # (codex P1 on PR #926, 2026-08-11). Both are now reported. + X = np.column_stack([rr * np.cos(tt), rr * np.sin(tt)]) + coef, *_ = np.linalg.lstsq(X, resid0, rcond=None) + wn1_con = X @ coef + + var_t = vals.var() + e1 = 1.0 - np.mean(resid0 ** 2) / var_t # profile-only R2 + e2 = 1.0 - np.mean(resid1 ** 2) / var_t # profile + per-ring wn-1 + e2c = 1.0 - np.mean((resid0 - wn1_con) ** 2) / var_t # profile + 2-param dipole + wn1_frac = wn1.var() / resid0.var() + + # amplitude-weighted dipole phase: bearing of the LOW pole + amp = np.hypot(a1, b1) + w = amp * np.arange(nb) # outer rings weightier + ph = np.arctan2(np.sum(b1 * w), np.sum(a1 * w)) # HIGH pole bearing + low_pole = (ph + np.pi) % (2 * np.pi) + r_mid = (np.arange(nb) + 0.5) * RING + a_corr = float(np.corrcoef(amp[1:], r_mid[1:])[0, 1]) + return {"R2_profile": float(e1), "R2_profile_wn1": float(e2), + "R2_profile_wn1_constrained_2param": float(e2c), + "n_params_profile": int(nb), + "n_params_profile_wn1_perring": int(nb + 2 * nb), + "n_params_profile_wn1_constrained": int(nb + 2), + "constrained_slope_pa_per_km": float(np.hypot(*coef)), + "constrained_bearing_deg": float(np.rad2deg(np.arctan2(coef[1], coef[0]))), + "wn1_frac_of_resid": float(wn1_frac), + "low_pole_bearing_rad": float(low_pole), + "amp_vs_r_corr": a_corr} + + +def track(name, c0_hint=None): + """Track one storm t -> t+6h and score its wn-1 dipole against the motion bearing.""" + ci0, cj0 = find_center(p0, near=c0_hint) + ci1, cj1 = find_center(p1, near=(ci0, cj0)) + dx, dy, _, _ = geom(ci0, cj0) + disp = np.array([dx[ci1, cj1], dy[ci1, cj1]]) + dist = float(np.hypot(*disp)) + motion_bearing = float(np.arctan2(disp[1], disp[0])) + d = decompose(p0, ci0, cj0) + + predicted = (motion_bearing + np.pi / 2) % (2 * np.pi) # left of motion + diff = (d["low_pole_bearing_rad"] - predicted + np.pi) % (2 * np.pi) - np.pi + res = { + "center_t0": {"lat": float(lat[ci0]), "lon": float(lon_deg[cj0])}, + "center_t1": {"lat": float(lat[ci1]), "lon": float(lon_deg[cj1])}, + "displacement_km": dist, + "motion_bearing_deg": float(np.rad2deg(motion_bearing)), + "low_pole_bearing_deg": float(np.rad2deg(d["low_pole_bearing_rad"])), + "predicted_low_pole_deg": float(np.rad2deg(predicted)), + "alignment_error_deg": float(np.rad2deg(diff)), + **d, + "CT_E1_wn1_frac": d["wn1_frac_of_resid"], + "CT_E2_trackable": dist >= 100.0, + "CT_E3_left_of_motion_within_45": bool(abs(np.rad2deg(diff)) <= 45), + "CT_E4_R2_profile_wn1": d["R2_profile_wn1"], + } + print(f"\n{name}: center ({lat[ci0]:.2f}N, {lon_deg[cj0]:.2f}E) " + f"-> ({lat[ci1]:.2f}N, {lon_deg[cj1]:.2f}E), {dist:.0f} km/6h " + f"(bearing {np.rad2deg(motion_bearing):.0f} deg CCW-from-east)") + print(f" CT-E1 wn-1 share of residual: {d['wn1_frac_of_resid']:.3f} " + f"(bar >= 0.40)") + print(f" CT-E2 trackable: {res['CT_E2_trackable']} ({dist:.0f} km)") + print(f" CT-E3 low pole {res['low_pole_bearing_deg']:.0f} deg vs " + f"predicted (motion+90) {res['predicted_low_pole_deg']:.0f} deg " + f"-> error {res['alignment_error_deg']:+.0f} deg " + f"({'HIT' if res['CT_E3_left_of_motion_within_45'] else 'MISS'}, " + f"bar +/-45)") + print(f" CT-E4 R2 profile-only {d['R2_profile']:.3f} -> " + f"profile+wn1 {d['R2_profile_wn1']:.3f} (bar >= 0.80)") + print(f" [param count: {d['n_params_profile_wn1_perring']} per-ring " + f"vs {d['n_params_profile_wn1_constrained']} constrained]") + print(f" CONSTRAINED 2-param dipole (the storage claim's actual " + f"model): R2 {d['R2_profile_wn1_constrained_2param']:.3f}") + print(f" CT-E5 corr(a1(r), r) = {d['amp_vs_r_corr']:.3f} " + f"(linear-background signature; observation, no bar)") + return res + + +print("== comet-tail probe: wavenumber-1 asymmetry vs storm motion ==") +s1 = track("STORM 1 (the arc's storm)") +# storm 2 hint: 67.0N 28.0E from go_territory_probe +hint = (int(round((90 - 67.0) / 0.25)), int(round(28.0 / 0.25))) +s2 = track("STORM 2 (replication)", c0_hint=hint) + +hits = sum(s["CT_E3_left_of_motion_within_45"] for s in (s1, s2)) +print(f"\nCT-E3 joint: {hits}/2 within +/-45 deg of left-of-motion " + f"(null prob per storm 0.25; 2/2 -> p=0.0625 — n=2, stated, " + f"not overclaimed)") + +json.dump({"store": B, "t0": T0, "t1": T1, "R_disk_km": R_DISK, + "storm1": s1, "storm2": s2, "CT_E3_joint_hits": int(hits)}, + open(pathlib.Path(__file__).with_name("comet_tail_probe.json"), "w"), indent=2) +print("wrote comet_tail_probe.json") diff --git a/probes/weather-p1/ev10_winter.py b/probes/weather-p1/ev10_winter.py index 25a89c5c..a22c10eb 100644 --- a/probes/weather-p1/ev10_winter.py +++ b/probes/weather-p1/ev10_winter.py @@ -35,6 +35,7 @@ ratio comes out above 2. """ import json +import pathlib import numpy as np @@ -132,6 +133,7 @@ def closed_form_flip_point(idx, edges, mass=FLIP_MASS): def main(): + """Run the winter-timestep arm and report the ratios plus the stale-fixture guard.""" result = { "variable": VAR, "run": "A (season/timestep only, variable held fixed at 2m_temperature)", @@ -198,7 +200,8 @@ def main(): result["NO_VERDICT"] = True result["no_verdict_reason"] = no_verdict_reason print(f"\nNO-VERDICT: {no_verdict_reason}") - json.dump(result, open("ev10_winter.json", "w"), indent=2) + with open(pathlib.Path(__file__).with_name("ev10_winter.json"), "w") as fh: + json.dump(result, fh, indent=2) raise SystemExit(1) # ---- per-season flip-points (both arms) ---- @@ -258,7 +261,8 @@ def main(): result["season_comparison"] = comparisons result["NO_VERDICT"] = False - json.dump(result, open("ev10_winter.json", "w"), indent=2) + with open(pathlib.Path(__file__).with_name("ev10_winter.json"), "w") as fh: + json.dump(result, fh, indent=2) print("\nwrote ev10_winter.json") diff --git a/probes/weather-p1/ev3_flip_points.py b/probes/weather-p1/ev3_flip_points.py index 01a94dba..0a382269 100644 --- a/probes/weather-p1/ev3_flip_points.py +++ b/probes/weather-p1/ev3_flip_points.py @@ -29,6 +29,7 @@ file); parameterized here over VARS instead of hardcoded to one variable. """ import json +import pathlib import numpy as np @@ -133,6 +134,7 @@ def closed_form_flip_point(idx, edges, mass=FLIP_MASS): def main(): + """Compute the closed-form flip points and report which bucket carries each crossing.""" out = {} for var in VARS: print(f"\n{var}") @@ -171,7 +173,8 @@ def main(): f"(two-regime expectation applies HERE, not to linear)" ) - json.dump(out, open("ev3_flip_points.json", "w"), indent=2) + with open(pathlib.Path(__file__).with_name("ev3_flip_points.json"), "w") as fh: + json.dump(out, fh, indent=2) print("\nwrote ev3_flip_points.json") diff --git a/probes/weather-p1/ev4_apparatus_sensitivity.py b/probes/weather-p1/ev4_apparatus_sensitivity.py index 2b3c0a87..df9ddfe8 100644 --- a/probes/weather-p1/ev4_apparatus_sensitivity.py +++ b/probes/weather-p1/ev4_apparatus_sensitivity.py @@ -12,6 +12,7 @@ Verdict is printed, not assumed. Emits ev4_apparatus_sensitivity.json. """ import json +import pathlib import numpy as np @@ -39,6 +40,7 @@ def ci_med(anom, w, eps, method): def shape(vals): + """Classify a sequence as increasing / decreasing / non-monotone — the shape verdict EV-4 reports.""" if all(x < y for x, y in zip(vals, vals[1:])): return "increasing" if all(x > y for x, y in zip(vals, vals[1:])): @@ -47,6 +49,7 @@ def shape(vals): def main(): + """Run the eps sweep and the method-spread comparison, printing the apparatus verdict.""" a = np.load(f"fixture/{VAR}.npy").astype(np.float64) anom = a - a.mean(axis=1, keepdims=True) out = {"variable": VAR, "windows": [list(w) for w in WINDOWS]} @@ -105,7 +108,8 @@ def main(): print(f" method max/min spread: {method_spread:.1f}x") print(f" scale-controlling tail flips across windows: {flips}") print(f"\n {out['verdict']['conclusion']}") - json.dump(out, open("ev4_apparatus_sensitivity.json", "w"), indent=2) + with open(pathlib.Path(__file__).with_name("ev4_apparatus_sensitivity.json"), "w") as fh: + json.dump(out, fh, indent=2) if __name__ == "__main__": diff --git a/probes/weather-p1/ev4_window_sweep.py b/probes/weather-p1/ev4_window_sweep.py index 9ec32ea7..3a6f6c85 100644 --- a/probes/weather-p1/ev4_window_sweep.py +++ b/probes/weather-p1/ev4_window_sweep.py @@ -61,6 +61,7 @@ def fz(s, eps=1e-9): def load_anom(var): + """Load a fixture variable and return its ZONAL anomaly (per-row climatology removed).""" a = np.load(f'fixture/{var}.npy').astype(np.float64) assert np.isfinite(a).all(), f"{var}: nonfinite values present in fixture" clim = a.mean(axis=1, keepdims=True) diff --git a/probes/weather-p1/go_territory_probe.json b/probes/weather-p1/go_territory_probe.json new file mode 100644 index 00000000..8bb0982f --- /dev/null +++ b/probes/weather-p1/go_territory_probe.json @@ -0,0 +1,104 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "time_index": 91246, + "band_lat": [ + 25, + 75 + ], + "A": { + "explained_by_k": [ + 0.1168690839787806, + 0.2127309490271857, + 0.26574796827848, + 0.2939871339283845, + 0.32918200372029927, + 0.3822168549947018, + 0.3903509061233552, + 0.4108893433743397, + 0.4380645265327874, + 0.523308291374023 + ], + "centers": [ + { + "lat": 55.75, + "lon": 334.5 + }, + { + "lat": 28.5, + "lon": 67.5 + }, + { + "lat": 67.0, + "lon": 28.0 + }, + { + "lat": 31.75, + "lon": 89.75 + }, + { + "lat": 74.25, + "lon": 318.25 + }, + { + "lat": 43.75, + "lon": 108.75 + }, + { + "lat": 36.0, + "lon": 70.5 + }, + { + "lat": 32.25, + "lon": 47.25 + }, + { + "lat": 54.5, + "lon": 128.75 + }, + { + "lat": 33.0, + "lon": 320.5 + } + ], + "E1": 0.523308291374023, + "E2_random": 0.40571381344120383, + "E3_decay_ratios": [ + 0.8202499907144846, + 0.553056413251751, + 0.5326434048669204, + 1.2463140812389346, + 1.5068915324297096, + 0.1533718099173651, + 2.5249948551017583, + 1.3231378233095652, + 3.136823930282756 + ], + "E3_mean": 1.3108315379014717 + }, + "B": { + "n_centers": 20, + "by_thresh": { + "0.7": { + "contested_frac": 0.3439227473742399, + "gradT_contested": 2.7313515511506186, + "gradT_secured": 3.546598376186566, + "ratio": 0.7701327473361868 + }, + "0.8": { + "contested_frac": 0.22770176893311223, + "gradT_contested": 2.8360323583495233, + "gradT_secured": 3.546598376186566, + "ratio": 0.7996485808463406 + }, + "0.9": { + "contested_frac": 0.11122857932559425, + "gradT_contested": 2.938374656664403, + "gradT_secured": 3.546598376186566, + "ratio": 0.8285050476518439 + } + }, + "E2_secured_le_bandmean": false, + "E3_random_ratio": 1.2384930773081844, + "gradT_band_mean_K_per_100km": 2.8310022821582104 + } +} \ No newline at end of file diff --git a/probes/weather-p1/go_territory_probe.py b/probes/weather-p1/go_territory_probe.py new file mode 100644 index 00000000..209b03ed --- /dev/null +++ b/probes/weather-p1/go_territory_probe.py @@ -0,0 +1,218 @@ +"""EXPLORATORY — 'overlapping golden spirals': eine Spirale pro Hoch/Tief, +Flaechen-vs-Nahkampf-Logik wie Go. (Operator, 2026-08-11.) NOT an EV; bars +mine, unaudited — measurements, not findings. + +Two falsifiable halves: + +A (FLAECHEN / territory): the pressure field as a SUPERPOSITION of radial + (sunflower-samplable) profiles around MANY centers — matching pursuit with + radial atoms. Directly tests the successor hypothesis from + sunflower_cyclone_probe's E3 fail: the single-storm 'azimuthal residual' + (36%) is largely the overlap of NEIGHBORING systems. +B (NAHKAMPF / contested zones): Go-style influence tessellation + (influence_i = |A_i| / r_i^2 — scale-free, deliberately NO length knob). + Contested cells (second/best influence > threshold) should carry the + FRONTS: elevated |grad T2m|. Secured territory should be calm. + +PRE-REGISTERED: + A-E1 greedy K=10 multi-center radial superposition explains >= 0.55 of + p_anom variance over the NH 25..75N band (fresh domain, fresh bar — + NOT comparable to the single-disk 0.639). + A-E2 control: K=10 RANDOM-center atoms explain <= HALF of A-E1's value. + A-E3 observation only, NO bar: marginal-gain decay ratio reported and + compared to 1/phi = 0.618 as a labelled CURIOSITY. + B-E1 can-fire: mean |grad T| in contested cells (ratio>0.8) >= 1.4x mean + in secured cells (ratio<0.3). Inertness: also reported at 0.7/0.9. + B-E2 silence: secured-territory mean |grad T| <= band mean. + B-E3 control: random-position centers, same rule -> ratio in [0.8, 1.2]. + +Same store/timestep as the whole arc (WB2, t=91246). Planar per-center km +approx with cos(lat_center) — consistent with the sibling probes, stated. +Hoehenprofil (geopotential stacking) deliberately NOT here — next probe. +""" +import json +import pathlib +import os +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T_IDX = 91246 +R_E = 6371.0 +RINGS_KM, R_MAX_KM = 100.0, 2000.0 +MIN_SEP_KM = 400.0 + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=180).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +lat = fetch("latitude", "0").astype(np.float64).ravel() +p = fetch("mean_sea_level_pressure", f"{T_IDX}.0.0")[0].astype(np.float64) +t2 = fetch("2m_temperature", f"{T_IDX}.0.0")[0].astype(np.float64) +phi = np.deg2rad(lat) +lon_deg = np.arange(p.shape[1]) * 0.25 + +r0, r1 = 60, 261 # 75N .. 25N band (descending lat) +band = slice(r0, r1) +pa = p - p.mean(axis=1, keepdims=True) +pa_band = pa[band].copy() +print(f"band: lat {lat[r0]:.1f}..{lat[r1-1]:.1f}, shape {pa_band.shape}, " + f"var {pa_band.var():.0f} Pa^2") + + +def dist_km(ci, cj): + """km distance from band gridpoints to center (band-row ci, col cj).""" + dlon = np.deg2rad((lon_deg[None, :] - lon_deg[cj] + 180) % 360 - 180) + dphi = phi[band][:, None] - phi[r0 + ci] + dx = R_E * np.cos(phi[r0 + ci]) * dlon + dy = R_E * dphi + return np.hypot(dx * np.ones_like(dy), dy * np.ones_like(dlon)) + + +def fit_atom(res, ci, cj): + """Ring-mean radial profile of `res` around (ci, cj), subtractable field.""" + r = dist_km(ci, cj) + rings = (r / RINGS_KM).astype(int) + atom = np.zeros_like(res) + for b in range(int(R_MAX_KM / RINGS_KM)): + m = rings == b + if m.any(): + atom[m] = res[m].mean() + atom[r > R_MAX_KM] = 0.0 + return atom, r + + +def matching_pursuit(field, k_max, centers=None): + """Greedy K radial atoms; centers picked from residual argmax (or given).""" + res = field.copy() + # Fixed denominator = the CENTERED field's variance; the numerator is the + # residual's MEAN SQUARE, not its variance. `res.var()` re-centres after + # every atom, so a non-zero residual mean was excluded from the error and + # explained variance came out inflated (coderabbit on PR #926). + v0 = float(np.mean((field - field.mean()) ** 2)) + used, explained = [], [] + for k in range(k_max): + if centers is not None: + ci, cj = centers[k] + else: + cand = np.abs(res).copy() + for (ui, uj) in used: + cand[dist_km(ui, uj) < MIN_SEP_KM] = 0 + ci, cj = np.unravel_index(np.argmax(cand), res.shape) + atom, _ = fit_atom(res, ci, cj) + res -= atom + used.append((int(ci), int(cj))) + explained.append(1.0 - float(np.mean(res ** 2)) / v0) + return used, explained + + +# ---- Part A ----------------------------------------------------------- +K = 10 +centers, expl = matching_pursuit(pa_band, K) +print("\nA: greedy multi-center radial superposition (variance explained):") +for k, e in enumerate(expl): + ci, cj = centers[k] + print(f" K={k+1:2} R2={e:.3f} center lat={lat[r0+ci]:6.2f} " + f"lon={lon_deg[cj]:6.2f} p'={pa_band[ci, cj]:.0f} Pa " + f"({'H' if pa_band[ci, cj] > 0 else 'T'})") +a_e1 = expl[-1] + +rng = np.random.default_rng(7) +rand_centers = [(int(rng.integers(0, pa_band.shape[0])), + int(rng.integers(0, pa_band.shape[1]))) for _ in range(K)] +_, expl_rand = matching_pursuit(pa_band, K, centers=rand_centers) +a_e2 = expl_rand[-1] +print(f" A-E1 K=10 matched: R2={a_e1:.3f} (bar >= 0.55)") +print(f" A-E2 K=10 random: R2={a_e2:.3f} (bar <= {a_e1/2:.3f})") + +gains = np.diff(np.concatenate([[0.0], expl])) +ratios = gains[1:] / np.maximum(gains[:-1], 1e-12) +print(f" A-E3 marginal-gain decay ratios: " + f"{[f'{x:.2f}' for x in ratios]} mean={ratios.mean():.3f} " + f"(curiosity vs 1/phi=0.618 — NO bar)") + +# ---- Part B ----------------------------------------------------------- +det = [] +cand = np.abs(pa_band).copy() +while len(det) < 20: + ci, cj = np.unravel_index(np.argmax(cand), cand.shape) + if cand[ci, cj] < 400: + break + det.append((int(ci), int(cj), float(pa_band[ci, cj]))) + cand[dist_km(ci, cj) < 500.0] = 0 +print(f"\nB: {len(det)} centers (|p'|>=400 Pa), " + f"{sum(1 for d in det if d[2] > 0)} highs / " + f"{sum(1 for d in det if d[2] < 0)} lows") + + +def contested_secured(center_list, thresh): + """Split the band into CONTESTED and SECURED cells by the runner-up/leader influence ratio. + + Influence is |amplitude|/r^2 (scale-free), floored at 50 km so a cell at a + centre does not diverge. `ratio = second-strongest / strongest`: high ratio + = two centres competing (contested), low = one centre dominant (secured).""" + inf = np.zeros((len(center_list),) + pa_band.shape) + for i, (ci, cj, amp) in enumerate(center_list): + r = np.maximum(dist_km(ci, cj), 50.0) + inf[i] = np.abs(amp) / r**2 + s = np.sort(inf, axis=0) + ratio = s[-2] / s[-1] + return ratio > thresh, ratio < 0.3 + + +dxm = R_E * 1e3 * np.cos(phi[band]) * np.deg2rad(0.25) +dym = R_E * 1e3 * np.deg2rad(0.25) +t2b = t2[band] +gTx = (np.roll(t2b, -1, 1) - np.roll(t2b, 1, 1)) / (2 * dxm[:, None]) +gTy = np.gradient(t2b, axis=0) / -dym # lat descends +gradT = np.hypot(gTx, gTy) * 1e5 # K per 100 km +band_mean = gradT.mean() + +out_b = {} +for th in (0.7, 0.8, 0.9): + contested, secured = contested_secured(det, th) + ratio = gradT[contested].mean() / gradT[secured].mean() + out_b[str(th)] = { + "contested_frac": float(contested.mean()), + "gradT_contested": float(gradT[contested].mean()), + "gradT_secured": float(gradT[secured].mean()), + "ratio": float(ratio)} + print(f" thresh {th}: contested {contested.mean()*100:4.1f}% of band, " + f"|gradT| contested/secured = {gradT[contested].mean():.3f}/" + f"{gradT[secured].mean():.3f} = {ratio:.2f}x") +b_e1 = out_b["0.8"]["ratio"] +_, secured08 = contested_secured(det, 0.8) +b_e2 = bool(gradT[secured08].mean() <= band_mean) + +rand_det = [(int(rng.integers(0, pa_band.shape[0])), + int(rng.integers(0, pa_band.shape[1])), amp) + for (_, _, amp) in det] +c_r, s_r = contested_secured(rand_det, 0.8) +b_e3 = float(gradT[c_r].mean() / gradT[s_r].mean()) +print(f" B-E1 ratio@0.8 = {b_e1:.2f}x (bar >= 1.4) " + f"B-E2 secured {gradT[secured08].mean():.3f} <= band {band_mean:.3f}: " + f"{b_e2} B-E3 random-centers ratio = {b_e3:.2f} (bar 0.8..1.2)") + +json.dump({ + "store": B, "time_index": T_IDX, "band_lat": [25, 75], + "A": {"explained_by_k": expl, "centers": [ + {"lat": float(lat[r0 + ci]), "lon": float(lon_deg[cj])} + for ci, cj in centers], + "E1": a_e1, "E2_random": a_e2, + "E3_decay_ratios": ratios.tolist(), "E3_mean": float(ratios.mean())}, + "B": {"n_centers": len(det), "by_thresh": out_b, + "E2_secured_le_bandmean": b_e2, "E3_random_ratio": b_e3, + "gradT_band_mean_K_per_100km": float(band_mean)}, +}, open(pathlib.Path(__file__).with_name("go_territory_probe.json"), "w"), indent=2) +print("\nwrote go_territory_probe.json") diff --git a/probes/weather-p1/l4_rail_probe.json b/probes/weather-p1/l4_rail_probe.json new file mode 100644 index 00000000..a3583eff --- /dev/null +++ b/probes/weather-p1/l4_rail_probe.json @@ -0,0 +1,125 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "t0": 91246, + "carves": { + "storm1": { + "uniform": { + "f64_rmse_pa": 241.6361781953603, + "f64_bias_pa": 8.567361346570961e-13, + "carveD_rmse_pa": 241.71119512018444, + "carveD_bias_pa": 1.587879693149758, + "carveA_rmse_pa": 299.9109562762598, + "carveA_bias_pa": 92.76221653277149, + "f64_r2": 0.9434382813469204, + "carveA_r2": 0.9128668641737903, + "carveB_r2": 0.634793668913492, + "carveD_r2": 0.943403156268636, + "ring_rmse_pa": 4.484955284768884, + "diag_14byte_r2": 0.9434184182770502, + "loss_quantization": 1.9863069870229566e-05, + "loss_dropped_rings": 0.030551554103259937 + }, + "fisherz": { + "f64_rmse_pa": 241.6361781953603, + "f64_bias_pa": 8.567361346570961e-13, + "carveD_rmse_pa": 244.26761112671338, + "carveD_bias_pa": 16.20184874441947, + "carveA_rmse_pa": 297.62202871400433, + "carveA_bias_pa": 91.1758913030388, + "f64_r2": 0.9434382813469204, + "carveA_r2": 0.9141917931977205, + "carveB_r2": 0.63359636825377, + "carveD_r2": 0.942199652266129, + "ring_rmse_pa": 32.22635385503535, + "diag_14byte_r2": 0.9422211176173282, + "loss_quantization": 0.001217163729592241, + "loss_dropped_rings": 0.028029324419607704 + }, + "fisherz_field": { + "f64_rmse_pa": 241.6361781953603, + "f64_bias_pa": 8.567361346570961e-13, + "carveD_rmse_pa": 242.24982848889988, + "carveD_bias_pa": 10.04382760223327, + "carveA_rmse_pa": 299.413949917866, + "carveA_bias_pa": 95.5304070007748, + "f64_r2": 0.9434382813469204, + "carveA_r2": 0.9131554154183784, + "carveB_r2": 0.634566955417994, + "carveD_r2": 0.9431506324649834, + "ring_rmse_pa": 16.13215666344166, + "diag_14byte_r2": 0.9431917047815523, + "loss_quantization": 0.00024657656536808403, + "loss_dropped_rings": 0.03003628936317393 + } + }, + "storm2": { + "uniform": { + "f64_rmse_pa": 241.08275703543822, + "f64_bias_pa": -2.1189001172580308e-12, + "carveD_rmse_pa": 241.11386850868683, + "carveD_bias_pa": 0.06394187896472865, + "carveA_rmse_pa": 250.5414872244609, + "carveA_bias_pa": 35.41997442833062, + "f64_r2": 0.909049958041129, + "carveA_r2": 0.9017732201135151, + "carveB_r2": 0.29426134128134374, + "carveD_r2": 0.9090264825142191, + "ring_rmse_pa": 3.1878395651185287, + "diag_14byte_r2": 0.9090264825142191, + "loss_quantization": 2.3475526909844113e-05, + "loss_dropped_rings": 0.007253262400704008 + }, + "fisherz": { + "f64_rmse_pa": 241.08275703543822, + "f64_bias_pa": -2.1189001172580308e-12, + "carveD_rmse_pa": 241.13716161276324, + "carveD_bias_pa": 1.7116952576486248, + "carveA_rmse_pa": 250.83166441880857, + "carveA_bias_pa": 37.53956185125506, + "f64_r2": 0.909049958041129, + "carveA_r2": 0.901545555802089, + "carveB_r2": 0.29425262761576254, + "carveD_r2": 0.9090089044463867, + "ring_rmse_pa": 3.920293632791127, + "diag_14byte_r2": 0.909017768848638, + "loss_quantization": 3.218919249092789e-05, + "loss_dropped_rings": 0.007472213046549103 + }, + "fisherz_field": { + "f64_rmse_pa": 241.08275703543822, + "f64_bias_pa": -2.1189001172580308e-12, + "carveD_rmse_pa": 242.4493237572494, + "carveD_bias_pa": -14.03074944463002, + "carveA_rmse_pa": 256.4010807615417, + "carveA_bias_pa": 41.8932699495118, + "f64_r2": 0.909049958041129, + "carveA_r2": 0.897124891138573, + "carveB_r2": 0.29325938028542453, + "carveD_r2": 0.9080159432790177, + "ring_rmse_pa": 21.860300207395742, + "diag_14byte_r2": 0.9080245215183, + "loss_quantization": 0.0010254365228289375, + "loss_dropped_rings": 0.010899630379727077 + } + } + }, + "cross": { + "storm1_cb_on_storm2": { + "shared_rmse_pa": 3.1878395651185287, + "own_rmse_pa": 3.1878395651185287, + "penalty_frac": 0.0, + "degenerate": true + }, + "storm2_cb_on_storm1": { + "shared_rmse_pa": 4.484955284768884, + "own_rmse_pa": 620.786190272059, + "penalty_frac": -0.9927753623468922, + "degenerate": false + } + }, + "ring_rmse_by_codebook": { + "uniform": 3.8363974249437067, + "fisherz": 18.07332374391324, + "fisherz_field": 18.9962284354187 + } +} \ No newline at end of file diff --git a/probes/weather-p1/l4_rail_probe.py b/probes/weather-p1/l4_rail_probe.py new file mode 100644 index 00000000..b36a9ab5 --- /dev/null +++ b/probes/weather-p1/l4_rail_probe.py @@ -0,0 +1,331 @@ +"""EXPLORATORY — does the storm spine fit the ACTUAL L4 carrier, 6 x (8:8)? +Operator correction (2026-08-11): "was ist mit 6x Palette256:Palette256 +centroid, was ja die Verteilung anzeigen soll — palette256 alleine ist ja nur +'attention header'". NOT an EV. + +WHAT I HAD WRONG. Every encoding in this arc treated "one scalar -> one byte" +as the unit. The shipped carrier is a PAIR: le-contract §3 row L4 — +`6 x (8:8)`, `palette256²`, "each byte pair indexes the 256x256 palette +distance/compose tables; similarity = ONE table read". A single palette256 +byte is only the SELECTOR (which archetype). The PAIR is a cell in the +centroid tile, and the tile is where the distribution lives — materialized +once, so a node stores a coordinate, not a value. Six pairs = six subspaces. + +Two things this reframes rather than replaces: + * my 2-tier rolling-floor cascade IS the sanctioned §3 reading + "area : location in stacked exactness" (each pair refines within its + area) — but it is ONE RAIL, not the whole payload. I built a rail and + called it the carrier. + * the Fisher-Z register is not a rival to the palette; le-contract §3 says + L4's cosine-replacement palette256 reads through the analytic Fisher-z + canon and "an L4 ClassView may declare an analytic codebook" — i.e. + Fisher-Z is how the 256 centroid entries PER AXIS are laid out. + +THE CARVE UNDER TEST (12 bytes total, exactly one V3 facet payload): + rail 0 dipole (amplitude_slope : bearing) <- natural pair + rail 1 rings 0,1 (ring0 : ring1) + rail 2 rings 2,3 + rail 3 rings 4,5 + rail 4 rings 6,7 + rail 5 rings 8,9 +Rings 10,11 do not fit. That is a REAL constraint of the carrier and is +reported, not engineered away: the alternative (drop the dipole rail to fit +all 12 rings) is measured too, so the trade is visible. + +Ring bytes use a SHARED codebook across storms (the tile is global, per +le-contract "similarity = ONE table read" — a per-storm codebook would make +two storms' rails incomparable, defeating the carrier). Codebook axes are +built two ways and compared: + UNIFORM 256 levels over the global ring-value range + FISHERZ 256 levels uniform in arctanh(2*rank-1) of the global ring + distribution — the analytic-codebook reading + +PRE-REGISTERED: + L1 carve A (dipole + rings 0..9) recovers in-disk R2 within 0.02 of the + f64 constrained spine (0.943 / 0.909 from comet_tail_probe). + L2 carve B (all 12 rings, no dipole rail) is WORSE than carve A on + storm 1 — the dipole is worth more than the two outermost rings. + If it is not, the carve should be B and the claim inverts. + L3 FISHERZ codebook >= UNIFORM on ring RMSE (the analytic-codebook + reading earns its place, or it does not). + L4x CROSS-STORM: with the shared codebook, storm1 and storm2 rails are + directly comparable — the same byte denotes the same centroid. + Falsifier: recovering storm2 through storm1's OWN codebook must be + NO WORSE than through the shared one by more than 1%. If it is + worse, the codebook is not actually global and the pair loses its + "one table read" property. +""" +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T0 = 91246 +R_E, R_DISK, RING = 6371.0, 1200.0, 100.0 +PAL = 256 + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=600).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +lat = fetch("latitude", "0").astype(np.float64).ravel() +p0 = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0].astype(np.float64) +NY, NX = p0.shape +phi = np.deg2rad(lat) +lon = np.arange(NX) * 0.25 + + +def geom(la, lo): + """dx, dy, r (km), azimuth (rad CCW from east) about a continuous centre.""" + dlon = np.deg2rad((lon[None, :] - lo + 180) % 360 - 180) + dx = R_E * np.cos(np.deg2rad(la)) * dlon * np.ones((NY, 1)) + dy = R_E * (phi[:, None] - np.deg2rad(la)) * np.ones((1, NX)) + return dx, dy, np.hypot(dx, dy), np.arctan2(dy, dx) + + +def find(field, near=None, rad=600.0): + """Deepest NH zonal-anomaly low, optionally limited to a disk about `near`.""" + fa = field - field.mean(axis=1, keepdims=True) + m = lat[:, None] > 15 + if near is not None: + m = m & (geom(*near)[2] < rad) + mk = np.where(m, fa, np.inf) + i, j = np.unravel_index(np.argmin(mk), field.shape) + return int(i), int(j) + + +def spine(la, lo): + """The f64 spine: ring-profile means + the CONSTRAINED 2-param dipole.""" + _, _, r, th = geom(la, lo) + disk = r <= R_DISK + v, rr, tt = p0[disk], r[disk], th[disk] + nb = int(R_DISK / RING) + rings = np.clip((rr / RING).astype(int), 0, nb - 1) + prof = np.array([v[rings == b].mean() if (rings == b).any() else 0.0 + for b in range(nb)]) + resid = v - prof[rings] + X = np.column_stack([rr * np.cos(tt), rr * np.sin(tt)]) + coef, *_ = np.linalg.lstsq(X, resid, rcond=None) + return dict(vals=v, rings=rings, tt=tt, rr=rr, prof=prof, coef=coef, + var=v.var(), nb=nb) + + +def r2_of(s, prof, coef): + """In-disk R2 of a (profile, dipole) reconstruction against the raw field. + + The numerator is the UNCENTERED mean squared error, NOT var(): var() drops + the squared MEAN residual, so a BIASED reconstruction is flattered. This + matters here and is not cosmetic -- carve A holds the two outer rings at a + fixed value, and under var() that bias was invisible (coderabbit, PR #926, + 2026-08-12; storm1 carve A moved 0.9212 -> 0.9129 on the fix). + """ + rec = prof[s["rings"]] + coef[0] * s["rr"] * np.cos(s["tt"]) \ + + coef[1] * s["rr"] * np.sin(s["tt"]) + return 1.0 - (np.mean((s["vals"] - rec) ** 2) / s["var"]) + + +def err_pa(s, prof, coef): + """(RMSE, mean bias) in Pa -- the quantity R2 is nearly BLIND to here. + + In-disk variance is ~1e5 Pa^2, so a systematic offset of a couple of Pa + moves R2 by ~1e-6 and vanishes at any printed precision. Reporting R2 + alone is what made a biased reconstruction look 'lossless'; these two + numbers are what actually distinguish the carves. + """ + rec = prof[s["rings"]] + coef[0] * s["rr"] * np.cos(s["tt"]) \ + + coef[1] * s["rr"] * np.sin(s["tt"]) + e = s["vals"] - rec + return float(np.sqrt(np.mean(e ** 2))), float(e.mean()) + + +def cb_uniform(pop): + """256 centroid levels, uniform over the population's value range.""" + lo, hi = pop.min(), pop.max() + return lo + (np.arange(PAL) + 0.5) / PAL * (hi - lo) + + +def cb_fisherz(pop): + """256 centroid levels, uniform in arctanh(2*rank-1) — analytic codebook. + + NOTE the population matters and is the whole of L3-vs-L3b: `pop` is the + distribution the RANKS are taken against, not merely the values to encode. + Passing the 24 ring means (L3) makes every sample its own quantile, so the + rim-stretch spends levels on 24 isolated points; passing the field (L3b) + is the reading le-contract actually describes. + """ + srt = np.sort(pop) + eps = 1e-9 + rr = (np.arange(len(srt)) + 0.5) / len(srt) + zr = np.arctanh(np.clip(2 * rr - 1, -1 + eps, 1 - eps)) + zc = zr.min() + (np.arange(PAL) + 0.5) / PAL * (zr.max() - zr.min()) + q = np.clip((np.tanh(zc) + 1) / 2, 0, 1) + return np.quantile(srt, q) + + +def enc(vals, cb): + """Nearest-centroid byte code + its reconstruction (the attention header).""" + idx = np.abs(vals[:, None] - cb[None, :]).argmin(axis=1) + return idx.astype(np.uint8), cb[idx] + + +# ---- storms + the SHARED (global) ring codebook ----------------------------- +storms = {} +for nm, hint in (("storm1", None), ("storm2", (67.0, 28.0))): + ci, cj = find(p0, near=hint) + storms[nm] = spine(lat[ci], lon[cj]) + print(f"{nm}: centre ({lat[ci]:.2f}N,{lon[cj]:.2f}E)") + +pop = np.concatenate([s["prof"] for s in storms.values()]) +# L3b population: the FIELD the ring means are drawn from (both storms' disks), +# not the 24 encoded values themselves. See cb_fisherz's note. +field_pop = np.concatenate([s["vals"] for s in storms.values()]) +CB = {"uniform": cb_uniform(pop), + "fisherz": cb_fisherz(pop), + "fisherz_field": cb_fisherz(field_pop)} + +# dipole rail: amplitude slope + bearing, one byte each +amp_pop = np.array([np.hypot(*s["coef"]) for s in storms.values()]) +AMP_CB = np.linspace(0.0, amp_pop.max() * 1.25, PAL) +BRG_CB = -np.pi + (np.arange(PAL) + 0.5) / PAL * 2 * np.pi # 1.406 deg/level + +out = {"store": B, "t0": T0, "carves": {}, "cross": {}} +print(f"\nbearing resolution: {360/PAL:.3f} deg/level\n") +print(f"{'storm':<9} {'codebook':<13} {'f64 spine':>10} {'A dip+r0-9':>12} " + f"{'D spread':>10} {'B 12 rings':>12} {'ring RMSE':>10} {'14B diag':>10}") + +for nm, s in storms.items(): + row = {} + f64_r2 = r2_of(s, s["prof"], s["coef"]) + for cbn, cb in CB.items(): + codes, rec = enc(s["prof"], cb) + ring_rmse = float(np.sqrt(np.mean((rec - s["prof"]) ** 2))) + + # carve A: rail0 = dipole, rails1-5 = rings 0..9. Rings 10,11 have NO + # byte at all — they fall back to the last encoded ring (a held edge), + # which is the honest cost of the 12-byte budget, not a hidden fit. + pa_A = np.concatenate([rec[:10], np.full(s["nb"] - 10, rec[9])]) + ai = int(np.abs(np.hypot(*s["coef"]) - AMP_CB).argmin()) + bi = int(np.abs(np.arctan2(s["coef"][1], s["coef"][0]) - BRG_CB).argmin()) + a_q, b_q = AMP_CB[ai], BRG_CB[bi] + coef_q = np.array([a_q * np.cos(b_q), a_q * np.sin(b_q)]) + r2_A = r2_of(s, pa_A, coef_q) + + # carve B: all 12 rings, NO dipole rail + r2_B = r2_of(s, rec, np.array([0.0, 0.0])) + + # DIAGNOSTIC ONLY, 14 bytes — OVER the facet budget. Not a candidate + # carve; it exists to split L1's loss into (i) byte quantization and + # (ii) the two rings the 12-byte budget cannot afford. + r2_over = r2_of(s, rec, coef_q) + + # carve D, still 12 bytes: same rails as A, but the 10 ring bytes are + # SPREAD over the full radius (positions linspace(0, nb-1, 10)) and the + # missing rings linearly interpolated. Costs nothing extra — the + # profile is smooth, so 10 samples across 12 beats 10 samples of the + # inner 12 plus a held edge. Pre-registered: D >= A on storm1. + pos = np.unique(np.round(np.linspace(0, s["nb"] - 1, 10)).astype(int)) + pd_ = np.interp(np.arange(s["nb"]), pos, rec[pos]) + r2_D = r2_of(s, pd_, coef_q) + + rm_f, bias_f = err_pa(s, s["prof"], s["coef"]) + rm_d, bias_d = err_pa(s, pd_, coef_q) + rm_a, bias_a = err_pa(s, pa_A, coef_q) + row[cbn] = {"f64_rmse_pa": rm_f, "f64_bias_pa": bias_f, + "carveD_rmse_pa": rm_d, "carveD_bias_pa": bias_d, + "carveA_rmse_pa": rm_a, "carveA_bias_pa": bias_a, + "f64_r2": float(f64_r2), "carveA_r2": float(r2_A), + "carveB_r2": float(r2_B), "carveD_r2": float(r2_D), + "ring_rmse_pa": ring_rmse, + "diag_14byte_r2": float(r2_over), + "loss_quantization": float(f64_r2 - r2_over), + "loss_dropped_rings": float(r2_over - r2_A)} + print(f"{nm:<9} {cbn:<13} {f64_r2:>10.4f} {r2_A:>12.4f} " + f"{r2_D:>10.4f} {r2_B:>12.4f} {ring_rmse:>10.2f} {r2_over:>10.4f}") + out["carves"][nm] = row + +# ---- L4x cross-storm: shared vs per-storm codebook -------------------------- +# Measured through UNIFORM — the codebook L3 names as best. Measuring the +# cross-storm property of a codebook the previous bar just rejected would be +# reporting a property of something we are not proposing to use. +# +# ANTI-VACUITY (this bar was measured vacuous on its first run and is kept +# here as the guard that caught it): a uniform codebook is fixed by its +# population's MIN and MAX alone, so if one storm's profile range CONTAINS the +# other's, that storm's "own" codebook IS the pooled codebook — the test then +# compares an array against itself and passes for free. storm1 ⊃ storm2 here, +# so the storm1->storm2 direction is degenerate BY CONSTRUCTION and only the +# storm2->storm1 direction can carry information. Both are reported. +print("\n=== L4x cross-storm codebook (uniform) ===") +out["cross"] = {} +for src, dst in (("storm1", "storm2"), ("storm2", "storm1")): + own_cb = cb_uniform(storms[src]["prof"]) + tgt = storms[dst]["prof"] + degenerate = bool(np.allclose(own_cb, CB["uniform"])) + _, rec_shared = enc(tgt, CB["uniform"]) + _, rec_own = enc(tgt, own_cb) + r_sh = float(np.sqrt(np.mean((rec_shared - tgt) ** 2))) + r_ow = float(np.sqrt(np.mean((rec_own - tgt) ** 2))) + pen = (r_sh - r_ow) / max(r_ow, 1e-9) + out["cross"][f"{src}_cb_on_{dst}"] = { + "shared_rmse_pa": r_sh, "own_rmse_pa": r_ow, + "penalty_frac": pen, "degenerate": degenerate} + tag = ("DEGENERATE — own codebook IS the shared one, carries no " + "information") if degenerate else "informative" + print(f" {src} codebook -> {dst}: shared {r_sh:.2f} Pa vs own " + f"{r_ow:.2f} Pa ({100*pen:+.1f}%) [{tag}]") + +print("\n=== pre-registered verdicts ===") +BEST = "uniform" # named by L3, below — reported, not assumed +for nm in storms: + r = out["carves"][nm][BEST] + d = abs(r["carveA_r2"] - r["f64_r2"]) + print(f"L1 {nm}: |carveA - f64| = {d:.4f} (bar <= 0.02): " + f"{'PASS' if d <= 0.02 else 'FAIL'}" + f" [quantization {r['loss_quantization']:+.4f}, " + f"dropped rings 10-11 {r['loss_dropped_rings']:+.4f}]") +for nm in storms: + r = out["carves"][nm][BEST] + d = abs(r["carveD_r2"] - r["f64_r2"]) + print(f"L1' {nm} carve D (spread rings, still 12 B): |D - f64| = {d:.4f} " + f"(bar <= 0.02): {'PASS' if d <= 0.02 else 'FAIL'}") +r1 = out["carves"]["storm1"][BEST] +print(f"L2 storm1 carve A > carve B ({r1['carveA_r2']:.4f} > " + f"{r1['carveB_r2']:.4f}): {'PASS' if r1['carveA_r2'] > r1['carveB_r2'] else 'FAIL'}") +print(f"L2b storm1 carve D >= carve A ({r1['carveD_r2']:.4f} >= " + f"{r1['carveA_r2']:.4f}): " + f"{'PASS' if r1['carveD_r2'] >= r1['carveA_r2'] else 'FAIL'}") +rm = {c: float(np.mean([out["carves"][n][c]["ring_rmse_pa"] for n in storms])) + for c in CB} +print(f"L3 FISHERZ(24 ring means) ring RMSE {rm['fisherz']:.2f} <= " + f"UNIFORM {rm['uniform']:.2f}: " + f"{'PASS' if rm['fisherz'] <= rm['uniform'] else 'FAIL'}") +print(f"L3b FISHERZ(field population) {rm['fisherz_field']:.2f} <= " + f"UNIFORM {rm['uniform']:.2f}: " + f"{'PASS' if rm['fisherz_field'] <= rm['uniform'] else 'FAIL'}" + " [POST-HOC refinement — the rank population is the FIELD, not the\n" + " 24 values being encoded. Stated as post-hoc, not pre-registered.]") +out["ring_rmse_by_codebook"] = rm +inf_dirs = {k: v for k, v in out["cross"].items() if not v["degenerate"]} +if not inf_dirs: + print("L4x VOID — every direction degenerate; the bar measured nothing.") +else: + worst = max(v["penalty_frac"] for v in inf_dirs.values()) + print(f"L4x shared-vs-own penalty (worst informative direction of " + f"{len(inf_dirs)}/{len(out['cross'])}) {100*worst:+.1f}% " + f"(bar <= +1%): {'PASS' if worst <= 0.01 else 'FAIL'}") + +with open(pathlib.Path(__file__).with_name("l4_rail_probe.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote l4_rail_probe.json") diff --git a/probes/weather-p1/sunflower_cyclone_probe.json b/probes/weather-p1/sunflower_cyclone_probe.json new file mode 100644 index 00000000..62709504 --- /dev/null +++ b/probes/weather-p1/sunflower_cyclone_probe.json @@ -0,0 +1,65 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "time_index": 91246, + "R_disk_km": 1200.0, + "center": { + "lat": 55.75, + "lon": 334.5, + "p_anom_Pa": -2954.4862955729186 + }, + "E1_axisym_storm": 0.6392718246695573, + "E4_axisym_offset750km": 0.004993305440683216, + "E2_budgets": { + "64": { + "spiral": { + "n": 64, + "rmse_Pa": 234.48255636818175 + }, + "grid": { + "n": 64, + "rmse_Pa": 269.01833911376923 + }, + "random": { + "n": 64, + "rmse_Pa": 319.37183755793615 + } + }, + "256": { + "spiral": { + "n": 256, + "rmse_Pa": 119.10369656260444 + }, + "grid": { + "n": 256, + "rmse_Pa": 123.32309279184084 + }, + "random": { + "n": 256, + "rmse_Pa": 164.15349088979696 + } + }, + "1024": { + "spiral": { + "n": 1024, + "rmse_Pa": 58.854942160393385 + }, + "grid": { + "n": 1024, + "rmse_Pa": 59.86267107651939 + }, + "random": { + "n": 1024, + "rmse_Pa": 84.22335048725353 + } + } + }, + "E3_delta_entropy_bits": { + "spiral_order": 7.127796366175735, + "raster_order": 5.946501707930685 + }, + "E5_u8": { + "rmse_Pa": 119.175017454153, + "bucket_Pa": 17.188925402801544, + "added_vs_f64_Pa": 0.07132089154855237 + } +} \ No newline at end of file diff --git a/probes/weather-p1/sunflower_cyclone_probe.py b/probes/weather-p1/sunflower_cyclone_probe.py new file mode 100644 index 00000000..6af3642a --- /dev/null +++ b/probes/weather-p1/sunflower_cyclone_probe.py @@ -0,0 +1,232 @@ +"""EXPLORATORY — 'die goldene Spirale encodiert das ganze Tiefdruckgebiet' +(operator, 2026-08-11, sunflower-ripples frame). NOT an EV; bars are mine, +unaudited — measurements, not findings, per this session's 0/11 lesson. + +The claim, made falsifiable: a cyclone is near-axisymmetric about its center, +so sampling it along the EQUAL-AREA golden spiral (r = sqrt(u)*R, theta = +n*golden_angle — literally helix::HemispherePoint::lift's lattice) turns the +2-D pressure structure into a low-entropy 1-D 'ripple' signal: the radial +profile carries the mass, consecutive spiral samples are near in radius, so +deltas are small. If true, bgz17-palette + highheelbgz spiral-ADDRESSING +(start, stride, len — values recomputed on demand) has something real to +encode. If false, the spiral is decoration. + +PRE-REGISTERED: + E1 Axisymmetry: the ring-mean radial profile explains >= 70% of pressure + variance in the storm disk (R=1200 km). + E2 Equal budget N=256: golden-spiral sampling reconstructs the disk with + RMSE <= uniform-grid sampling AND <= random sampling (nearest-neighbor + reconstruction, same N, same reconstruction rule for every arm). + E3 Ripples: Shannon entropy of u8 first-differences ALONG THE SPIRAL is + lower than along a raster scan of the SAME samples — the spiral + ordering is what makes the signal low-entropy, not the quantization. + E4 CONTROL (can-the-metric-fail): recentering the disk 750 km off-storm + drops the axisymmetry index by >= 0.15 absolute. If it does not, the + index does not measure centeredness and E1 is decoration. + E5 u8 palette quantization of the spiral samples adds reconstruction + error of at most one bucket width on top of E2's sampling error. + +Data: WB2 ERA5 6h msl, t=91246 (2021-06-15 12Z) — the SAME timestep and the +SAME storm (55.75N, 334.5E) as voxel_chess_probe.py E6 and ndarray's +examples/geostrophic_stencil.rs. cos(lat) spacing throughout. +""" +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T_IDX = 91246 +R_EARTH_KM = 6371.0 +R_DISK_KM = 1200.0 +GOLDEN_ANGLE = np.pi * (3.0 - np.sqrt(5.0)) + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=180).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +lat = fetch("latitude", "0").astype(np.float64).ravel() +p = fetch("mean_sea_level_pressure", f"{T_IDX}.0.0")[0].astype(np.float64) +phi = np.deg2rad(lat) + +# Storm center: deepest NH low in zonal-anomaly space (same rule as E6). +p_anom = p - p.mean(axis=1, keepdims=True) +nh = lat > 15 +ci, cj = np.unravel_index( + np.argmin(np.where(nh[:, None], p_anom, np.inf)), p.shape) +print(f"storm center: lat={lat[ci]:.2f} lon={cj * 0.25:.2f} " + f"p'={p_anom[ci, cj]:.0f} Pa") + + +def km_grid(ci_, cj_): + """(dx_km, dy_km, r_km) of every gridpoint relative to center (ci_, cj_).""" + lon = np.arange(p.shape[1]) * 0.25 + dlon = np.deg2rad((lon[None, :] - lon[cj_] + 180) % 360 - 180) + dphi = phi[:, None] - phi[ci_] + dx = R_EARTH_KM * np.cos(phi[ci_]) * dlon * np.ones_like(p) + dy = R_EARTH_KM * dphi * np.ones_like(p) + return dx, dy, np.hypot(dx, dy) + + +def bilinear(field, ci_, cj_, dx_km, dy_km): + """Sample field at km-offsets from center (ci_, cj_), bilinear, lon-wrapped.""" + dlat = np.rad2deg(dy_km / R_EARTH_KM) + dlon = np.rad2deg(dx_km / (R_EARTH_KM * np.cos(phi[ci_]))) + ri = ci_ - dlat / 0.25 # lat rows DESCEND 90..-90 + rj = (cj_ + dlon / 0.25) % p.shape[1] + i0 = np.clip(np.floor(ri).astype(int), 0, p.shape[0] - 2) + j0 = np.floor(rj).astype(int) % p.shape[1] + fi, fj = ri - i0, rj - j0 + j1 = (j0 + 1) % p.shape[1] + return (field[i0, j0] * (1 - fi) * (1 - fj) + + field[i0 + 1, j0] * fi * (1 - fj) + + field[i0, j1] * (1 - fi) * fj + + field[i0 + 1, j1] * fi * fj) + + +def axisym_index(ci_, cj_): + """1 - var(residual after removing the ring-mean profile)/var, in-disk.""" + _, _, r = km_grid(ci_, cj_) + disk = r <= R_DISK_KM + vals, rr = p[disk], r[disk] + rings = np.clip((rr / 50.0).astype(int), 0, 23) + prof = np.array([vals[rings == b].mean() if (rings == b).any() else np.nan + for b in range(24)]) + resid = vals - prof[rings] + return 1.0 - np.mean(resid ** 2) / vals.var(), disk + + +def spiral_pts(n): + """n sunflower/golden-angle points on the disk (equal-area: r = sqrt((k+0.5)/n)*R).""" + k = np.arange(n) + r = np.sqrt((k + 0.5) / n) * R_DISK_KM + th = k * GOLDEN_ANGLE + return r * np.cos(th), r * np.sin(th) + + +def grid_pts(n): + """EXACTLY n points on a uniform grid clipped to the disk. + + E2 is an EQUAL-BUDGET comparison, so this must return n, not "about n". + An earlier version returned every in-disk lattice point, which handed the + grid arm 80 samples against the spiral's 64 (and 293 vs 256, 1085 vs 1024) + — nearest-neighbour reconstruction improves with samples, so the arm being + compared was systematically advantaged and the verdict was not a controlled + comparison (codex P2 + coderabbit on PR #926, 2026-08-11). + + The lattice is grown until it holds at least n in-disk points, then the n + CLOSEST to the disk centre are kept — a deterministic, spatially even + subset with no RNG and no dependence on iteration order. + """ + side = int(np.ceil(np.sqrt(n * 4 / np.pi))) + while True: + g = (np.arange(side) + 0.5) / side * 2 * R_DISK_KM - R_DISK_KM + gx, gy = np.meshgrid(g, g) + rr = np.hypot(gx, gy) + m = rr <= R_DISK_KM + if m.sum() >= n: + break + side += 1 + gx, gy, rr = gx[m], gy[m], rr[m] + keep = np.argsort(rr, kind="stable")[:n] + return gx[keep], gy[keep] + + +def rand_pts(n, seed=7): + """n uniform random points on the disk, seeded for reproducibility.""" + rng = np.random.default_rng(seed) + r = np.sqrt(rng.uniform(0, 1, n)) * R_DISK_KM + th = rng.uniform(0, 2 * np.pi, n) + return r * np.cos(th), r * np.sin(th) + + +def recon_rmse(sx, sy, svals, dxg, dyg, disk): + """Nearest-sample reconstruction of the in-disk field (one rule, all arms).""" + tx, ty = dxg[disk], dyg[disk] + truth = p[disk] + # brute-force nearest over <=1024 samples x ~7e3 targets — fine + d2 = (tx[:, None] - sx[None, :]) ** 2 + (ty[:, None] - sy[None, :]) ** 2 + rec = svals[np.argmin(d2, axis=1)] + return float(np.sqrt(((rec - truth) ** 2).mean())) + + +def delta_entropy(q_u8): + """Shannon entropy (bits/sample) of the FIRST DIFFERENCES of a u8 sequence. + + Measures how predictable the sequence is in the given traversal order — the + quantity E3 compares between spiral order and raster order.""" + d = np.diff(q_u8.astype(np.int16)) + _, counts = np.unique(d, return_counts=True) + pr = counts / counts.sum() + return float(-(pr * np.log2(pr)).sum()) + + +out = {"store": B, "time_index": T_IDX, "R_disk_km": R_DISK_KM, + "center": {"lat": float(lat[ci]), "lon": float(cj * 0.25), + "p_anom_Pa": float(p_anom[ci, cj])}} + +# E1 + E4 -------------------------------------------------------------- +ax_storm, disk = axisym_index(ci, cj) +off_rows = int(round((750.0 / R_EARTH_KM) * 180 / np.pi / 0.25)) +ax_off, _ = axisym_index(ci + off_rows, cj) # 750 km south of the storm +out["E1_axisym_storm"] = ax_storm +out["E4_axisym_offset750km"] = ax_off +print(f"\nE1 axisymmetry (storm-centered): {ax_storm:.3f} (bar >= 0.70)") +print(f"E4 axisymmetry (750 km off): {ax_off:.3f} " + f"(bar: drop >= 0.15 -> {'FIRES' if ax_storm - ax_off >= 0.15 else 'fails'})") + +# E2 ------------------------------------------------------------------- +dxg, dyg, _ = km_grid(ci, cj) +out["E2_budgets"] = {} +for n in (64, 256, 1024): + row = {} + for name, (sx, sy) in [("spiral", spiral_pts(n)), ("grid", grid_pts(n)), + ("random", rand_pts(n))]: + vals = bilinear(p, ci, cj, sx, sy) + row[name] = {"n": len(sx), + "rmse_Pa": recon_rmse(sx, sy, vals, dxg, dyg, disk)} + out["E2_budgets"][str(n)] = row + print(f"E2 N~{n:4}: spiral={row['spiral']['rmse_Pa']:7.1f} Pa " + f"(n={row['spiral']['n']}) grid={row['grid']['rmse_Pa']:7.1f} " + f"(n={row['grid']['n']}) random={row['random']['rmse_Pa']:7.1f}") + +# E3 + E5 -------------------------------------------------------------- +n = 256 +sx, sy = spiral_pts(n) +vals = bilinear(p, ci, cj, sx, sy) +lo, hi = vals.min(), vals.max() +q = np.clip(np.floor((vals - lo) / (hi - lo) * 256), 0, 255).astype(np.uint8) +h_spiral = delta_entropy(q) +# raster order of the SAME samples: sort by (y-band, x) +order = np.lexsort((sx, np.floor(sy / 150.0))) +h_raster = delta_entropy(q[order]) +h_raw = delta_entropy(np.concatenate([[q[0]], q])) # abs values, reference +out["E3_delta_entropy_bits"] = {"spiral_order": h_spiral, + "raster_order": h_raster} +print(f"\nE3 delta-entropy (u8, bits/sample): spiral-order={h_spiral:.2f} " + f"raster-order={h_raster:.2f} (bar: spiral < raster)") + +deq = lo + (q.astype(np.float64) + 0.5) / 256 * (hi - lo) +rmse_q = recon_rmse(sx, sy, deq, dxg, dyg, disk) +bucket = (hi - lo) / 256 +out["E5_u8"] = {"rmse_Pa": rmse_q, "bucket_Pa": float(bucket), + "added_vs_f64_Pa": rmse_q - out["E2_budgets"]["256"]["spiral"]["rmse_Pa"]} +print(f"E5 u8-palette spiral recon: {rmse_q:.1f} Pa " + f"(f64 sampling: {out['E2_budgets']['256']['spiral']['rmse_Pa']:.1f}; " + f"bucket={bucket:.1f} Pa; added={out['E5_u8']['added_vs_f64_Pa']:.1f}, " + f"bar <= {bucket:.1f})") + +with open(pathlib.Path(__file__).with_name("sunflower_cyclone_probe.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote sunflower_cyclone_probe.json") diff --git a/probes/weather-p1/three_register_probe.json b/probes/weather-p1/three_register_probe.json new file mode 100644 index 00000000..c9c0d3cc --- /dev/null +++ b/probes/weather-p1/three_register_probe.json @@ -0,0 +1,1853 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "t0": 91246, + "palette": 256, + "provenance": { + "time_units": "hours since 1959-01-01", + "time_calendar": "proleptic_gregorian", + "grid_shape": [ + 721, + 1440 + ], + "chunk_shape": [ + 1, + 721, + 1440 + ], + "dtype": " rank -> inverse CDF + zc = zlo + ((b.astype(np.float64) + 0.5) / PALETTE) * (zhi - zlo) + rc = np.clip((np.tanh(zc) + 1.0) / 2.0, 0.0, 1.0) + recon = np.quantile(ref_sorted, rc) + return b, recon + + +# ---- data ------------------------------------------------------------------ + +print("fetching global MSLP, 2m temperature, 10m wind ...", flush=True) +p = fetch("mean_sea_level_pressure", f"{T0}.0.0")[0].astype(np.float64) +t2m = fetch("2m_temperature", f"{T0}.0.0")[0].astype(np.float64) +u10 = fetch("10m_u_component_of_wind", f"{T0}.0.0")[0].astype(np.float64) +v10 = fetch("10m_v_component_of_wind", f"{T0}.0.0")[0].astype(np.float64) +wind = np.hypot(u10, v10) + +pf = p.ravel() +ref = np.sort(pf) +lo, hi = pf.min(), pf.max() +print(f"\nglobal MSLP at t={T0}: n={pf.size}, " + f"range [{lo/100:.1f}, {hi/100:.1f}] hPa, " + f"median {np.median(pf)/100:.1f} hPa") +print(f" a UNIFORM byte over this range = {(hi-lo)/PALETTE:.1f} Pa/level") + +bands = { + "storm tail (bottom 1%)": (0.0, 0.01), + "lower shoulder (1-10%)": (0.01, 0.10), + "bulk (40-60%)": (0.40, 0.60), + "high tail (top 1%)": (0.99, 1.0), +} + +codes = {} +recons = {} +for name, fn in (("A_affine", lambda v: reg_a_affine(v, lo, hi)), + ("B_rank", lambda v: reg_b_rank(v, ref)), + ("C_fisher_rank", lambda v: reg_c_fisher_rank(v, ref))): + c, rc = fn(pf) + codes[name], recons[name] = c, rc + +print("\n=== reconstruction error in Pa, by percentile band (1 byte each) ===") +print(f"{'band':<26} {'n':>8} {'A affine':>12} {'B rank':>12} {'C fisher':>12}") +out_bands = {} +for name, (qa, qb) in bands.items(): + valo, vahi = np.quantile(pf, qa), np.quantile(pf, qb) + m = (pf >= valo) & (pf <= vahi) + row = {} + for reg in codes: + row[reg] = float(np.sqrt(np.mean((recons[reg][m] - pf[m]) ** 2))) + out_bands[name] = {"n": int(m.sum()), **row} + print(f"{name:<26} {int(m.sum()):>8} {row['A_affine']:>12.2f} " + f"{row['B_rank']:>12.2f} {row['C_fisher_rank']:>12.2f}") + +overall = {reg: float(np.sqrt(np.mean((recons[reg] - pf) ** 2))) for reg in codes} +print(f"{'OVERALL':<26} {pf.size:>8} {overall['A_affine']:>12.2f} " + f"{overall['B_rank']:>12.2f} {overall['C_fisher_rank']:>12.2f}") + +st = out_bands["storm tail (bottom 1%)"] +bulk = out_bands["bulk (40-60%)"] +a_flat = max(v[r] for v in out_bands.values() for r in ["A_affine"]) / \ + min(v["A_affine"] for v in out_bands.values()) +print("\n=== pre-registered verdicts ===") +print(f"R1 A flat across bands (max/min ratio {a_flat:.2f}, bar <= 1.5): " + f"{'PASS' if a_flat <= 1.5 else 'FAIL'}") +print(f"R2 B beats A in the bulk ({bulk['B_rank']:.2f} < {bulk['A_affine']:.2f}): " + f"{'PASS' if bulk['B_rank'] < bulk['A_affine'] else 'FAIL'}") +print(f"R3 B worse than A in the storm tail " + f"({st['B_rank']:.2f} > {st['A_affine']:.2f}): " + f"{'PASS' if st['B_rank'] > st['A_affine'] else 'FAIL'}") +print(f"R4 C beats B in the storm tail " + f"({st['C_fisher_rank']:.2f} < {st['B_rank']:.2f}): " + f"{'PASS' if st['C_fisher_rank'] < st['B_rank'] else 'FAIL'}" + f" [{st['B_rank']/max(st['C_fisher_rank'],1e-9):.1f}x tighter]") + +# ---- R5: cross-variable comparability -------------------------------------- +print("\n=== R5 cross-variable: does one u8 mean the same rarity everywhere? ===") +fields = {"mslp": pf, "t2m": t2m.ravel(), "wind10m": wind.ravel()} +rank_codes = {} +for fname, fv in fields.items(): + fref = np.sort(fv) + rr = _ranks(fv, fref) + rank_codes[fname] = np.clip(np.floor(rr * PALETTE), 0, PALETTE - 1).astype(np.uint8) + +# EVERY byte decides the verdict, not a sample of five. The first version of +# this bar probed [8, 64, 128, 192, 248] only (CodeRabbit, PR #926): a +# 5-of-256 sample cannot support a claim quantified over all 256, and the +# worst byte is precisely the one a sparse probe is most likely to miss. +# Five representative rows are still PRINTED, but the PASS/FAIL is computed +# over the full sweep. +all_spreads = [] +r5_rows = [] +for b in range(PALETTE): + fr = {f: float((rank_codes[f] <= b).mean()) for f in fields} + spread = max(fr.values()) - min(fr.values()) + all_spreads.append(spread) + r5_rows.append({"byte": b, **fr, "spread": spread}) +max_spread = max(all_spreads) +argworst = int(np.argmax(all_spreads)) +print(f" {'byte':>6} " + " ".join(f"{f:>12}" for f in fields) + " spread") +for b in [8, 64, 128, 192, 248, argworst]: + r = r5_rows[b] + tag = " <- WORST of all 256" if b == argworst else "" + print(f" {b:>6} " + " ".join(f"{r[f]:>12.4f}" for f in fields) + + f" {r['spread']:.5f}{tag}") +one_bucket = 1.0 / PALETTE +print(f"\nR5 max spread over ALL {PALETTE} bytes {max_spread:.5f} " + f"(worst at byte {argworst}) vs one bucket {one_bucket:.5f}: " + f"{'PASS' if max_spread <= one_bucket else 'FAIL'}") +print(" ABSOLUTE register: the same comparison is UNDEFINED — Pa, K and m/s") +print(" share no unit, so byte 128 of each denotes no common quantity at all.") + +# Provenance: pin what the store actually served, so a future re-run can tell +# "the numbers moved" from "the store moved" (CodeRabbit, PR #926). +tattrs = json.loads(op.open(B + "/time/.zattrs", timeout=90).read()) +pz = meta["mean_sea_level_pressure/.zarray"] +prov = {"time_units": tattrs.get("units"), + "time_calendar": tattrs.get("calendar"), + "grid_shape": list(pz["shape"][1:]), + "chunk_shape": list(pz["chunks"]), + "dtype": pz["dtype"], + "compressor": (pz.get("compressor") or {}).get("id"), + "n_points_per_field": int(pf.size)} +print("\nprovenance: " + ", ".join(f"{k}={v}" for k, v in prov.items())) + +json.dump({"store": B, "t0": T0, "palette": PALETTE, "provenance": prov, + "global_range_pa": [float(lo), float(hi)], + "uniform_step_pa": float((hi - lo) / PALETTE), + "bands": out_bands, "overall": overall, + "R5_cross_variable": r5_rows, + "R5_max_spread": float(max_spread), + "R5_one_bucket": one_bucket}, + open(pathlib.Path(__file__).with_name("three_register_probe.json"), "w"), indent=2) +print("\nwrote three_register_probe.json") diff --git a/probes/weather-p1/voxel_chess_probe.json b/probes/weather-p1/voxel_chess_probe.json new file mode 100644 index 00000000..6d774d5c --- /dev/null +++ b/probes/weather-p1/voxel_chess_probe.json @@ -0,0 +1,100 @@ +{ + "store": "https://storage.googleapis.com/weatherbench2/datasets/era5/1959-2022-6h-1440x721.zarr", + "time_index": 91246, + "preregistered": "E1>0.5, E2<0.5 (inversion = control), E3 mirrored, E4 |raw-u8|<=0.05, E5 corr>0.5 all bands, E6 rise-peak-decay", + "raw": { + "E1_nh_lows_ccw": 0.6356173111797284, + "n_nh_lows": 40806, + "E2_sh_lows_ccw": 0.4021482952191861, + "n_sh_lows": 70009, + "E3_nh_highs_ccw": 0.1514411607869552, + "n_nh_highs": 35631, + "E3_sh_highs_ccw": 0.7192818869510401, + "n_sh_highs": 76144, + "nh_corr_u": 0.3886449660159403, + "nh_corr_v": 0.4334605594972074, + "sh_corr_u": 0.8623853874270709, + "sh_corr_v": 0.7355679117824572 + }, + "palette_u8": { + "E1_nh_lows_ccw": 0.6338667820069204, + "n_nh_lows": 41616, + "E2_sh_lows_ccw": 0.39936678308771334, + "n_sh_lows": 70434, + "E3_nh_highs_ccw": 0.15619026814483097, + "n_nh_highs": 36622, + "E3_sh_highs_ccw": 0.7194711726511773, + "n_sh_highs": 76698, + "nh_corr_u": 0.3837072803428654, + "nh_corr_v": 0.42140908709596736, + "sh_corr_u": 0.8456117083342138, + "sh_corr_v": 0.7089358037738284 + }, + "E4_max_popfrac_dev": 0.004749107357875787, + "E6_rankine": { + "center_lat": 55.75, + "center_lon": 334.5, + "profile": [ + { + "r_mid_km": 75, + "vt_mean": 10.234071543181926, + "n": 161 + }, + { + "r_mid_km": 225, + "vt_mean": 12.190355700609066, + "n": 482 + }, + { + "r_mid_km": 375, + "vt_mean": 12.163085321612744, + "n": 822 + }, + { + "r_mid_km": 525, + "vt_mean": 12.40477837840327, + "n": 1130 + }, + { + "r_mid_km": 675, + "vt_mean": 10.932072086933266, + "n": 1456 + }, + { + "r_mid_km": 825, + "vt_mean": 10.033053294346114, + "n": 1800 + }, + { + "r_mid_km": 975, + "vt_mean": 9.2468191601141, + "n": 2114 + }, + { + "r_mid_km": 1125, + "vt_mean": 6.655831785528652, + "n": 2430 + }, + { + "r_mid_km": 1275, + "vt_mean": 3.864982062335603, + "n": 2766 + }, + { + "r_mid_km": 1425, + "vt_mean": 1.206079088789252, + "n": 3076 + } + ], + "peak_ring": 3, + "cyclonic_at_peak": true, + "monotone_tol_ms": 0.05, + "rises_to_peak": true, + "decays_after_peak": true, + "net_rise_ms": 2.1707068352213437, + "net_decay_ms": 11.198699289614018, + "net_rise_exceeds_tol": true, + "net_decay_exceeds_tol": true, + "rises_then_decays": true + } +} \ No newline at end of file diff --git a/probes/weather-p1/voxel_chess_probe.py b/probes/weather-p1/voxel_chess_probe.py new file mode 100644 index 00000000..27379d73 --- /dev/null +++ b/probes/weather-p1/voxel_chess_probe.py @@ -0,0 +1,232 @@ +"""EXPLORATORY — 'volumetrisches Voxel-Schach': rotation physics as BITBOARD ops, +raw vs u8-palette substrate. (Operator frame, 2026-08-11.) + +The claim under test, stated as chess machinery: threshold masks over the voxel +grid are BITBOARDS, and the rotation physics of pressure systems is decidable by +POPCOUNTS over mask intersections — Stockfish's popcount(attacks & targets), +with weather masks. If the u8 palette substrate carries the physics, the same +popcounts on quantized fields give the same verdicts. + +PRE-REGISTERED, before the run (NOT independently audited — measurements, not +an EV; promotion to an EV requires the adversarial audit gate per plan section 8): + E1 NH lows rotate counterclockwise: pop(L & Zpos & NH)/pop(L & NH) > 0.5 + E2 SH lows rotate clockwise: pop(L & Zpos & SH)/pop(L & SH) < 0.5 + -- E1 vs E2 is the built-in two-sided control: the SAME statistic must + INVERT across the equator. A wiring bug that ignores hemisphere + cannot produce the inversion. + E3 Highs mirror lows in both hemispheres (anticyclonic). + E4 The u8-palette arm reproduces every E1-E3 verdict, fractions within + 0.05 of raw. + E5 Geostrophic move-generation: corr(u_g, u) and corr(v_g, v) > 0.5 in + each hemisphere band (|lat| in 20..70), raw AND palette. + E6 Rankine ('Gluecksrad'): around the deepest NH low, azimuthal-mean + tangential wind is positive (cyclonic), rises from the core (torque / + solid-body zone), peaks, then decays outward (momentum zone). + +Data: WeatherBench2 ERA5 6-hourly 0.25 deg, t=91246 (2021-06-15 12Z). +Fields: mean_sea_level_pressure, 10m u, 10m v. cos(lat) zonal spacing per the +EV-1 audit lesson; |lat| < 15 deg masked (f -> 0, geostrophy undefined). +""" +import json +import pathlib +import urllib.request + +import numcodecs +import numpy as np + +B = ("https://storage.googleapis.com/weatherbench2/datasets/era5/" + "1959-2022-6h-1440x721.zarr") +T_IDX = 91246 +OMEGA, R_EARTH, RHO0 = 7.2921e-5, 6.371e6, 1.225 + +op = urllib.request.build_opener(urllib.request.ProxyHandler({})) +meta = json.loads(op.open(B + "/.zmetadata", timeout=90).read())["metadata"] + + +def fetch(var, key): + """Fetch and decode one zarr chunk from the WB2 store.""" + za = meta[f"{var}/.zarray"] + raw = op.open(f"{B}/{var}/{key}", timeout=180).read() + dec = numcodecs.get_codec(za["compressor"]).decode(raw) + return np.frombuffer(dec, dtype=np.dtype(za["dtype"])).reshape(za["chunks"]) + + +lat = fetch("latitude", "0").astype(np.float64).ravel() +p = fetch("mean_sea_level_pressure", f"{T_IDX}.0.0")[0].astype(np.float64) +u = fetch("10m_u_component_of_wind", f"{T_IDX}.0.0")[0].astype(np.float64) +v = fetch("10m_v_component_of_wind", f"{T_IDX}.0.0")[0].astype(np.float64) +print(f"fetched: lat[{lat[0]:.1f}..{lat[-1]:.1f}], p/u/v {p.shape}") + +phi = np.deg2rad(lat) +y = R_EARTH * phi # meridional coordinate (m) +dlam = 2 * np.pi / p.shape[1] +dx = R_EARTH * np.cos(phi) * dlam # zonal spacing per row (m) +f_cor = 2 * OMEGA * np.sin(phi) + + +def ddx(a): + """Periodic central zonal derivative with per-row cos(lat) spacing.""" + return (np.roll(a, -1, 1) - np.roll(a, 1, 1)) / (2 * dx[:, None]) + + +def ddy(a): + """Meridional derivative along the latitude axis (per-metre, via the y metric).""" + return np.gradient(a, y, axis=0) + + +def physics(pf, uf, vf): + """Derive (relative vorticity, zonal pressure anomaly, geostrophic u, geostrophic v). + + v_g = (1/rho*f) k x grad p, so u_g takes -dp/dy and v_g takes +dp/dx. + Returned as one tuple so the raw and palette arms run the IDENTICAL chain.""" + zeta = ddx(vf) - ddy(uf) + p_anom = pf - pf.mean(axis=1, keepdims=True) + ug = -(1 / (RHO0 * f_cor[:, None])) * ddy(pf) + vg = (1 / (RHO0 * f_cor[:, None])) * ddx(pf) + return zeta, p_anom, ug, vg + + +def quant_u8(a, q=(0.4, 99.6)): + """The shipped palette scheme: linear 256 buckets over the percentile + window (mirrors helix quantize.rs), decoded at bucket centers.""" + lo, hi = np.percentile(a, q) + idx = np.clip(np.floor((a - lo) / (hi - lo) * 256), 0, 255) + return lo + (idx + 0.5) / 256 * (hi - lo) + + +def popfrac(mask_num, mask_den): + """Fraction of `mask_den` cells that also satisfy `mask_num`, plus the denominator. + + Returns NaN (not 0) for an empty denominator so an absent population is + visibly absent rather than silently reported as a zero rate.""" + d = int(mask_den.sum()) + return (int((mask_num & mask_den).sum()) / d if d else float("nan")), d + + +def board_eval(zeta, p_anom, tag): + """The chess move: popcounts over bitboard intersections.""" + band = np.abs(lat)[:, None] >= 15.0 + nh = (lat[:, None] > 0) & band & np.ones_like(p_anom, bool) + sh = (lat[:, None] < 0) & band & np.ones_like(p_anom, bool) + sig = p_anom[band[:, 0], :].std() + lows, highs, zpos = p_anom < -sig, p_anom > sig, zeta > 0 + r = {} + r["E1_nh_lows_ccw"], r["n_nh_lows"] = popfrac(zpos, lows & nh) + r["E2_sh_lows_ccw"], r["n_sh_lows"] = popfrac(zpos, lows & sh) + r["E3_nh_highs_ccw"], r["n_nh_highs"] = popfrac(zpos, highs & nh) + r["E3_sh_highs_ccw"], r["n_sh_highs"] = popfrac(zpos, highs & sh) + print(f" [{tag}] lows: NH ccw-frac={r['E1_nh_lows_ccw']:.3f} " + f"(n={r['n_nh_lows']}) SH ccw-frac={r['E2_sh_lows_ccw']:.3f} " + f"(n={r['n_sh_lows']})") + print(f" [{tag}] highs: NH ccw-frac={r['E3_nh_highs_ccw']:.3f} " + f"SH ccw-frac={r['E3_sh_highs_ccw']:.3f}") + return r + + +def geo_corr(ug, vg, u_obs, v_obs, tag): + """Correlate the geostrophic estimate against the OBSERVED winds passed in. + + The observed fields are explicit parameters: an earlier version closed over + the module-level raw `u`/`v`, so the palette arm compared palette-derived + geostrophic winds against RAW observations — a hybrid, not the + pre-registered palette result (codex/coderabbit on PR #926, 2026-08-11). + """ + r = {} + for name, lo_b, hi_b in [("nh", 20, 70), ("sh", -70, -20)]: + m = (lat[:, None] >= lo_b) & (lat[:, None] <= hi_b) \ + & np.ones_like(u_obs, bool) + cu = np.corrcoef(ug[m], u_obs[m])[0, 1] + cv = np.corrcoef(vg[m], v_obs[m])[0, 1] + r[f"{name}_corr_u"], r[f"{name}_corr_v"] = float(cu), float(cv) + print(f" [{tag}] geostrophic {name.upper()}: corr(u_g,u)={cu:.3f} " + f"corr(v_g,v)={cv:.3f}") + return r + + +out = {"store": B, "time_index": T_IDX, "preregistered": + "E1>0.5, E2<0.5 (inversion = control), E3 mirrored, E4 |raw-u8|<=0.05, " + "E5 corr>0.5 all bands, E6 rise-peak-decay"} + +print("\n== RAW arm ==") +zeta, p_anom, ug, vg = physics(p, u, v) +out["raw"] = {**board_eval(zeta, p_anom, "raw"), **geo_corr(ug, vg, u, v, "raw")} + +print("\n== u8-PALETTE arm (p, u, v each quantized to 256 buckets) ==") +p8, u8, v8 = quant_u8(p), quant_u8(u), quant_u8(v) +zeta8, p_anom8, ug8, vg8 = physics(p8, u8, v8) +out["palette_u8"] = {**board_eval(zeta8, p_anom8, "u8"), + **geo_corr(ug8, vg8, u8, v8, "u8")} + +# E4: substrate fidelity of the popcount verdicts +keys = ["E1_nh_lows_ccw", "E2_sh_lows_ccw", "E3_nh_highs_ccw", + "E3_sh_highs_ccw"] +devs = {k: abs(out["raw"][k] - out["palette_u8"][k]) for k in keys} +out["E4_max_popfrac_dev"] = max(devs.values()) +print(f"\n E4 max |raw - u8| popcount-fraction deviation: " + f"{out['E4_max_popfrac_dev']:.4f}") + +# E6: Rankine profile around the deepest NH low +print("\n== E6 Rankine ('Gluecksrad') around the deepest NH low ==") +nh_rows = lat > 15 +# E6 says "the deepest NH low", so select on MSLP itself. Selecting on the +# zonal ANOMALY picks a different point and silently changes the wind profile +# and the serialized center (coderabbit on PR #926, 2026-08-11). +p_nh = np.where(nh_rows[:, None], p, np.inf) +ci, cj = np.unravel_index(np.argmin(p_nh), p_nh.shape) +print(f" center: lat={lat[ci]:.2f} lon={cj * 0.25:.2f} " + f"p'={p_anom[ci, cj]:.0f} Pa") +lon = np.arange(p.shape[1]) * 0.25 +dlon = np.deg2rad((lon[None, :] - lon[cj] + 180) % 360 - 180) +dphi = phi[:, None] - phi[ci] +dx_m = R_EARTH * np.cos(phi[ci]) * dlon * np.ones_like(p) +dy_m = R_EARTH * dphi * np.ones_like(p) +r_km = np.hypot(dx_m, dy_m) / 1e3 +alpha = np.arctan2(dy_m, dx_m) +v_t = -u * np.sin(alpha) + v * np.cos(alpha) # >0 = counterclockwise +prof = [] +for r0 in range(0, 1500, 150): + ring = (r_km >= r0) & (r_km < r0 + 150) + prof.append({"r_mid_km": r0 + 75, "vt_mean": float(v_t[ring].mean()), + "n": int(ring.sum())}) + print(f" r={r0 + 75:>5} km v_t={prof[-1]['vt_mean']:+7.2f} m/s " + f"(n={prof[-1]['n']})") +vts = [q["vt_mean"] for q in prof] +pk = int(np.argmax(vts)) +# A Rankine profile must actually RISE to the peak and DECAY after it. The +# earlier test only required an interior maximum plus a lower final value, +# which accepted a profile that DECREASED then rose to the peak — the +# committed run did exactly that (12.190 -> 12.163 m/s before the 525 km peak) +# and still reported true (coderabbit on PR #926, 2026-08-11). TOL absorbs +# ring-to-ring sampling noise; the monotonicity is what is being asserted. +TOL = 0.05 # m/s, ~0.4% of the observed peak +rise_ok = all(vts[i + 1] >= vts[i] - TOL for i in range(pk)) +decay_ok = all(vts[i + 1] <= vts[i] + TOL for i in range(pk, len(vts) - 1)) +# ...and the SHAPE must be non-trivial. A TOL-tolerant monotone test accepts a +# FLAT arm — [1, 2, 2] passes `decay_ok` because every step falls by <= TOL — +# so a profile that peaks and then plateaus would report "decays" (coderabbit +# on PR #926, second pass). Require a net change exceeding the same tolerance +# the monotonicity is allowed to absorb, on BOTH arms: the rise side has the +# identical hole and fixing only the one that was pointed at would leave a +# half-vacuous assertion behind. +net_rise_ok = vts[pk] > vts[0] + TOL +net_decay_ok = vts[-1] < vts[pk] - TOL +out["E6_rankine"] = {"center_lat": float(lat[ci]), "center_lon": float(lon[cj]), + "profile": prof, "peak_ring": pk, + "cyclonic_at_peak": vts[pk] > 0, + "monotone_tol_ms": TOL, + "rises_to_peak": bool(rise_ok), + "decays_after_peak": bool(decay_ok), + "net_rise_ms": float(vts[pk] - vts[0]), + "net_decay_ms": float(vts[pk] - vts[-1]), + "net_rise_exceeds_tol": bool(net_rise_ok), + "net_decay_exceeds_tol": bool(net_decay_ok), + "rises_then_decays": bool(0 < pk < len(vts) - 1 + and rise_ok and decay_ok + and net_rise_ok and net_decay_ok)} +print(f" peak at ring {pk} ({prof[pk]['r_mid_km']} km): torque zone inside, " + f"momentum zone outside -> rises_then_decays=" + f"{out['E6_rankine']['rises_then_decays']}") + +with open(pathlib.Path(__file__).with_name("voxel_chess_probe.json"), "w") as fh: + json.dump(out, fh, indent=2) +print("\nwrote voxel_chess_probe.json")