Skip to content

plan(weather): substrate weather POC v1 — encoder bake-off first #914

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AdaWorldAPI merged 1 commit into from
Aug 10, 2026
Merged

plan(weather): substrate weather POC v1 — encoder bake-off first #914

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .claude/board/EPIPHANIES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,19 @@
## 2026-08-08 — E-THREE-NAMED-PROBES-ARE-ONE-MEASUREMENT — F-1, helix's unrun fidelity gate, and the weather-encoder question are the SAME probe

**Status:** FINDING `[G]` on the identity (three docs, one measurement shape), `[H]` on the outcome (unrun). Surfaced by the 5-agent recon behind `.claude/plans/weather-substrate-poc-v1.md`.

Three probes have been carried independently, in three places, each named as blocking:

1. **F-1 (codebook fidelity)** — `substrate-unification-thesis.md` §4.1: does hierarchical-4⁴ (a byte's nibbles = its centroid's ancestry) preserve rank-distance vs flat k-means-256? Named in the OGAR canon, **un-run**. Its KILL: if it fails, "prefix-is-ancestry" degrades from *code* to *router* and a large fraction of the `[H]`/`[S]` map collapses at once.
2. **helix's own gate** — `crates/helix/KNOWLEDGE.md:338-343`: fidelity vs certified `Base17Fz` is **CONJECTURE, probe NOT RUN**, gate ≥0.9980 Pearson.
3. **The weather-encoder question** — does a 48-bit `Signed360` preserve *synoptic* distance? (the gate the whole retrieval/analogue lane stands on).

**They are one measurement:** *does a quantized hierarchical code rank like the continuous quantity it encodes?* Only the dataset differs. And `bgz17` already ships **both arms** — `Palette` (flat k-means 256, `palette.rs:112`) and `HierarchicalPalette` (16 coarse × 16 leaves with `coarse_is_ancestor_of`, `palette.rs:214,451-493`) — plus measured anchors on other data (`PaletteResolution`: Full256 ρ=0.992 / Half128 ρ=0.965 / Quarter64 ρ=0.738, `palette.rs:543-582`). So the *instrument* has been sitting in-tree, built, the entire time; what was missing was a dataset whose ground truth the workspace did not author.

**Why weather is the right dataset for it, not merely a convenient one.** Every prior corroboration in this arc has been internal — four sessions converging, probes designed by the same people holding the thesis (thesis §5 warns explicitly that convergence can be shared blind spots as easily as shared truth). Weather supplies a **physical** ground-truth distance (field RMSE) that no one here chose, at a scale (10⁶ points) that exercises the hierarchy rather than a toy. Running the bake-off on weather answers all three questions with one number — and a **negative is thesis-relevant, not weather-relevant**: it would demote prefix-ancestry workspace-wide, which is precisely why it must be run before more is built on top.

**Consequence for planning:** do not schedule these as three separate probes in three repos. One bake-off (`D-WX-5`), arms C vs D carrying F-1, arms A/B carrying helix's gate, all under Jirak-bounded significance (`I-NOISE-FLOOR-JIRAK` — grid fields are weakly dependent by construction; classical Berry-Esseen is wrong here). Mandatory anti-vacuity per `E-VACUOUS-ASSERTION-IS-THE-HOUSE-STYLE-1`: include a shuffled-codebook encoder that **must** fail — a bake-off where every arm passes has measured nothing.

Comment on lines +1 to +16

@coderabbitai coderabbitai Bot Aug 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟠 Major | ⚡ Quick win

Restore newest-first board ordering.

The new 2026-08-08 entry appears before the existing 2026-08-09 entry at Line [17]. Move the complete 2026-08-08 entry below the complete 2026-08-09 entry. Preserve the existing historical content.

As per coding guidelines, board files must remain append-only. Based on learnings, entries must remain ordered newest-first.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/board/EPIPHANIES.md around lines 1 - 16, Move the complete
2026-08-08 entry in EPIPHANIES.md below the existing 2026-08-09 entry,
preserving all entry content and maintaining append-only history with
newest-first ordering.

Sources: Coding guidelines, Learnings

## 2026-08-09

## E-A-RECONCILED-HEAD-IS-NOT-A-PUBLICATION-1 (2026-08-09)
Expand Down
4 changes: 4 additions & 0 deletions .claude/board/INTEGRATION_PLANS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
## 2026-08-08 — weather-substrate-poc-v1 (PLAN; encoder bake-off first)

Plan: `.claude/plans/weather-substrate-poc-v1.md`. Operator ask: weather POC over official data + helix + "stockfish acceleration" + "deepseek", choosing the best discovery instrument. **Grounded on a 5-agent parallel recon of `f675a0ff`** (file:line throughout), which found the POC is **composition + measurement, not construction** — and produced six corrections now carried in the plan's §0: the **~1.7° helix figure is NOT in the tree** (documented is sub-degree, 0.45°/0.35°; `Signed360` azimuth u16 ⇒ 0.0055°/step); the 125ms/233ms + 3DGS 500ms numbers are **operator-reported, out-of-tree** (zero weather work exists in lance-graph; the board's `measure-64k-axes` is 65,536 *mailbox owners*, not weather — do not conflate); **`kanban_actor`'s actor/ack/tick surface was deleted 2026-08-05** (`cycle_driver` behind feature `cycle-driver` is the complete path); **`soa_to_lance` is an example binary, not a lib fn** (the pub surface is `lance_graph::dev_s3_env`); **NNUE/"stockfish" is already graded prior art** (`stockfish-nnue-as-perturbation-cascade.md` — reuse its mechanism-vs-rhyme ledger, incl. the fence that Walsh-Hadamard has NO NNUE analog); **`0x0F` is `Geo`, not free** (free: 0x03–0x06). "DeepSeek" in this workspace = **GRPO/RLVR** (arXiv 2402.03300), not the LLM — Stage 3, gated. **The POC choice: a Stage-0 encoder bake-off** (7 arms + a deliberately-broken control over 1 yr Z500 at 0.25° = 1,038,240 pts; Spearman ρ/ICC vs physical RMSE under **Jirak** bounds, gate ρ≥0.98). Chosen because it is the only stage whose failure kills everything downstream, the cheapest decisive one, and it **simultaneously discharges F-1** (bgz17 `HierarchicalPalette` 16×16 ancestry vs flat `Palette`) **and helix's own unrun ≥0.9980 probe** (`helix/KNOWLEDGE.md:338-343`). **No new repo** — new workspace-EXCLUDED `crates/weather-poc` on the `perturbation-sim` template (zero-dep default, heavy deps behind off-by-default features); GRIB2 never enters Rust (Python-side slab conversion in the operator's `ecmwf-opendata` fork); no classid mint needed for Stage 0. Storage: `/volume01` hot + S3 via the existing `dev_s3_env` var set. Deliverables D-WX-0..6, all gated on D-WX-5. Doc-only. Rides a PR on jirak.

## 2026-08-09 — persistence-artifact-backed-commit v1 — RATIFIED (Phase A implemented; B–F planned) — main thread

The canonical persistence contract, replacing the #911 cycle-persistence model.
Expand Down
Loading