fix(docs): correct four stale version pins + make the toolchain comment un-staleable

.claude/board/EPIPHANIES.md

@@ -1,3 +1,84 @@
+## 2026-08-10 — E-CLAUDE-MD-KEY-DEPENDENCIES-WENT-STALE-AND-PROPAGATED-A-WRONG-PIN-INTO-A-PLAN-1
+
+**Status:** FINDING `[G]` (verified against `Cargo.lock` + every crate manifest).
+
+`CLAUDE.md` § Key Dependencies — **the mandatory first read for every session** —
+carried four stale pins, dated *"Verified against Cargo.lock 2026-06-14"*, which
+pre-dates the lance-9 sweep (`b2b08b07`, 2026-08-05):
+
+| CLAUDE.md said | tree actually has |
+|---|---|
+| `lance = "=7.0.0"` | **`=9.0.0`** |
+| `lance-linalg = "=7.0.0"` | **`=9.0.0`** |
+| `lancedb = "=0.30.0"` | **`=0.33.0`** |
+| `datafusion = "53"` | **`54`** |
+
+**The propagation is the finding, not the staleness.** The wrong `datafusion 53`
+travelled: stale `CLAUDE.md` → restated in-session → written into
+`weather-substrate-poc-v2.md` §6 (PR #915, merged) → *and* mischaracterized there,
+because `datafusion 54.1.0` in `Cargo.lock` was filed as suspicious drift when 54 is
+in fact our correct, deliberate, **MEASURED** direct pin
+(`.claude/plans/lance9-datafusion54-upgrade-probe-v1.md`, 2026-08-05). **A POC
+session trusting either document would have pinned lance 7 against a lance-9 tree
+and failed to build.**
+
+**⊘ Self-correction, same hour (operator-caught).** The first version of THIS entry
+then called `datafusion 53.1.0` a *"residual transitive."* Wrong, and more dangerous
+than the original error: it frames a required dependency as cruft, inviting a future
+session to collapse `Cargo.lock` to a single major and silently break the `delta`
+feature. **Both majors are REQUIRED and the dual state is documented upstream:**
+`deltalake-core 0.32.4` pins `datafusion 53.1.0` (+ `datafusion-datasource`,
+`datafusion-physical-expr-adapter` at 53.1.0); the lance family pins 54. Cargo
+permits the coexistence precisely because they are different semver majors. It
+resolves only when deltalake moves to DF 54 — not by any action here.
+
+**The meta-lesson:** "this version looks unexpected" has three possible causes —
+*stale doc*, *real drift*, and **legitimate multi-major coexistence**. I reached for
+the first two and skipped the third, twice in opposite directions. Before labelling
+a version anomalous, read *who requires it*; a lockfile entry with a live requirer
+is a constraint, never a leftover.
+
+The lockstep discipline itself was never violated: every manifest carries exact
+`=9.0.0` / `=0.33.0`. **Only the docs lagged** — which is the more dangerous
+failure, because the code compiles and the doc is what a new session reads first.
+
+**Rule:** a version block asserting *"verified against Cargo.lock <date>"* is a
+claim with an expiry. When a dependency sweep lands, the sweep's PR must update
+every doc that restates its pins — the same-commit board-hygiene rule, applied to
+version facts. Grep for the old version string across `.claude/` and `CLAUDE.md`
+before closing a bump.
+
+Corrected in this PR: `CLAUDE.md` (all four, + `lance-index`, + a `rust` line
+pointing at `rust-toolchain.toml` as authoritative), the plan's §6 with a dated
+`⊘ CORRECTION` block (append-only, not a silent edit).
+
+## 2026-08-10 — E-A-COMMENT-THAT-RESTATES-A-PINNED-VALUE-GOES-STALE-EVERY-BUMP-1
+
+**Status:** FINDING `[G]` (two recurrences in the same file, both in git history).
+
+`rust-toolchain.toml`'s comment restated the pinned version in prose. It has
+therefore been **wrong twice**:
+
+1. said `1.94.1` after a bump → fixed by `10f87fb6`
+   (*"docs(toolchain): fix stale 1.94.1 comment"*);
+2. said `1.95.0` after the bump to `1.97.1` (`b2b08b07`) — **the identical failure,
+   ~3 months later**, because the earlier fix corrected the *value* and left the
+   *structure* that guarantees the value goes stale.
+
+A bump edits `channel = "…"` and nobody re-reads the paragraph below it. The
+one-off correction resets the clock; it does not stop it.
+
+**Structural fix (this PR):** the comment **no longer restates the version at all**
+— it points at the `channel` line as authoritative — and carries an **append-only
+bump log** instead, one line per bump with its commit/PR and reason. Appending
+cannot contradict; re-narrating always can.
+
+**Generalizes beyond this file:** any prose that duplicates a machine-readable
+value has a half-life. Either derive it, or make the duplicate append-only. (Same
+shape as `E-CLAUDE-MD-KEY-DEPENDENCIES-WENT-STALE-…-1` above, found the same hour —
+one is a config comment, the other a doc table, both duplicated a pin and both went
+wrong.)
+
 ## 2026-08-10 — E-JC-AND-NDARRAY-BOTH-SHIP-A-RELIABILITY-BATTERY-WITH-DIFFERENT-DEGENERATE-CONTRACTS-1
 
 **Status:** FINDING `[G]` (source-verified 2026-08-10, both files read).

.claude/plans/weather-substrate-poc-v2.md

@@ -271,19 +271,40 @@ session needs them added.
 
 ```
 rust        1.97.1        (rust-toolchain.toml)
-lance       9.0.0         lance-encoding 9.0.0 · lance-linalg 9.0.0
-lancedb     0.33.0
+lance       =9.0.0        lance-encoding 9.0.0 · lance-linalg =9.0.0 · lance-index =9.0.0
+lancedb     =0.33.0
 arrow       58.3.0
-datafusion  53
+datafusion  54            ← NOT 53; see the correction below
 ```
 
-The lance family moves in **exact lockstep**; a bump is one deliberate PR, never a
-drift. Two drift items recorded during verification, neither blocking:
+The lance family moves in **exact lockstep** (`=X.Y.Z` in every manifest, verified);
+a bump is one deliberate PR, never a drift.
 
-1. `Cargo.lock` resolves **datafusion 53.1.0 *and* 54.1.0** — plausibly a transitive
-   pulling 54, but the lockstep discipline exists to prevent exactly this shape.
-2. `rust-toolchain.toml`'s channel is `1.97.1` while its own comment still says
-   *"Pinned to 1.95.0"* — the comment did not follow the bump.
+> **⊘ CORRECTION (2026-08-10, same day, before any POC work started; corrected
+> twice).** This section's first version said **`datafusion 53`** and filed the
+> presence of 54.1.0 in `Cargo.lock` as suspicious drift. **That was backwards.**
+> `datafusion = "54"` is our direct pin in every crate manifest, and `lance` /
+> `lancedb` / `lance-index` / `lance-datafusion` all require 54; the move is
+> recorded and **MEASURED** (`lance9-datafusion54-upgrade-probe-v1.md`, 2026-08-05).
+>
+> **Second correction (operator):** the first fix then called 53 a *"residual
+> transitive"* — also wrong, and more dangerous, because it invites someone to
+> collapse the lock to one version. **Both majors are REQUIRED.**
+> `deltalake-core 0.32.4` pins `datafusion 53.1.0` (+ `-datasource`,
+> `-physical-expr-adapter`) upstream, backing the optional `delta` feature. Two
+> semver majors coexisting is the *correct, documented* state — not a defect to
+> tidy. It lifts only when deltalake moves to DF 54.
+>
+> **Root cause, worth more than the fact:** `CLAUDE.md`'s Key Dependencies block —
+> the mandatory first read for every session — was itself stale (`lance = "=7.0.0"`,
+> `lancedb = "=0.30.0"`, `datafusion = "53"`, dated 2026-06-14, pre-dating the
+> lance-9 sweep). The wrong pin propagated *from* that block *into* this plan. A POC
+> session that trusted either would have pinned lance 7 against a lance-9 tree and
+> failed to build. Both are corrected in the same PR as this note.
+
+The `rust-toolchain.toml` comment drift noted in the first version is also fixed
+there — **structurally**: the comment no longer restates the channel value (which is
+what made it go stale twice), and carries an append-only bump log instead.
 
 ---
 

CLAUDE.md

@@ -1150,12 +1150,34 @@ cd crates/lance-graph-python && maturin develop
 ## Key Dependencies
 
 ```toml
-# Verified against Cargo.lock 2026-06-14. The lance family moves in lockstep at =7.0.0 (PR #445).
-arrow = "58"
-datafusion = "53"
-lance = "=7.0.0"          # exact-pinned: lancedb 0.30.0 transitively requires lance =7.0.0
-lance-linalg = "=7.0.0"
-lancedb = "=0.30.0"       # 0.30 → lance 7 → object_store 0.13.2 (0.29 → lance 6 → object_store 0.12)
+# Verified against Cargo.lock + every crate manifest 2026-08-10. The lance family
+# moves in EXACT lockstep — currently =9.0.0 (lance 7 was the 2026-06-14 state,
+# superseded by the lance-9 sweep, b2b08b07 / PR #896 arc).
+arrow = "58"          # 58.3.0 resolved; unmoved by the lance-9 sweep
+datafusion = "54"     # OUR direct pin, in every crate that DEPENDS on it
+                      # (lance-graph, -catalog, -callcenter, -python, holograph),
+                      # and the sub-crates move with it: datafusion-common / -expr
+                      # / -sql / -functions-aggregate are all "54". lance /
+                      # lancedb / lance-index / lance-datafusion require 54 too.
+                      # (Grep caveat: `lance-graph-planner/Cargo.toml` has
+                      # `datafusion = []` under [features] — a FEATURE NAME, not a
+                      # version pin. It is not a counter-example.)
+                      #
+                      # ⚠ BOTH MAJORS ARE REQUIRED — do NOT "fix" Cargo.lock to one.
+                      # `datafusion 53.1.0` (+ -datasource, -physical-expr-adapter)
+                      # also resolves because `deltalake-core 0.32.4` pins DF 53
+                      # upstream, and it backs the optional `delta` feature. Two
+                      # semver majors legitimately coexist; collapsing them breaks
+                      # `delta`. 53 is NOT a stale leftover — it is a live upstream
+                      # constraint that lifts only when deltalake moves to DF 54.
+                      # Probe: .claude/plans/lance9-datafusion54-upgrade-probe-v1.md
+lance = "=9.0.0"          # exact-pinned: lancedb 0.33.0 requires lance =9.0.0
+lance-linalg = "=9.0.0"
+lance-index = "=9.0.0"
+lancedb = "=0.33.0"       # the lance-9 pairing. NOTE: lancedb 0.36 is the PyPI
+                          # package, versioned independently — the Rust crate
+                          # tops out at 0.33.0 (probe §1).
+rust = "1.97.1"           # rust-toolchain.toml is authoritative; see its bump log
 ndarray = { path = "../../../ndarray" }  # AdaWorldAPI fork, default, optional fallback
 nom = "7.1"
 snafu = "0.8"

rust-toolchain.toml

@@ -1,19 +1,31 @@
 [toolchain]
 channel = "1.97.1"
-# Pinned to 1.95.0 (bumped from 1.94.1 in PR #367, 2026-05-13). Aligns
-# with bevy (edition 2024 → 1.95 MSRV) and ndarray (bumped in parallel
-# per LATEST_STATE 2026-05-13). 1.95 added clippy lints
-# (`clippy::manual_checked_ops`, `clippy::unnecessary_sort_by`,
-# `clippy::len_zero` cleanups) that PR #367 + #368 closed across the
-# workspace. Never auto-track `stable` — bump explicitly when a future
-# version is reviewed and workspace clippy passes clean.
+# The pinned version is the `channel` line ABOVE — deliberately not restated
+# in this comment. Restating it is what made this comment wrong twice: it said
+# "1.94.1" until `10f87fb6` corrected it, then said "1.95.0" after the bump to
+# 1.97.1, because a bump edits `channel` and leaves the prose behind. Keep the
+# rationale here and the version there; append to the log, never re-narrate.
+#
+# Bump log (append one line per bump):
+#   1.94.0 → 1.94.1   matches sibling repos                        (f638fa27)
+#   1.94.1 → 1.95.0   bevy edition-2024 MSRV + ndarray in parallel (PR #367,
+#                     2026-05-13); #367 + #368 closed the lints 1.95 added
+#                     (`manual_checked_ops`, `unnecessary_sort_by`, `len_zero`)
+#   1.95.0 → 1.97.1   lance 9 / lancedb 0.33 / DataFusion 54 sweep (b2b08b07,
+#                     2026-08-05); toolchain leg measured ALONE on the old pins
+#                     first, so a toolchain failure could not be mistaken for a
+#                     dependency failure —
+#                     `.claude/plans/lance9-datafusion54-upgrade-probe-v1.md`
+#
+# Never auto-track `stable` — bump explicitly when a future version is
+# reviewed and workspace clippy passes clean.
 #
 # rustfmt + clippy are mandatory CI components and must be present
 # whenever the pinned channel is installed. Without this, `cargo fmt`
 # under `actions-rust-lang/setup-rust-toolchain@v1` resolves to the
-# pinned `1.94.x` channel but rustfmt is only installed for `stable`
+# pinned channel but rustfmt is only installed for `stable`
 # (the action's default), producing:
-#   error: 'cargo-fmt' is not installed for the toolchain '1.94.x-...'
+#   error: 'cargo-fmt' is not installed for the toolchain '<pinned>-...'
 # Declaring components here makes rustup install them as part of the
 # toolchain itself, which is also developer-friendly: a fresh
 # `rustup show` in this workspace materializes a complete CI-equivalent