osm slab: hydrate from S3 onto the persistent volume, checksum-pinned

claude-notes/plans/2026-08-08-osm-soa-cockpit-wiring.md

@@ -721,10 +721,13 @@ and Berlin's densest hip tile holds 3,844, so a z13 tile is bounded above by
 Berlin-class bake. (The bound is itself slack: it assumes 64 adjacent
 maximum-density tiles, where the one measured z14 tile holds 15,016.)
 
-- [ ] Build the hip-cell representative form for overview zooms and compare it
+- [x] Build the hip-cell representative form for overview zooms and compare it
       against the stride at equal budget — coverage, and what a user actually
       loses. That comparison is what promotes `OVERVIEW_ROW_BUDGET` from
       CONJECTURE, and it is the "dynamic compression bucket threshold" thread.
+      **Done in Phase 5** — generalised to a per-tile prefix depth rather than
+      fixed-at-hip (hip is the `zz=16` case); stride keeps 14/316 isolated
+      features at z8 where the cell form keeps 316/316.
 - [ ] Re-measure `CITY_ZOOM_FLOOR` against a denser extract than Berlin before
       treating 13 as anything but a policy floor.
 
@@ -812,8 +815,10 @@ tiles. The old 5,000 cap masked it. It is survivable at the load the evidence
 supports (measured working below), so it is recorded rather than fixed in this
 pass.
 
-- [ ] `render()` is O(tiles x markers) — rebuild incrementally, or diff, rather
-      than clearing `#tiles` on every tile arrival.
+- [x] `render()` is O(tiles x markers) — rebuild incrementally, or diff, rather
+      than clearing `#tiles` on every tile arrival. **Done in the same phase**
+      — `paintFeatures()` + `drawnCells`, measured 1,550,957 -> 64,707 appends
+      (23.97x -> 1.00x), wall-clock 1.72x, behaviour identical.
 - [ ] The viewport bound is the honest one; `OVERVIEW_ROW_BUDGET` is its
       per-tile share at one window size. A budget derived from the actual
       tiles-in-view count would not drift with window size.
@@ -913,3 +918,90 @@ in `osm_features.rs` is grounded in — so the real envelope is at least 1.6x
 what those constants assume. Not acted on: this was at city zoom with different
 fetch timing, and a budget should not be widened on one incidental observation.
 It does mean `OVERVIEW_ROW_BUDGET = 3_000` is conservative rather than tight.
+
+## Phase 6 — S3 → volume01 hydration (2026-08-11)
+
+**Operator: "railway has exactly the same S3 variables like you."** That single
+fact retired the blocker. Phase 4 was recorded as *"verifiable only on deploy"*
+because `/volume01` does not exist in this sandbox — but the volume is a
+**path**, and the part that actually needed proving (do these credentials reach
+that bucket, and does a 1.29 GiB round trip work) is verifiable right here.
+`OSM_SLAB_CACHE_DIR` substitutes for the mount, so everything except the
+literal mount point is exercised locally.
+
+**Operator, on shape:** *"S3 → lancedb → volume01"*, *"persist the lance file
+to volume01"*, *"it's just a convenience to persist lance across rebuilds."*
+The third message is the one that fixes the semantics: **the volume is a cache,
+S3 is the source of truth.** Deleting the volume costs a re-download and
+nothing else.
+
+### Three assumptions I had written down, all wrong
+
+The Phase 4 sketch claimed this needed ~5 new dependencies (`reqwest` + rustls,
+`sha2`, `hex`, `chrono`, `tokio` sync), hand-rolled SigV4, and a sync→async
+conversion of `open_slab()`. Checked rather than repeated:
+
+| claimed | measured |
+|---|---|
+| needs `reqwest` + a new HTTP stack | **`object_store` v0.13.2 is already in the graph** via `datafusion`/`lance`; adding it direct pulls no new transitive tree |
+| needs hand-rolled SigV4 | **`AmazonS3Builder::from_env()`** reads it all — and `aws_endpoint_url` is an accepted alias for the endpoint key (`builder.rs:498`), so a non-AWS endpoint needs no special casing |
+| `open_slab()` must become async | **`main()` is already `async`** — hydrating there, before the listener binds, leaves the read path untouched and stops the first request paying for the download |
+
+Net new direct deps: `object_store` (already in lockfile at this exact
+version), `sha2`, `hex`, `futures` — all already resolved.
+
+### The artifact question, and why it did not block
+
+`openstreetmap-website-rs/Cargo.toml` says in its own header: *"no lance, no
+datafusion: the ABI row is a byte layout, not an engine."* The bake emits raw
+`.soa` + `.books` and has no Lance writer — and `lance_abi` was **removed**
+from that repo (`a27b06a`) for doing a full-table materialising read. Lance
+lives on the **consumer** side, where `cockpit-server` already has the stack.
+
+So the hydration layer is written **format-agnostic**: it moves named objects
+from a prefix onto the volume and verifies them. Whether the artifact travels
+as `.soa` or as a Lance dataset is a separate decision it does not prejudge.
+That is also why this could ship without resolving it.
+
+Worth carrying from the `lance_abi` removal, because it names the correct
+shape if the Lance path is taken later: *"this crate already has fragment.rs
+(64k-row fragments, so a tile prefix lands in a fragment) and
+`RowSlab::tile_range`. The module used neither."* `object_store` exposes
+`get_range(location, Range<u64>)` — the range read that removal asked for.
+
+### What shipped
+
+`crates/cockpit-server/src/osm_slab_hydrate.rs`, called once from `main()`:
+
+1. `OSM_SLAB_PATH` set and a real file ⇒ use it. Local dev never touches S3.
+2. Else `AWS_S3_BUCKET_NAME` + (`OSM_SLAB_CACHE_DIR` | `RAILWAY_VOL`) ⇒
+   for each of `berlin.soa`, `berlin.books`: cache hit **verified by
+   checksum**, else stream from S3 → hash while writing → rename into place.
+3. Else `None` ⇒ the endpoint answers 503 exactly as before. Absent
+   configuration is not an error.
+
+**Checksum pinning, on the cache hit too.** `SHA256SUMS` is fetched from the
+same prefix; a missing entry is a refusal, not a warning. Verifying on a hit
+(not only after download) guards the half-written file left by a container
+killed mid-download — precisely the case a "we already have it" check waves
+through. Downloads land on `.part` and are renamed only after the hash
+matches, so a kill leaves nothing that looks complete; the hit-side check
+stays anyway because the volume outlives this code.
+
+### Bucket conventions — followed, not invented
+
+The bucket already had 50+ objects, all MedCare-rs, with a clear shape:
+`<repo>/bakes/<version>/<artifact>` plus `SHA256SUMS`, and Lance datasets
+shipped as `.lance.tar` (`all-lanes.lance.tar`, 378 MB). q2's upload follows
+it exactly at `q2/bakes/berlin-v1/` and touches nothing under `MedCare-rs/`.
+
+| key | bytes |
+|---|---|
+| `q2/bakes/berlin-v1/berlin.soa` | 1,292,826,624 |
+| `q2/bakes/berlin-v1/berlin.books` | 58,763,338 |
+| `q2/bakes/berlin-v1/SHA256SUMS` | 156 |
+
+`sha256(berlin.soa) = cbf5989ab45bc921d8a85fdbdb71c8e5029cd904a3d230a898c2b5eb81d7ebe7`
+
+- [x] `OSM_SLAB_PATH` gains an S3 sibling: hydrate → volume, then mmap.
+- [x] Checksum-pin the object; no unverified-fetch path.

crates/cockpit-server/Cargo.toml

@@ -70,6 +70,15 @@ osm-soa-bake = { path = "../../../openstreetmap-website-rs" }
 # memory). Pinned to the same 0.5 line osm-soa-bake itself uses, so the
 # dependency tree carries one memmap2 version, not two.
 memmap2 = "0.5"
+# S3 -> volume hydration for the OSM slab (`osm_slab_hydrate`). `object_store`
+# is already in the graph via lance/datafusion at this exact version, so this
+# adds no new transitive tree — and its `AmazonS3Builder::from_env()` reads the
+# AWS_* vars directly, so there is no hand-rolled SigV4 here.
+object_store = { version = "0.13.2", features = ["aws"] }
+sha2 = "0.10"
+hex = "0.4"
+futures = "0.3"
+
 
 # ── Quarto rendering pipeline ───────────────────────────────────────
 # pampa parses .qmd, quarto-core renders, deno_core executes JS/TS cells

crates/cockpit-server/src/main.rs

@@ -43,6 +43,7 @@ mod osint_classview;
 mod osm_tiles;
 mod osm;
 mod osm_features;
+mod osm_slab_hydrate;
 
 // ── Embed the Vite build at compile time ─────────────────────────────────────
 // The cockpit/ directory is built by `cd cockpit && npm run build` which
@@ -165,9 +166,36 @@ struct McpError {
 #[tokio::main]
 async fn main() {
     tracing_subscriber::fmt()
-        .with_env_filter("cockpit_server=info,tower_http=info")
+        // The binary target is `q2-cockpit`, so `module_path!()` on this
+        // crate's own logs starts with `q2_cockpit` — NOT `cockpit_server`,
+        // which is the *package* name. The filter said `cockpit_server=info`,
+        // so every `tracing::*` call in this binary was silently discarded.
+        // Found when the S3 hydrate ran correctly and logged nothing: on a
+        // deploy that would have been 503s with no explanation. Both names are
+        // listed so a future rename of either cannot silence it again, and
+        // `RUST_LOG` wins when set, which is the operational control a deploy
+        // needs.
+        .with_env_filter(
+            tracing_subscriber::EnvFilter::try_from_default_env().unwrap_or_else(|_| {
+                tracing_subscriber::EnvFilter::new(
+                    "q2_cockpit=info,cockpit_server=info,tower_http=info",
+                )
+            }),
+        )
         .init();
 
+    // Hydrate the OSM slab BEFORE the listener binds: S3 is the source of
+    // truth, `$RAILWAY_VOL` is a cache that survives container rebuilds. Doing
+    // it here rather than lazily keeps `osm_features::open_slab` synchronous —
+    // it still just mmaps `OSM_SLAB_PATH` — and stops the first request paying
+    // for a 1.29 GiB download. A `None` here is not an error: the endpoint
+    // answers 503 exactly as it did before, and local dev sets OSM_SLAB_PATH
+    // directly and never reaches S3.
+    if let Some(path) = osm_slab_hydrate::ensure_slab_local().await {
+        // SAFETY: single-threaded startup, before any task or listener exists.
+        unsafe { std::env::set_var("OSM_SLAB_PATH", &path) };
+    }
+
     let (tx, _rx) = broadcast::channel::<SseEvent>(256);
     let scene_state = shader_stream::new_scene_state();
     let state = Arc::new(AppState { tx, scene_state });