deploy: make the cold-boot hydrate survive the container healthcheck

[AdaWorldAPI]

The root Dockerfile already builds and deploys q2-cockpit — no new deploy surface was needed. But the S3 hydrate merged in #112 would have failed the very deploy it exists to enable.

⚠ The defect, mine

ensure_slab_local().await (main.rs:194) blocks TcpListener::bind (main.rs:346). On a cold boot that is ~60–90 s of transfer before the port opens — and HEALTHCHECK had --interval=30s --timeout=3s with no --start-period, so Docker probes immediately and the default 3 retries mark the container unhealthy inside ~90 s.

The unhealthy window overlapped the hydrate window exactly: the deploy would have failed while doing precisely what it was supposed to do.

Fixed with --start-period=600s. Failures during the start period don't count, so this only widens the window for a legitimately slow first start — and the cold path runs once per volume, not once per deploy, since persisting across rebuilds is the volume's whole purpose.

It is not masking a hang — verified, not asserted

Pointed at a nonexistent prefix:

11:25:12 INFO  osm slab: resolving from S3 (cold boot transfers ~1.35 GB …)
               bucket=… prefix=q2/bakes/does-not-exist dir=…/probevol/osm
11:25:13 ERROR osm slab: SHA256SUMS not readable — 404 … in 765.929422ms
               (server still starts and serves)

A misconfiguration costs 0.8 s and names the exact URL. Only a real transfer is slow.

The boot line above is also new, and emitted before the transfer. Without it a cold start logs nothing for 60–90 s, which is indistinguishable from a hang for whoever is watching a deploy — the same observability gap as the q2_cockpit filter bug, one layer up.

Checked rather than assumed

	finding
toolchain	Dockerfile pins 1.94.0, but rust-toolchain.toml pins nightly-2026-04-28 and rustup honours the toml — the build already resolves to a toolchain satisfying ogar-vocab's ≥1.95 floor. The apparent mismatch is not one.
TLS	object_store resolves to rustls (hyper-rustls), not native-tls — the S3 leg adds no OpenSSL requirement to the runtime image.
features	Deploy builds --features embed-cockpit,planner. planner is already default; embed-cockpit only gates include_dir for static assets. Neither touches the hydrate path.

What Railway needs

variable	purpose
AWS_ENDPOINT_URL AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION	read by AmazonS3Builder::from_env()
AWS_S3_BUCKET_NAME	set explicitly via with_bucket_name
RAILWAY_VOL	volume root; <vol>/osm/ holds the cache

Volume needs ≥ 2 GB — 1.35 GB of artifacts plus headroom for a .part during re-fetch, which transiently doubles one file. OSM_SLAB_S3_PREFIX overrides the default q2/bakes/berlin-v1; OSM_SLAB_CACHE_DIR overrides the volume root and is what made this testable off-Railway.

Verification

cargo test --bin q2-cockpit — 90 passed, 0 failed, 3 ignored.

⚠️ CI on this fork is skipped, not passing (if: github.repository == 'quarto-dev/q2').

Still unverified: the literal /volume01 mount and a real container build (no Docker daemon here). First deploy tests both — and thanks to the tracing fix and the new boot line, a failure will now say why instead of producing silent 503s.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NMeiLmtDKhomJNSo2ecbJw

---

Generated by Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved container startup reliability during initial data hydration by allowing up to 10 minutes before health-check failures are counted.
  • Added startup visibility for hydration progress, including storage location and expected cold-boot or warm-cache timing.

• Documentation

  • Added deployment guidance covering hydration configuration, environment settings, cache sizing, and verification steps.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 15ffbe80-84f8-450c-9e77-3c4bcc8ccbde

📥 Commits

Reviewing files that changed from the base of the PR and between 1e30bd3 and 358d459.

📒 Files selected for processing (3)

• Dockerfile
• claude-notes/plans/2026-08-08-osm-soa-cockpit-wiring.md
• crates/cockpit-server/src/osm_slab_hydrate.rs

---

📝 Walkthrough

Walkthrough

The runtime Docker health check now waits 600 seconds before counting failures. ensure_slab_local logs S3 hydration details, and deployment notes document configuration, timing, cache sizing, and verification.

Changes

Cold-start hydration readiness

Layer / File(s)	Summary
Hydration readiness and observability crates/cockpit-server/src/osm_slab_hydrate.rs, Dockerfile, claude-notes/plans/...	ensure_slab_local logs S3 resolution and timing details. The Docker health check uses a 600-second start period. Deployment notes document hydration behavior, configuration, cache sizing, and verification.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• AdaWorldAPI/q2#41: Introduced the Dockerfile health check and /health endpoint extended by this change.
• AdaWorldAPI/q2#112: Added the S3 hydration flow extended with startup logging and health-check timing.

Suggested reviewers: claude

Poem

A rabbit watched the cold boot glow,
While slab data crossed the snow.
Six hundred seconds, probes wait,
Logs mark the cache’s warming state.
The cockpit wakes, hydrated and bright.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_5a26b3a5-1733-4a42-9cad-9db7144b5be1)