Skip to content

graph/cycle_sink: the concrete Lance-backed WalSink — storage-proven cognitive-cycle sink (§I.6) - #911

Merged
AdaWorldAPI merged 5 commits into
mainfrom
claude/medcare-rs-continue-ufsazd
Aug 9, 2026
Merged

graph/cycle_sink: the concrete Lance-backed WalSink — storage-proven cognitive-cycle sink (§I.6)#911
AdaWorldAPI merged 5 commits into
mainfrom
claude/medcare-rs-continue-ufsazd

Conversation

@AdaWorldAPI

@AdaWorldAPI AdaWorldAPI commented Aug 9, 2026

Copy link
Copy Markdown
Owner

What

Closes the gap persist_sink deliberately left ("this module builds NO concrete Lance sink — compile+test green ≠ storage proven"): lance_graph::graph::cycle_sink::LanceCycleSink is a concrete implementation of lance_graph_planner::persist_sink::WalSink over the official Lance 9 insert path (Dataset::write / Dataset::append — the same InsertBuilder transaction machinery every Lance writer uses). No bespoke ledger, no acknowledgement protocol, no parallel replay system: Lance's own manifest/version chain IS the WAL.

The §I.6 invariant made physical

64k thoughts read sealed Vn → write-side temporal deinterlace →
one detached cycle batch → ONE official Lance commit →
exactly one real DatasetVersion Vn+1 → no open or partial visibility
  • One durable append per cycle, all-or-nothing.
  • The epistemic fence, both halves. Pre-commit: the store's current version must equal the cycle's sealed predecessor base (empty store ⇒ head DatasetVersion(0)), else refused with nothing written. Post-commit: the published version must be exactly base + 1 — Lance auto-resolves append-append conflicts, so a foreign interleaved writer surfaces as a loud timeline anomaly instead of silently shifting the cycle ↔ version identity reads derive.
  • Zero sidecar state. A per-cycle frame row seals the cycle ↔ version mapping inside the same atomic commit; versions() and scan_sealed() read the reopened store, and sealed_version = base_version + 1 is an identity the commit path verifies, never assumes.
  • Order is a write-side property. Landings are stored in the already-deinterlaced DetachedCycleBatch::freeze order and scanned back with scan_in_order(true) — this sink never sorts on read.

Domain-0x09 witness contract (module doc)

The patient SoA at classid domain 0x09 is the only place patient reasoning is written to Lance, so the store's schema is witness-focused and maximally rich: payload carries the 512-byte canonical EpisodicWitness node (visited ontology addresses, executed crosswalk mappings, exact RO/ontology edge ids, supporting/contradicting/missing observations, NARS truth + confidence, differential branches), whose edges point INTO the immutable domain-0x03 ontology address space. A cycle takes ontology immutability for granted for its representation window (base_version names the sealed predecessor it read) and never restates ontology content — the sealed versioning is a reflection of the thinking; downstream displays read the sealed version, never a live recomputation.

Gates

  • 6 tokio tests, every guarantee proven against a reopened dataset (fresh sink instance + fresh Dataset::open, never an in-memory echo): seal survives restart; stale base fenced with nothing written (version chain, landings, timeline all checked untouched); sequential cycles chain V1→V2→V3 with strictly-after filtering; a zero-landing cycle advances the timeline only; an empty store reads empty (DatasetNotFound is a state, not an error); move-nullability + 512-byte payload byte-exact round-trip. The real Lance version chain is cross-checked against the returned DatasetVersion in-test.
  • cargo clippy -p lance-graph --lib clean; cargo fmt clean; gated on the default-on planner feature (the trait lives in the optional planner dep).
  • Board hygiene (LATEST_STATE inventory + PR_ARC_INVENTORY entry) in the same commit.

The branch also carries the two pre-existing post-#879 follow-up commits that were already on the remote branch (pre-commit failure contract correction + MedCare plan consolidation), merged with current main (lance-graph-supervisor re-checked green post-merge).

Deferred

The MedCare consumer arc (production drive_cohort_thoughts caller, witness-seal of real observation + CURIE/ELK/RO nodes, views reading the sealed version) — next PR, in MedCare-rs. recover_and_apply wiring against this sink in a production driver. Object-store URI smoke (only local paths exercised).

🤖 Generated with Claude Code

https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp


Generated by Claude Code

Summary by CodeRabbit

  • New Features

    • Added durable storage for cycle history with ordered recovery after restarts.
    • Preserved optional workflow moves, witness information, and final cycle images.
    • Added safeguards against stale writes and inconsistent version ordering.
  • Bug Fixes

    • Improved failure handling so incomplete writes are discarded and safely regenerated.
    • Clarified recovery behavior for pre-commit and post-commit failures.
  • Documentation

    • Added guidance describing the active MedCare integration path, persistence guarantees, validation plans, and current implementation boundaries.

claude added 4 commits August 2, 2026 13:02
…ive plan

Six-agent read-only investigation + unshallowed git history (5 grafted
roots -> 4162 commits) established that lance-graph holds TWO MedCare
lineages, not one incomplete runtime:

- LIVE (consumer-pull): MedCare-rs medcare-bridge -> vendored
  lance-graph-ogar -> MedcareBridge = UnifiedBridge<HealthcarePort> ->
  OGAR canonical Health codebook 0x0901..0x090C. Bridge migration
  COMPLETED at ddb6c84 (2026-06-21); deprecated alias only. Contract
  codebook mirror verified in sync slot-for-slot; the 7-alias-vs-12-slot
  gap is intentional (harvest mints, no OGIT entity).
- DEAD (host-side scaffold): modules/medcare/manifest.yaml ->
  CallcenterSupervisor -> MedcareConsumerActor, frozen since birth
  2026-05-13. Manifest compile-time parsed but runtime-orphaned (one
  caller: a test); entity codes / action_capabilities / message_type
  discarded pre-codegen; medcare_policy nonexistent; StubConsumerActor
  hard-coded; Dispatch rejected before any child; MedCareActor /
  MedCareMessage exist in NEITHER repository.

New ACTIVE plan medcare-consumer-pull-thinking-proof-v1: prove one real
medical thought over the live consumer-pull path (HealthcarePort
classids -> cognitive-shader-driver + real MailboxSoA -> owner_adapter
cast -> the #879 sparse cycle loop), falsifiers F1-F4 including the
currently-absent Healthcare fail-closed unknown-actor test. ogar-obo
(MONDO/HPO/Uberon/PATO, verified real, zero consumer edges) is an
optional slice. Older MedCare plans classified (completed / dormant-
decision-required / stale-but-unmarked); open decisions carried, not
blocked on (OQ-2 retention 2190/3650, Ueberweisung/Anamnese canon gap,
dead-lineage retire-vs-revive, .grok lineage, NoopAuditSink default).

Documentation-only: no Rust, no tests, no manifests changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
…t to deterministic regeneration; re-home the proof to the live consumer

Round-2 grain-of-salt realignment (operator-ruled), post-#879-merge.

1) Pre-commit failure contract. The authoritative rule is deterministic
   regeneration, not retained-batch retry:

     sealed Vn + unchanged Kanban task + deterministic computation
       = the same provisional intent on the next sweep

   Commit fails before Vn+1 exists -> publish nothing, mutate no owner,
   advance no watermark, discard provisional slots / held moves /
   planning results, rerun the unchanged task from Vn.

   - SealFailure{casts} reclassified: OPTIONAL retry cache /
     implementation convenience only — never the correctness mechanism,
     never a provisional-planning ledger; dropping it is always sound.
   - recover_fleet doc-pinned as COMMITTED-HISTORY recovery ONLY
     (Vn+1 exists, application/restart interrupted); explicitly
     separated from ordinary pre-commit write failure — no shared state.
   - HeldIntent doc-pinned as within-success scheduling convenience,
     discarded on a failed seal, regenerated by the next thought pass.
   - NEW authoritative falsifier
     pre_commit_failure_discards_everything_and_regenerates_from_vn:
     derive cycle C deterministically from Vn, inject commit failure,
     DROP the SealFailure cache, assert no version/phase/watermark
     change, rerun the unchanged task from Vn, assert the same SEMANTIC
     sparse cycle regenerates, allow commit, assert exactly one Vn+1 and
     one advance per represented owner. Object identity of the first
     heap batch deliberately not asserted.
   - The prior byte-identical-retry test demoted to an optional-cache
     probe (secondary, convenience path).

   20 cycle_driver tests green; clippy + fmt clean. Latency figures in
   review prose are operator-provided measurements, not
   workspace-reproduced benchmarks; their values are not restated.

2) Proof re-homed (medcare-consumer-pull-thinking-proof-v1 section 4):
   primary home = MedCare-rs (the live composition root); lance-graph
   contributes only genuinely-missing GENERIC seams — no MedCare-shaped
   host adapter here (that would rebuild the dead lineage in miniature).
   Hard requirement: the proof must invoke the existing
   cognitive-shader-driver + MailboxSoA operational unit; shade_owner
   only as the driver's existing downstream gate, never a substitute.
   Trace-and-report obligation added; F1 strengthened to a
   discriminating driver outcome.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
…cognitive-cycle sink

Closes the gap persist_sink deliberately left ('this module builds NO
concrete Lance sink'): LanceCycleSink implements
lance_graph_planner::persist_sink::WalSink over the official Lance 9
insert path (Dataset::write / Dataset::append — the same InsertBuilder
transaction machinery every Lance writer uses). No bespoke ledger, no
acknowledgement protocol, no parallel replay system: Lance's own
manifest/version chain IS the WAL.

The §I.6 invariant made physical: one detached cycle batch → ONE
official Lance commit → exactly one real DatasetVersion base+1, with
the epistemic fence in both halves — pre-commit head==base (a stale
base is refused with nothing written; an empty store's head is
DatasetVersion(0)) and post-commit published==base+1 (Lance
auto-resolves append-append conflicts, so a foreign interleaved writer
surfaces as a loud timeline anomaly instead of silently shifting the
cycle↔version identity reads derive). The per-cycle frame row seals
the cycle↔version mapping inside the same atomic commit, so the
coarse timeline survives restart with zero sidecar state. Order is a
write-side property: landings are stored in the already-deinterlaced
freeze order and scanned back with scan_in_order(true) — never sorted
on read.

Domain-0x09 witness contract (module doc): the patient SoA is the
ONLY place patient reasoning is written to Lance, so the store is
witness-focused and maximally rich — payload carries the 512-byte
canonical EpisodicWitness node whose edges point INTO the immutable
domain-0x03 ontology address space; the cycle takes ontology
immutability for granted for its representation window via
base_version, and the sealed versioning is a reflection of the
thinking.

Gated on the default-on 'planner' feature (the trait lives in the
optional planner dep). 6 tokio tests, every guarantee proven against
a REOPENED dataset (fresh sink instance + fresh Dataset::open): seal
survives restart; stale base fenced writing nothing; sequential
cycles chain V1→V2→V3 with strictly-after filtering; a zero-landing
cycle advances the timeline only; an empty store reads empty
(DatasetNotFound is a state, not an error); move-nullability +
payload byte-exact round-trip. Board hygiene (LATEST_STATE +
PR_ARC_INVENTORY) in this same commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp

# Conflicts:
#	.claude/board/LATEST_STATE.md
@cursor

cursor Bot commented Aug 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_4b5e21b3-bb5e-4a3d-9322-d5b1c43725d3)

@coderabbitai

coderabbitai Bot commented Aug 9, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

This PR adds a Lance-backed WalSink for cycle persistence, updates pre-commit failure recovery to use deterministic regeneration, and documents the active MedCare consumer-pull proof path and related repository findings.

Changes

Cycle persistence and recovery

Layer / File(s) Summary
Lance cycle sink implementation
crates/lance-graph/src/graph/cycle_sink.rs, crates/lance-graph/src/graph/mod.rs, .claude/board/LATEST_STATE.md, .claude/board/PR_ARC_INVENTORY.md
Adds the feature-gated LanceCycleSink, Arrow schema, atomic Lance writes, version fencing, ordered recovery, timeline reads, durable images, and tests for restart, ordering, empty cycles, stale bases, nullable moves, and witness payloads.
Pre-commit failure recovery
crates/lance-graph-supervisor/src/cycle_driver.rs, .claude/board/LATEST_STATE.md
Defines failed seals as disposable provisional state. Retries regenerate from unchanged Vn; committed-history recovery remains separate. Tests verify no publication or state mutation before regeneration.
MedCare integration proof documentation
.claude/plans/medcare-consumer-pull-thinking-proof-v1.md, .claude/board/INTEGRATION_PLANS.md, .claude/board/EPIPHANIES.md
Documents the live consumer-pull lineage, dormant host-side scaffold, proof flow, falsifiers, unresolved lineage findings, and current implementation status.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related PRs

Poem

A rabbit stores cycles in rows neat and bright,
Version fences keep every landing in sight.
Failed seals return to Vn for a new run,
MedCare traces the path when the proof has begun.
Lance holds each witness through day and night.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the concrete Lance-backed WalSink added in the main changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e081ba32b8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/lance-graph/src/graph/cycle_sink.rs Outdated
Comment thread crates/lance-graph/src/graph/cycle_sink.rs
Comment thread crates/lance-graph/src/graph/cycle_sink.rs Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.claude/board/EPIPHANIES.md:
- Around line 871-879: Move the entire `2026-08-02 —
E-TWO-MEDCARE-LINEAGES-THE-LIVE-ONE-PULLS-THE-DEAD-ONE-HOSTS-1` entry to the
topmost newest-first ledger position, ahead of all older entries and the other
existing `2026-08-02` item. Preserve the exact entry content and retain the
relative order of every other entry.

In @.claude/board/INTEGRATION_PLANS.md:
- Around line 147-169: Move the complete 2026-08-02
medcare-consumer-pull-thinking-proof v1 entry to the head of the ledger in
INTEGRATION_PLANS.md, before the existing 2026-08-02 entry. Preserve the entry’s
content exactly and do not edit any older entries.

In @.claude/board/LATEST_STATE.md:
- Around line 6-8: Define and document reconciliation for durable post-commit
version mismatches in commit_cycle(): do not treat PersistError::Write as
blindly retryable after rows may have been appended; serialize commits or return
an explicit ambiguous/committed result, and provide a rule to detect and
reconcile an existing frame-row cycle-to-version mapping before retrying the
same SealFailure.casts. Update the `#879` entries at .claude/board/LATEST_STATE.md
lines 6-8 and 150-152 and .claude/board/PR_ARC_INVENTORY.md lines 38-40; all
three sites require the reconciliation behavior to be recorded.

In @.claude/plans/medcare-consumer-pull-thinking-proof-v1.md:
- Line 29: Update the three Markdown diagram code fences in the document to
specify the text language identifier, including the fences near the referenced
sections, so each opening fence uses text and satisfies markdownlint MD040.

In `@crates/lance-graph/src/graph/cycle_sink.rs`:
- Around line 204-227: Enforce the canonical 512-byte payload size in the
landing-write loop around payload.append_value, rejecting any s.payload whose
length is not exactly 512 bytes before persistence. Update existing
short-payload fixtures to 512 bytes and add a test verifying invalid landing
payloads are rejected while preserving the canonical node row stride and 16-byte
edge reservation.
- Around line 369-408: Prevent interleaved cycle commits from appending under a
shifted DatasetVersion: update the cycle sink commit path at
crates/lance-graph/src/graph/cycle_sink.rs:369-408 to use an expected-version
conditional append, or return a distinct committed-anomaly result containing the
actual published version. Update
crates/lance-graph-supervisor/src/cycle_driver.rs:26-50 and :1016-1117 so this
result is routed through committed-history handling rather than classified as an
uncommitted CycleError::Seal; preserve the actual published version when
reporting the anomaly.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 01554bd8-d079-4c0f-86d7-aa25f719acf6

📥 Commits

Reviewing files that changed from the base of the PR and between 8997948 and e081ba3.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • .claude/board/EPIPHANIES.md
  • .claude/board/INTEGRATION_PLANS.md
  • .claude/board/LATEST_STATE.md
  • .claude/board/PR_ARC_INVENTORY.md
  • .claude/plans/medcare-consumer-pull-thinking-proof-v1.md
  • crates/lance-graph-supervisor/src/cycle_driver.rs
  • crates/lance-graph/src/graph/cycle_sink.rs
  • crates/lance-graph/src/graph/mod.rs

Comment on lines +871 to +879
## 2026-08-02 — E-TWO-MEDCARE-LINEAGES-THE-LIVE-ONE-PULLS-THE-DEAD-ONE-HOSTS-1 — MedCare is integrated through consumer-pull; the host-side actor/manifest scaffold froze on its birth day

**Status:** FINDING (six-agent read-only investigation + unshallowed git history, HEAD `71d1db1`). **Confidence:** High — every claim carries a commit sha or file:line; contradicting prose ledgers were checked against code. Plan: `.claude/plans/medcare-consumer-pull-thinking-proof-v1.md`.

**The finding.** lance-graph does NOT contain "one incomplete MedCare runtime" — it contains **two lineages**. The LIVE one is consumer-pull: MedCare-rs `medcare-bridge` → vendored `lance-graph-ogar` → `MedcareBridge = UnifiedBridge<HealthcarePort>` (collapse commit `ddb6c840`, 2026-06-21; deprecated alias since `10e717d4`) → OGAR canonical Health codebook (`0x0901–0x090C`; the contract mirror verified in sync slot-for-slot; the 7-alias-vs-12-slot gap is INTENTIONAL per OGAR `ports.rs:598-606`). The DEAD one is the host-side scaffold born in the sprint-7 wave (2026-05-13, `927788e9`/`3f67aed6`) and **never functionally touched again**: `modules/medcare/manifest.yaml` is compile-time parsed (`contract/build.rs`) into `MANIFEST_METADATA` whose ONLY caller is a test; its richest fields (entity codes 100–105, all five `action_capabilities`, `actor.message_type`) are parsed then DISCARDED pre-codegen; `rbac_policy: medcare_policy` names a nonexistent symbol; `CallcenterSupervisor::spawn_consumer_actor` unconditionally spawns `StubConsumerActor` (`supervisor.rs:369`); `DispatchToG` returns `DispatchNotImplemented` before any child actor for every non-Health envelope; `MedcareConsumerActor` has ZERO constructors repo-wide; and the manifest's declared `MedCareActor`/`MedCareMessage` exist in NEITHER repository (verified against the MedCare-rs sibling directly).

**Load-bearing distinctions this pins:** (a) TWO same-named `UnifiedBridge` types — ogar's `<P: PortSpec>` resolver (NO authorize method) vs callcenter's `<B: NamespaceBridge>` authorize/audit wrapper — never composed, and currently UNCOMPOSABLE (callcenter has zero dep on lance-graph-ogar); classify composition as later hardening, never as a shipped edge. (b) `lance-graph-ogar` is workspace-EXCLUDED — a green parent-workspace build compiles none of it (`--manifest-path` required; 83 tests green there). (c) The audit default is `NoopAuditSink` + `SuperDomain::Unknown`/salt 0 with swallowed emit errors, and `MEDCARE_AUDIT_SALT` is read fail-open then discarded — for a HIPAA-regime domain, "configured" vs "recording" is indistinguishable (P0 candidate). (d) Retention 2190d (`StepDomain::Medcare`, tested) vs 3650d (manifest, dead) is the DOCUMENTED open question OQ-2 (HIPAA vs BMV-Ä §57), not an accident. (e) `Anamnese`/`Ueberweisung` are first-class consumer concepts (97/81 refs) with no port alias / no canonical id at all — a real canon gap, not naming noise. (f) A third, unreconciled `.grok/` lineage (Zone-3 via spear, 2026-05-08) contradicts both.

**The lesson (generalizes):** when a repo hosts a consumer-shaped scaffold AND the consumer separately pulls the repo's libraries, the scaffold can freeze at birth while every ledger keeps narrating it as "the integration" — dormancy is invisible in prose because infra work *around* the frozen files (branch names, "first-thought loop" commits) keeps the term alive. The falsifier that cut through: per-file `git log` dormancy (81 days = age of the file) + constructor-count (zero) + the two-repo actor-name check. Also: the clone was SHALLOW (5 grafted roots, 11-day window) — `git fetch --unshallow` converted a "between 06-23 and 06-26 per prose" estimate into the exact commit `ddb6c840` two days earlier. **Archaeology on a shallow clone is prose archaeology; unshallow first.**

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

file=".claude/board/EPIPHANIES.md"
rg -n '^## [0-9]{4}-[0-9]{2}-[0-9]{2} — ' "$file"
git diff --unified=3 -- "$file"

Repository: AdaWorldAPI/lance-graph

Length of output: 50380


Move the MedCare finding to the newest-first ledgers position.

:2026-08-02 — E-TWO-MEDCARE-LINEAGES… is currently after older entries and before another existing 2026-08-02 item. This board must keep new entries prepended and preserve prior order.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.claude/board/EPIPHANIES.md around lines 871 - 879, Move the entire
`2026-08-02 — E-TWO-MEDCARE-LINEAGES-THE-LIVE-ONE-PULLS-THE-DEAD-ONE-HOSTS-1`
entry to the topmost newest-first ledger position, ahead of all older entries
and the other existing `2026-08-02` item. Preserve the exact entry content and
retain the relative order of every other entry.

Sources: Coding guidelines, Learnings

Comment thread .claude/board/INTEGRATION_PLANS.md Outdated
Comment thread .claude/board/LATEST_STATE.md Outdated
Comment thread .claude/plans/medcare-consumer-pull-thinking-proof-v1.md Outdated
Comment thread crates/lance-graph/src/graph/cycle_sink.rs
Comment on lines +369 to +408
let published = match self.open_if_exists().await? {
None => {
// Empty store: sealed head is DatasetVersion(0) by convention.
if base.0 != 0 {
return Err(WriteFailed(format!(
"stale base {base:?}: sealed head is DatasetVersion(0) (empty store)"
)));
}
let reader = RecordBatchIterator::new(vec![Ok(record_batch)], schema);
let params = WriteParams {
mode: WriteMode::Create,
..Default::default()
};
let ds = Dataset::write(reader, &self.dataset_path, Some(params))
.await
.map_err(|e| WriteFailed(format!("create commit: {e}")))?;
ds.version().version
}
Some(mut ds) => {
let head = ds.version().version;
if head != base.0 {
return Err(WriteFailed(format!(
"stale base {base:?}: sealed head is DatasetVersion({head})"
)));
}
let reader = RecordBatchIterator::new(vec![Ok(record_batch)], schema);
ds.append(reader, None)
.await
.map_err(|e| WriteFailed(format!("append commit: {e}")))?;
ds.version().version
}
};
if published != base.0 + 1 {
// The data IS committed at `published` — but the one-writer §I.6
// timeline is broken (a foreign writer interleaved). Surface loudly;
// never let a shifted identity pass as a sealed cycle.
return Err(WriteFailed(format!(
"timeline anomaly: committed at DatasetVersion({published}), expected {}",
base.0 + 1
)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

🧩 Analysis chain

🌐 Web query:

For Lance 9.0.0 Rust, which Dataset write or append API can make an append conditional on an expected dataset version, so a concurrent writer cannot turn a cycle based on Vn into a successful commit at Vn+2?

💡 Result:

In Lance 9.0.0 (Rust), you can implement a conditional append by using the lower-level Dataset::commit function, which allows you to explicitly provide a read_version parameter [1]. The standard high-level append method (e.g., Dataset::append) generally handles the current dataset version automatically [2], which may not provide the strict conditional control you need to prevent a concurrent writer from evolving the state (e.g., from Vn to Vn+2) before your commit completes. To ensure your write is conditional on a specific expected version (Vn), follow these steps: 1. Read the current version (Vn) of your dataset [3]. 2. Prepare your operation (e.g., an Append operation) with your data [4][5]. 3. Use the Dataset::commit method to finalize the transaction, passing Vn as the read_version argument [1][3]. The signature for Dataset::commit is: pub async fn commit( dest: impl Into<WriteDestination<'_>>, operation: Operation, read_version: Option, store_params: Option, commit_handler: Option<Arc>, session: Arc, enable_v2_manifest_paths: bool) -> Result<>; By explicitly providing read_version = Some(Vn), the commit process enforces optimistic concurrency control [4]. If a concurrent writer has already committed a newer version (Vn+1) to the underlying storage, the Lance commit mechanism will detect the conflict and fail the operation [4][6], preventing your transaction from creating a cycle or incorrectly applying changes based on an stale manifest [4][5]. If you are using the high-level InsertBuilder or append path, you are typically operating within a more abstracted flow that may mask this manual version management [2]. For maximum control over concurrency and version coupling, calling commit directly is the recommended approach [1].

Citations:


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate relevant files =="
git ls-files | rg 'crates/(lance-graph/src/graph/cycle_sink\.rs|lance-graph-supervisor/src/cycle_driver\.rs)$' || true

echo
echo "== cycle_sink outline =="
ast-grep outline crates/lance-graph/src/graph/cycle_sink.rs --view compact || true

echo
echo "== relevant cycle_sink lines =="
sed -n '330,430p' crates/lance-graph/src/graph/cycle_sink.rs

echo
echo "== cycle_driver relevant imports/types/calls =="
rg -n "run_cycle|write|WriteFailed|Determine|DatasetVersion|sealed|SealFailure|recover" crates/lance-graph-supervisor/src/cycle_driver.rs | head -n 200

Repository: AdaWorldAPI/lance-graph

Length of output: 18330


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== cycle_sink constructor and append path outline =="
ast-grep outline crates/lance-graph/src/graph/*cycle_sink*.rs --view signatures

echo
echo "== cycle_sink full file (focused around methods) =="
wc -l crates/lance-graph/src/graph/cycle_sink.rs
sed -n '1,180p' crates/lance-graph/src/graph/cycle_sink.rs

echo
echo "== cycle_driver run_cycle and run_cycle tests =="
sed -n '318,358p' crates/lance-graph-supervisor/src/cycle_driver.rs
sed -n '486,530p' crates/lance-graph-supervisor/src/cycle_driver.rs
sed -n '984,1120p' crates/lance-graph-supervisor/src/cycle_driver.rs

Repository: AdaWorldAPI/lance-graph

Length of output: 19532


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== run_cycle error variants and callers =="
rg -n "enum CycleError|CycleError\::|seal_cycle\(|recover_fleet\(|recover_and_apply\(" crates/lance-graph-supervisor -g '*.rs' | head -n 200

echo
echo "== run_circular/fleet lifecycle callers =="
for f in $(git ls-files crates/lance-graph-supervisor/src | rg '\.rs$'); do
  echo "--- $f"
  rg -n "run_cycle|seal_cycle|recover_fleet|CycleError" "$f" || true
done

echo
echo "== persist_sink traits =="
rg -n "struct|enum|trait|persist_cycle|commit_cycle|WriteFailed|DetachedCycleBatch" crates/*/src crates/lance-graph-supervisor/src -g '*.rs' | head -n 240

echo
echo "== Lance dependency version =="
rg -n 'name = "lance"\n|lance ="|lance = "' -g 'Cargo.toml' -C 3 .

Repository: AdaWorldAPI/lance-graph

Length of output: 28182


🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== cycle_driver error enum =="
sed -n '136,180p' crates/lance-graph-supervisor/src/cycle_driver.rs
sed -n '467,518p' crates/lance-graph-supervisor/src/cycle_driver.rs

echo
echo "== persist_sink definitions/usages =="
rg -n "pub (trait WalSink|enum WriteFailed|struct DetachedCycleBatch|struct PersistError|fn persist_cycle|fn commit_cycle)|WriteFailed|persist_cycle|commit_cycle" crates/lance-graph-planner crates/lance-graph src crates/lance-graph-supervisor -g '*.rs'

echo
echo "== Lance dependency declarations =="
rg --multiline -- '^\s*name = "lance"\n|^\s*name = "lance-graph"\n|^\s*name = "lance-graph-contract"\n|^\s*name = "lance-graph-supervisor"\n' Cargo.lock -g 'Cargo.lock' -C 2 || true

echo
echo "== deterministic verifier for race window and row sealing =="
python3 - <<'PY'
from dataclasses import dataclass
from enum import auto, Enum

class State(Enum):
    base_read = auto()
    foreign_commit_n_plus_1 = auto()
    append_starts_after_foreign_commit = auto()
    append_commits_unconditionally = auto()
    post_check_mismatch = auto()

`@dataclass`(equivalent=True)
class Run:
    state: State

trace = []
for state in State:
    run = Run(state)
    trace.append(run)
    if state is State.base_read:
        base = 0
        head = 0
    elif state is State.foreign_commit_n_plus_1:
        head = base + 1
    elif state is State.append_starts_after_foreign_commit:
        base_for_append = 0
        head_for_append = base + 1
    elif state is State.append_commits_unconditionally:
        published = head_for_append + 1
    elif state is State.post_check_mismatch:
        expected = base + 1
        mismatch_publish = published
        row_base_version = base
        row_derived_version = row_base_version + 1
print({
    "race_trace": [run.state.name for run in trace],
    "published_after_race": mismatch_publish,
    "row_derives": row_derived_version,
    "expected": expected,
})
PY

Repository: AdaWorldAPI/lance-graph

Length of output: 21275


Fail the cycle commit before appending an interleaved version.

A foreign writer can publish DatasetVersion(base.0 + 1) after this code reads the current head. ds.append() then commits under the new manifest and can return DatasetVersion(base.0 + 2), but the rows are still emitted with sealed_version() == base + 1. run_cycle then classifies this as CycleError::Seal and treats it as uncommitted regeneration. Make the append expected-version conditional, or return a separate committed-anomaly result including the actual published version and route it to committed-history handling.

📍 Affects 2 files
  • crates/lance-graph/src/graph/cycle_sink.rs#L369-L408 (this comment)
  • crates/lance-graph-supervisor/src/cycle_driver.rs#L26-L50
  • crates/lance-graph-supervisor/src/cycle_driver.rs#L1016-L1117
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/lance-graph/src/graph/cycle_sink.rs` around lines 369 - 408, Prevent
interleaved cycle commits from appending under a shifted DatasetVersion: update
the cycle sink commit path at crates/lance-graph/src/graph/cycle_sink.rs:369-408
to use an expected-version conditional append, or return a distinct
committed-anomaly result containing the actual published version. Update
crates/lance-graph-supervisor/src/cycle_driver.rs:26-50 and :1016-1117 so this
result is routed through committed-history handling rather than classified as an
uncommitted CycleError::Seal; preserve the actual published version when
reporting the anomaly.

…alesced image, 512-byte witness gate, projected reads

Addresses the #911 review findings (Codex P1x2/P2, CodeRabbit):

- Fence made effective even post-publication: Lance has no expected-
  version conditional append (the rebase runs even on a single-attempt
  Append commit; strict mode is Overwrite-only — measured in
  lance-9.0.0/src/io/commit.rs), so a foreign interleaved writer can
  land the batch at base+2. Detecting that now triggers a compensating
  official Dataset::delete scoped to exactly this cycle's rows (the
  cycle id is an unsealed identity, so the predicate is exact) BEFORE
  the retryable WriteFailed returns — 'write failed' is true at the
  visible head, the driver's regenerate-from-Vn contract stays sound,
  and no rows survive under a shifted sealed_version identity. The one
  manual-reconciliation corner (the compensating delete itself fails)
  names the orphaned version explicitly.

- The coalesced image is now DURABLE: kind=2 rows (row -> final payload
  after the per-row fold) persist in the same atomic commit as the
  landings, and LanceCycleSink::scan_image(cycle) reads a sealed
  cycle's coherent end-state (projected row+payload under the
  kind+cycle predicate) while scan_sealed keeps the per-cast history.

- Witness payloads are gated to exactly EPISODIC_WITNESS_BYTES = 512
  (the canonical key(16)|edges(16)|value(480) node stride) before
  anything durable happens; malformed rows refuse the whole cycle.

- versions() projects only cycle+base_version under kind=0 and
  scan_sealed pushes kind=1 into the scan, so the coarse-timeline
  lookup never materializes a witness payload.

- Board entries updated to the corrected contract; MD040 text fences in
  the medcare plan doc; INTEGRATION_PLANS 2026-08-02 entry moved ahead
  of its same-date sibling (EPIPHANIES placement already coherent).

8 reopened-dataset tests green (2 new: malformed-payload refusal with
nothing written; durable per-cycle image with later-stream-position-
wins and intact cast history).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KCGhDYoQBXs3poaR7sFuqp
@AdaWorldAPI
AdaWorldAPI marked this pull request as draft August 9, 2026 11:02
@AdaWorldAPI
AdaWorldAPI marked this pull request as ready for review August 9, 2026 11:37
@cursor

cursor Bot commented Aug 9, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_54530d94-a5ed-414a-9b5a-c142b85054ee)

@AdaWorldAPI
AdaWorldAPI merged commit 8a5be50 into main Aug 9, 2026
6 of 7 checks passed

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/lance-graph/src/graph/cycle_sink.rs`:
- Around line 517-547: The compensating delete in the post-publication branch
advances the physical dataset version, so treating the append as an uncommitted
retryable failure leaves sealed history inconsistent. Update the `ds.delete`
compensation and subsequent commit handling to use an expected-version
conditional, or record the actual published version as committed history; never
regenerate from the recovered sealed predecessor after the append was externally
visible.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 3351047e-9880-410c-addc-641bc5c60ae0

📥 Commits

Reviewing files that changed from the base of the PR and between e081ba3 and ba4f06c.

📒 Files selected for processing (5)
  • .claude/board/INTEGRATION_PLANS.md
  • .claude/board/LATEST_STATE.md
  • .claude/board/PR_ARC_INVENTORY.md
  • .claude/plans/medcare-consumer-pull-thinking-proof-v1.md
  • crates/lance-graph/src/graph/cycle_sink.rs
🚧 Files skipped from review as they are similar to previous changes (4)
  • .claude/board/PR_ARC_INVENTORY.md
  • .claude/plans/medcare-consumer-pull-thinking-proof-v1.md
  • .claude/board/LATEST_STATE.md
  • .claude/board/INTEGRATION_PLANS.md

Comment on lines +517 to +547
let published = ds.version().version;
if published != base.0 + 1 {
// A foreign writer interleaved between the fence check and
// the commit; Lance's append rebase landed this batch at a
// shifted version. Make the fence effective retroactively:
// remove exactly this cycle's just-appended rows, then
// report the (now-true) retryable failure. The cycle id is
// an unsealed identity at this point — no earlier sealed
// rows can carry it — so the predicate is exact.
let compensate = ds
.delete(&format!(
"cycle = {} AND base_version = {}",
batch.frame.cycle.0, base.0
))
.await;
return Err(match compensate {
Ok(_) => WriteFailed(format!(
"fenced post-publication: a foreign writer moved the head past \
{base:?} (batch landed at DatasetVersion({published})); the \
cycle's rows were deleted again — nothing of cycle {} is \
visible; regenerate from the current sealed head",
batch.frame.cycle.0
)),
Err(e) => WriteFailed(format!(
"TIMELINE ANOMALY, MANUAL RECONCILIATION REQUIRED: cycle {} \
committed at DatasetVersion({published}) (expected {}), and the \
compensating delete failed: {e}",
batch.frame.cycle.0,
base.0 + 1
)),
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Do not treat a published append as an uncommitted failure.

The compensating Dataset::delete publishes another dataset version. If the append lands at base + 2, the delete advances the physical head again. versions() still reports only frame-derived versions, so a subsequent commit using the recovered sealed predecessor fails the head != base.0 check at Line 508.

Use an expected-version commit, or return the actual published version as committed history. Do not regenerate from a cycle that was externally visible.

For Lance Rust version 9.0.0, does `Dataset::delete` create a new dataset version? Which official API supports a write conditional on an expected dataset version or read version?
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/lance-graph/src/graph/cycle_sink.rs` around lines 517 - 547, The
compensating delete in the post-publication branch advances the physical dataset
version, so treating the append as an uncommitted retryable failure leaves
sealed history inconsistent. Update the `ds.delete` compensation and subsequent
commit handling to use an expected-version conditional, or record the actual
published version as committed history; never regenerate from the recovered
sealed predecessor after the append was externally visible.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba4f06cd5c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +526 to +530
let compensate = ds
.delete(&format!(
"cycle = {} AND base_version = {}",
batch.frame.cycle.0, base.0
))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not delete the competing writer's cycle

When two writers derive the same cycle ID from the same sealed base, the first writer's successful rows and this writer's rebased rows both satisfy this predicate. The compensating delete therefore removes the winning writer's already-acknowledged frame, landings, and image as well, even though that writer may already have applied its transitions; after restart, its durable recovery history is gone. Fresh evidence beyond the earlier publication finding is that the new compensation path identifies rows only by the shared (cycle, base_version) rather than by the exact appended fragments or transaction.

Useful? React with 👍 / 👎.

Comment on lines +225 to +231
for s in &batch.landings {
if s.payload.len() != EPISODIC_WITNESS_BYTES {
return Err(WriteFailed(format!(
"landing payload for row {} is {} bytes, expected the canonical {EPISODIC_WITNESS_BYTES}",
s.row,
s.payload.len()
)));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve a valid payload when restaging held moves

When an owner produces multiple intent moves, restage_held explicitly stages each deferred move with Vec::new() (cycle_driver.rs:300-307), and the following cycle carries that empty payload into this check. The concrete sink then rejects the cycle because every landing must be exactly 512 bytes, so the documented held-intent progression can never commit through LanceCycleSink; restaging must retain or synthesize a valid row payload, or intent-only landings must be represented without violating the payload invariant.

Useful? React with 👍 / 👎.

AdaWorldAPI added a commit that referenced this pull request Aug 9, 2026
Phase A: artifact-backed commits + the sole owned Lance writer (supersedes the #911 cycle contract)
AdaWorldAPI pushed a commit that referenced this pull request Aug 18, 2026
…t deprecated (#879/#911/#912/#913)

Recorded in the RP-SEAL plan header + the pin-ruling EPIPHANIES entry;
the workflow script was corrected in place (source map + the two cell
briefs naming rustynum); the in-flight independent pass could not be
force-stopped in this harness build, so the ruling binds consolidation
as a hard filter. Deltalake removal ratified same exchange.

Co-Authored-By: Claude <noreply@anthropic.com>
AdaWorldAPI added a commit that referenced this pull request Aug 18, 2026
…e.dev debug=0 (#962)

* ogar_codebook: sync the ConceptDomain wire-mirror -- Ontology, Blocks, and the C-band

The contract's mirror of OGAR's ConceptDomain ended at Geo (0x0F) while
OGAR carries Ontology (0x03, populated by the DisMech 0x0333 mints),
Blocks (0x17), and the C-band JavaRuntime/Analytics/BinaryLifting
(0xC0/0xC1/0xC4 -- the altitude ruling, OGAR #276+#277; 0xC0 is Panama
FFM alone, Valhalla being a property of the C0 vocabulary rather than an
addressable concept). Both sides' docs demand they update together; this
is the catch-up, found by the lance-graph-java session and verified
independently by the ruff/R2IL session at db488f5, with ownership of the
sync explicitly handed here so the ruff arc's PR3 rebases trivially.

The real finding is WHY the drift guard never fired: domains_agree +
assert_codebook_parity only walk ids that carry concept rows, so a
reserved-EMPTY domain added to one enum but not the other is invisible to
a content walk. Proven live -- the first disable-run (dropping the new
BinaryLifting pair from domains_agree) stayed GREEN. Repaired with
reserved_empty_domains_agree_across_the_mirror: one id per new domain,
the populated 0x0333, the deliberate 0xC2-0xC3 gap pinned like OGAR's own
0x10-0x16, the band edges, and the 0x0C/0xC0 digit-swap two-sided. Both
disable-runs (bridge pair dropped; contract arm dropped) now go red on
exactly that test; the contract's own domain_routes_on_high_byte
independently catches the arm removal.

Gates: lance-graph-contract 1162/1162 + doctests, clippy --all-targets
clean; lance-graph-ogar (workspace-excluded, tested via manifest-path)
64/64 incl. assert_codebook_parity green -- content parity holds, this
was domain-level drift only. The crate's 11 pre-existing clippy warnings
are measured identical with this diff stashed and left untouched.

Board: EPIPHANIES E-OGAR-CODEBOOK-MIRROR-DOMAIN-DRIFT-SYNCED-1 prepended
in the same commit, per the board-hygiene rule.

* lotus Phase 0/1: frontier audit + F-ORD-REAL pre-registered falsifier

Research charter deliverables 1-2 (no fix included, by design):

- docs/lotus/LOTUS-FRONTIER-AUDIT.md — Phase 0 archaeology across the
  write path, persistence capability, placement/comma prior art, and
  frontier visibility, every statement graded VERIFIED / INFERENCE /
  HYPOTHESIS / BLOCKER. Headline findings: content_hash folds
  arrival-minted stream_position values into batch_hash
  (persist_sink.rs:414), contradicting DetachedCycleBatch's own
  order-independence doc; the seal is O(batch bytes) x3 passes with the
  batch resident up to 3x at seal; SweepSlot's caller-supplied semantic
  order-key contract vs collect_casts' arrival mint; the lance crate
  source is absent from this sandbox (prepared-artifact capability audit
  BLOCKED). Section 6 answers the permeability question: the cycle does
  not become permeable, it becomes thin — trailing-publication pipelining
  (Regime A) needs no epistemic weakening; rung-qualified frontier
  visibility (temporal.rs EpistemicMode ladder) stratifies rather than
  reopens the retired race; texts stay linear, tiles get derived
  placement, resolved per class.

- docs/lotus/F-ORD-REAL-FALSIFIER.md — the defect mechanism in four
  verified steps + the test design (perturb the process that creates the
  key: permute cast() call order, never post-mint slots).

- cycle_driver.rs tests: f_ord_real_defect_pin_... (GREEN, two-sided —
  anti-vacuity proves the perturbation reaches the key mint; semantic
  set + image pinned arrival-independent; batch_hash pinned
  arrival-DEPENDENT; fails loudly when a fix lands) and
  f_ord_real_publication_identity_... (#[ignore]d RED falsifier — the
  desired property, red under --ignored on the real chain).

Gates: cargo test -p lance-graph-supervisor --features cycle-driver
28 passed / 1 ignored (+ suites green); RED verified red under
--ignored; fmt clean; clippy adds zero new warnings (8 pre-existing
recover_fleet lints only visible under this non-default feature).

Board: EPIPHANIES E-FORD-REAL-PUBLICATION-IDENTITY-IS-ARRIVAL-DEPENDENT-1;
STATUS_BOARD lotus-seal-fractal-commit-frontier section (D-LOTUS-1..9).

Co-Authored-By: Claude <noreply@anthropic.com>

* board: #961 arc entry + LATEST_STATE + D-LOTUS-1/2 Shipped flips

Co-Authored-By: Claude <noreply@anthropic.com>

* operator pin ruling: DF 54.1 only — remove broken delta feature; RP-SEAL charter + boards

Ruling (E-PIN-LANCE9-LANCEDB033-DF541-ARROW58-NO-DF53-1): lance 9 /
lancedb 0.33 / datafusion 54.1 (no DF 53) / arrow 58, always, across
AdaWorldAPI forks, usually via [patch -> upstream repository git].

Measured before acting: the only DF-53 source was deltalake 0.32.4
(^53.1.0) behind the already-non-default, already-broken delta feature;
registry check shows no DF-54 deltalake exists. Removed: delta feature,
deltalake + url optional deps, DeltaTableReader (module docs + Cargo.toml
carry dated removal notes; DataSourceFormat::Delta stays as a catalog
tag). Post-removal Cargo.lock: exactly ONE datafusion = 54.1.0, zero
deltalake entries. cargo check -p lance-graph green; fmt clean; full
suite running as the PR gate. Docker pins surveyed: root + avx512 =
Rust 1.97.1 + protobuf-compiler, no delta references — removal is
docker-safe (stale flag noted: crates/symbiont/Dockerfile rust:1.95).

Also: RP-SEAL research charter committed as
.claude/plans/erasure-seals-compaction-research-v1.md (15-researcher
program, independent pass dispatched as background workflow
wf_ca974718-1b4; adversaries strongest-tier, builders/scouts grindwork
tier); INTEGRATION_PLANS + STATUS_BOARD entries; D-LOTUS-6 BLOCKER
lifted (operator-sanctioned upstream-git source consult; exact v9.0.0
tag on disk matching the lock checksum) with the audit carrying the
dated lift note; CLAUDE.md 'BOTH MAJORS ARE REQUIRED' note superseded
in place.

Co-Authored-By: Claude <noreply@anthropic.com>

* scope pivot (operator): rustynum struck — everything ndarray; symbiont deprecated (#879/#911/#912/#913)

Recorded in the RP-SEAL plan header + the pin-ruling EPIPHANIES entry;
the workflow script was corrected in place (source map + the two cell
briefs naming rustynum); the in-flight independent pass could not be
force-stopped in this harness build, so the ruling binds consolidation
as a hard filter. Deltalake removal ratified same exchange.

Co-Authored-By: Claude <noreply@anthropic.com>

* correct the delta-removal notes: delta-rs MAIN is already DF 54 + arrow 58

Operator-pointed, verified from delta-io/delta-rs main Cargo.toml
(datafusion = 54.0.0, arrow = 58): only the crates.io releases top out at
DF 53. Restoration is available now via a git-pin on upstream + a reader
refactor to the current builder API — as its own deliberate PR if a
consumer needs Delta. The removal itself stands on the need ruling
('we don't need deltalake'), not on availability; notes in Cargo.toml,
CLAUDE.md, and the EPIPHANIES entry corrected accordingly.

Co-Authored-By: Claude <noreply@anthropic.com>

* profile.dev debug=0 — smaller/faster builds (operator ruling)

Full-debuginfo test binaries grew target/ to 17 GB and SIGBUS'd the
linker on a full disk this session; debug=0 shrinks the
lance/datafusion-stack test binaries ~an order of magnitude. Matches the
gate already running with CARGO_PROFILE_DEV_DEBUG=0 (same resolved
profile, cache reuse). line-tables-only noted as the fallback if
line-numbered backtraces are ever needed.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
AdaWorldAPI added a commit that referenced this pull request Aug 18, 2026
* ogar_codebook: sync the ConceptDomain wire-mirror -- Ontology, Blocks, and the C-band

The contract's mirror of OGAR's ConceptDomain ended at Geo (0x0F) while
OGAR carries Ontology (0x03, populated by the DisMech 0x0333 mints),
Blocks (0x17), and the C-band JavaRuntime/Analytics/BinaryLifting
(0xC0/0xC1/0xC4 -- the altitude ruling, OGAR #276+#277; 0xC0 is Panama
FFM alone, Valhalla being a property of the C0 vocabulary rather than an
addressable concept). Both sides' docs demand they update together; this
is the catch-up, found by the lance-graph-java session and verified
independently by the ruff/R2IL session at db488f5, with ownership of the
sync explicitly handed here so the ruff arc's PR3 rebases trivially.

The real finding is WHY the drift guard never fired: domains_agree +
assert_codebook_parity only walk ids that carry concept rows, so a
reserved-EMPTY domain added to one enum but not the other is invisible to
a content walk. Proven live -- the first disable-run (dropping the new
BinaryLifting pair from domains_agree) stayed GREEN. Repaired with
reserved_empty_domains_agree_across_the_mirror: one id per new domain,
the populated 0x0333, the deliberate 0xC2-0xC3 gap pinned like OGAR's own
0x10-0x16, the band edges, and the 0x0C/0xC0 digit-swap two-sided. Both
disable-runs (bridge pair dropped; contract arm dropped) now go red on
exactly that test; the contract's own domain_routes_on_high_byte
independently catches the arm removal.

Gates: lance-graph-contract 1162/1162 + doctests, clippy --all-targets
clean; lance-graph-ogar (workspace-excluded, tested via manifest-path)
64/64 incl. assert_codebook_parity green -- content parity holds, this
was domain-level drift only. The crate's 11 pre-existing clippy warnings
are measured identical with this diff stashed and left untouched.

Board: EPIPHANIES E-OGAR-CODEBOOK-MIRROR-DOMAIN-DRIFT-SYNCED-1 prepended
in the same commit, per the board-hygiene rule.

* lotus Phase 0/1: frontier audit + F-ORD-REAL pre-registered falsifier

Research charter deliverables 1-2 (no fix included, by design):

- docs/lotus/LOTUS-FRONTIER-AUDIT.md — Phase 0 archaeology across the
  write path, persistence capability, placement/comma prior art, and
  frontier visibility, every statement graded VERIFIED / INFERENCE /
  HYPOTHESIS / BLOCKER. Headline findings: content_hash folds
  arrival-minted stream_position values into batch_hash
  (persist_sink.rs:414), contradicting DetachedCycleBatch's own
  order-independence doc; the seal is O(batch bytes) x3 passes with the
  batch resident up to 3x at seal; SweepSlot's caller-supplied semantic
  order-key contract vs collect_casts' arrival mint; the lance crate
  source is absent from this sandbox (prepared-artifact capability audit
  BLOCKED). Section 6 answers the permeability question: the cycle does
  not become permeable, it becomes thin — trailing-publication pipelining
  (Regime A) needs no epistemic weakening; rung-qualified frontier
  visibility (temporal.rs EpistemicMode ladder) stratifies rather than
  reopens the retired race; texts stay linear, tiles get derived
  placement, resolved per class.

- docs/lotus/F-ORD-REAL-FALSIFIER.md — the defect mechanism in four
  verified steps + the test design (perturb the process that creates the
  key: permute cast() call order, never post-mint slots).

- cycle_driver.rs tests: f_ord_real_defect_pin_... (GREEN, two-sided —
  anti-vacuity proves the perturbation reaches the key mint; semantic
  set + image pinned arrival-independent; batch_hash pinned
  arrival-DEPENDENT; fails loudly when a fix lands) and
  f_ord_real_publication_identity_... (#[ignore]d RED falsifier — the
  desired property, red under --ignored on the real chain).

Gates: cargo test -p lance-graph-supervisor --features cycle-driver
28 passed / 1 ignored (+ suites green); RED verified red under
--ignored; fmt clean; clippy adds zero new warnings (8 pre-existing
recover_fleet lints only visible under this non-default feature).

Board: EPIPHANIES E-FORD-REAL-PUBLICATION-IDENTITY-IS-ARRIVAL-DEPENDENT-1;
STATUS_BOARD lotus-seal-fractal-commit-frontier section (D-LOTUS-1..9).

Co-Authored-By: Claude <noreply@anthropic.com>

* board: #961 arc entry + LATEST_STATE + D-LOTUS-1/2 Shipped flips

Co-Authored-By: Claude <noreply@anthropic.com>

* operator pin ruling: DF 54.1 only — remove broken delta feature; RP-SEAL charter + boards

Ruling (E-PIN-LANCE9-LANCEDB033-DF541-ARROW58-NO-DF53-1): lance 9 /
lancedb 0.33 / datafusion 54.1 (no DF 53) / arrow 58, always, across
AdaWorldAPI forks, usually via [patch -> upstream repository git].

Measured before acting: the only DF-53 source was deltalake 0.32.4
(^53.1.0) behind the already-non-default, already-broken delta feature;
registry check shows no DF-54 deltalake exists. Removed: delta feature,
deltalake + url optional deps, DeltaTableReader (module docs + Cargo.toml
carry dated removal notes; DataSourceFormat::Delta stays as a catalog
tag). Post-removal Cargo.lock: exactly ONE datafusion = 54.1.0, zero
deltalake entries. cargo check -p lance-graph green; fmt clean; full
suite running as the PR gate. Docker pins surveyed: root + avx512 =
Rust 1.97.1 + protobuf-compiler, no delta references — removal is
docker-safe (stale flag noted: crates/symbiont/Dockerfile rust:1.95).

Also: RP-SEAL research charter committed as
.claude/plans/erasure-seals-compaction-research-v1.md (15-researcher
program, independent pass dispatched as background workflow
wf_ca974718-1b4; adversaries strongest-tier, builders/scouts grindwork
tier); INTEGRATION_PLANS + STATUS_BOARD entries; D-LOTUS-6 BLOCKER
lifted (operator-sanctioned upstream-git source consult; exact v9.0.0
tag on disk matching the lock checksum) with the audit carrying the
dated lift note; CLAUDE.md 'BOTH MAJORS ARE REQUIRED' note superseded
in place.

Co-Authored-By: Claude <noreply@anthropic.com>

* scope pivot (operator): rustynum struck — everything ndarray; symbiont deprecated (#879/#911/#912/#913)

Recorded in the RP-SEAL plan header + the pin-ruling EPIPHANIES entry;
the workflow script was corrected in place (source map + the two cell
briefs naming rustynum); the in-flight independent pass could not be
force-stopped in this harness build, so the ruling binds consolidation
as a hard filter. Deltalake removal ratified same exchange.

Co-Authored-By: Claude <noreply@anthropic.com>

* correct the delta-removal notes: delta-rs MAIN is already DF 54 + arrow 58

Operator-pointed, verified from delta-io/delta-rs main Cargo.toml
(datafusion = 54.0.0, arrow = 58): only the crates.io releases top out at
DF 53. Restoration is available now via a git-pin on upstream + a reader
refactor to the current builder API — as its own deliberate PR if a
consumer needs Delta. The removal itself stands on the need ruling
('we don't need deltalake'), not on availability; notes in Cargo.toml,
CLAUDE.md, and the EPIPHANIES entry corrected accordingly.

Co-Authored-By: Claude <noreply@anthropic.com>

* profile.dev debug=0 — smaller/faster builds (operator ruling)

Full-debuginfo test binaries grew target/ to 17 GB and SIGBUS'd the
linker on a full disk this session; debug=0 shrinks the
lance/datafusion-stack test binaries ~an order of magnitude. Matches the
gate already running with CARGO_PROFILE_DEV_DEBUG=0 (same resolved
profile, cache reuse). line-tables-only noted as the fallback if
line-numbered backtraces are ever needed.

Co-Authored-By: Claude <noreply@anthropic.com>

* drift cleanup: rustynum/symbiont no-go applied to live surfaces

- lance-graph-cognitive: rustynum_accel was a name-only shim (zero
  rustynum dependency); renamed simd_accel (module + file + 11 call
  sites), rerouted through the sanctioned ndarray::simd re-export
  instead of ndarray::hpc::bitwise. Default compile green; the wip
  feature's pre-existing not-yet-compiling state has zero errors naming
  the rename.
- docs/lotus audit: symbiont::domino::morton4 candidate row struck
  (census history only).
- CLAUDE.md: symbiont removed from the binding-consumer lists via
  dated annotations.
- EPIPHANIES: cleanup recorded in the pin-ruling entry.

Audit also confirmed NO redo needed: A1's research report is clean
(0 citations), the Java/Panama arc never touched either (ndarray::simd
enforced), board history stays append-only.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
AdaWorldAPI added a commit that referenced this pull request Aug 19, 2026
… core, compaction = optional economics (#964)

* ogar_codebook: sync the ConceptDomain wire-mirror -- Ontology, Blocks, and the C-band

The contract's mirror of OGAR's ConceptDomain ended at Geo (0x0F) while
OGAR carries Ontology (0x03, populated by the DisMech 0x0333 mints),
Blocks (0x17), and the C-band JavaRuntime/Analytics/BinaryLifting
(0xC0/0xC1/0xC4 -- the altitude ruling, OGAR #276+#277; 0xC0 is Panama
FFM alone, Valhalla being a property of the C0 vocabulary rather than an
addressable concept). Both sides' docs demand they update together; this
is the catch-up, found by the lance-graph-java session and verified
independently by the ruff/R2IL session at db488f5, with ownership of the
sync explicitly handed here so the ruff arc's PR3 rebases trivially.

The real finding is WHY the drift guard never fired: domains_agree +
assert_codebook_parity only walk ids that carry concept rows, so a
reserved-EMPTY domain added to one enum but not the other is invisible to
a content walk. Proven live -- the first disable-run (dropping the new
BinaryLifting pair from domains_agree) stayed GREEN. Repaired with
reserved_empty_domains_agree_across_the_mirror: one id per new domain,
the populated 0x0333, the deliberate 0xC2-0xC3 gap pinned like OGAR's own
0x10-0x16, the band edges, and the 0x0C/0xC0 digit-swap two-sided. Both
disable-runs (bridge pair dropped; contract arm dropped) now go red on
exactly that test; the contract's own domain_routes_on_high_byte
independently catches the arm removal.

Gates: lance-graph-contract 1162/1162 + doctests, clippy --all-targets
clean; lance-graph-ogar (workspace-excluded, tested via manifest-path)
64/64 incl. assert_codebook_parity green -- content parity holds, this
was domain-level drift only. The crate's 11 pre-existing clippy warnings
are measured identical with this diff stashed and left untouched.

Board: EPIPHANIES E-OGAR-CODEBOOK-MIRROR-DOMAIN-DRIFT-SYNCED-1 prepended
in the same commit, per the board-hygiene rule.

* lotus Phase 0/1: frontier audit + F-ORD-REAL pre-registered falsifier

Research charter deliverables 1-2 (no fix included, by design):

- docs/lotus/LOTUS-FRONTIER-AUDIT.md — Phase 0 archaeology across the
  write path, persistence capability, placement/comma prior art, and
  frontier visibility, every statement graded VERIFIED / INFERENCE /
  HYPOTHESIS / BLOCKER. Headline findings: content_hash folds
  arrival-minted stream_position values into batch_hash
  (persist_sink.rs:414), contradicting DetachedCycleBatch's own
  order-independence doc; the seal is O(batch bytes) x3 passes with the
  batch resident up to 3x at seal; SweepSlot's caller-supplied semantic
  order-key contract vs collect_casts' arrival mint; the lance crate
  source is absent from this sandbox (prepared-artifact capability audit
  BLOCKED). Section 6 answers the permeability question: the cycle does
  not become permeable, it becomes thin — trailing-publication pipelining
  (Regime A) needs no epistemic weakening; rung-qualified frontier
  visibility (temporal.rs EpistemicMode ladder) stratifies rather than
  reopens the retired race; texts stay linear, tiles get derived
  placement, resolved per class.

- docs/lotus/F-ORD-REAL-FALSIFIER.md — the defect mechanism in four
  verified steps + the test design (perturb the process that creates the
  key: permute cast() call order, never post-mint slots).

- cycle_driver.rs tests: f_ord_real_defect_pin_... (GREEN, two-sided —
  anti-vacuity proves the perturbation reaches the key mint; semantic
  set + image pinned arrival-independent; batch_hash pinned
  arrival-DEPENDENT; fails loudly when a fix lands) and
  f_ord_real_publication_identity_... (#[ignore]d RED falsifier — the
  desired property, red under --ignored on the real chain).

Gates: cargo test -p lance-graph-supervisor --features cycle-driver
28 passed / 1 ignored (+ suites green); RED verified red under
--ignored; fmt clean; clippy adds zero new warnings (8 pre-existing
recover_fleet lints only visible under this non-default feature).

Board: EPIPHANIES E-FORD-REAL-PUBLICATION-IDENTITY-IS-ARRIVAL-DEPENDENT-1;
STATUS_BOARD lotus-seal-fractal-commit-frontier section (D-LOTUS-1..9).

Co-Authored-By: Claude <noreply@anthropic.com>

* board: #961 arc entry + LATEST_STATE + D-LOTUS-1/2 Shipped flips

Co-Authored-By: Claude <noreply@anthropic.com>

* operator pin ruling: DF 54.1 only — remove broken delta feature; RP-SEAL charter + boards

Ruling (E-PIN-LANCE9-LANCEDB033-DF541-ARROW58-NO-DF53-1): lance 9 /
lancedb 0.33 / datafusion 54.1 (no DF 53) / arrow 58, always, across
AdaWorldAPI forks, usually via [patch -> upstream repository git].

Measured before acting: the only DF-53 source was deltalake 0.32.4
(^53.1.0) behind the already-non-default, already-broken delta feature;
registry check shows no DF-54 deltalake exists. Removed: delta feature,
deltalake + url optional deps, DeltaTableReader (module docs + Cargo.toml
carry dated removal notes; DataSourceFormat::Delta stays as a catalog
tag). Post-removal Cargo.lock: exactly ONE datafusion = 54.1.0, zero
deltalake entries. cargo check -p lance-graph green; fmt clean; full
suite running as the PR gate. Docker pins surveyed: root + avx512 =
Rust 1.97.1 + protobuf-compiler, no delta references — removal is
docker-safe (stale flag noted: crates/symbiont/Dockerfile rust:1.95).

Also: RP-SEAL research charter committed as
.claude/plans/erasure-seals-compaction-research-v1.md (15-researcher
program, independent pass dispatched as background workflow
wf_ca974718-1b4; adversaries strongest-tier, builders/scouts grindwork
tier); INTEGRATION_PLANS + STATUS_BOARD entries; D-LOTUS-6 BLOCKER
lifted (operator-sanctioned upstream-git source consult; exact v9.0.0
tag on disk matching the lock checksum) with the audit carrying the
dated lift note; CLAUDE.md 'BOTH MAJORS ARE REQUIRED' note superseded
in place.

Co-Authored-By: Claude <noreply@anthropic.com>

* scope pivot (operator): rustynum struck — everything ndarray; symbiont deprecated (#879/#911/#912/#913)

Recorded in the RP-SEAL plan header + the pin-ruling EPIPHANIES entry;
the workflow script was corrected in place (source map + the two cell
briefs naming rustynum); the in-flight independent pass could not be
force-stopped in this harness build, so the ruling binds consolidation
as a hard filter. Deltalake removal ratified same exchange.

Co-Authored-By: Claude <noreply@anthropic.com>

* correct the delta-removal notes: delta-rs MAIN is already DF 54 + arrow 58

Operator-pointed, verified from delta-io/delta-rs main Cargo.toml
(datafusion = 54.0.0, arrow = 58): only the crates.io releases top out at
DF 53. Restoration is available now via a git-pin on upstream + a reader
refactor to the current builder API — as its own deliberate PR if a
consumer needs Delta. The removal itself stands on the need ruling
('we don't need deltalake'), not on availability; notes in Cargo.toml,
CLAUDE.md, and the EPIPHANIES entry corrected accordingly.

Co-Authored-By: Claude <noreply@anthropic.com>

* profile.dev debug=0 — smaller/faster builds (operator ruling)

Full-debuginfo test binaries grew target/ to 17 GB and SIGBUS'd the
linker on a full disk this session; debug=0 shrinks the
lance/datafusion-stack test binaries ~an order of magnitude. Matches the
gate already running with CARGO_PROFILE_DEV_DEBUG=0 (same resolved
profile, cache reuse). line-tables-only noted as the fallback if
line-numbered backtraces are ever needed.

Co-Authored-By: Claude <noreply@anthropic.com>

* drift cleanup: rustynum/symbiont no-go applied to live surfaces

- lance-graph-cognitive: rustynum_accel was a name-only shim (zero
  rustynum dependency); renamed simd_accel (module + file + 11 call
  sites), rerouted through the sanctioned ndarray::simd re-export
  instead of ndarray::hpc::bitwise. Default compile green; the wip
  feature's pre-existing not-yet-compiling state has zero errors naming
  the rename.
- docs/lotus audit: symbiont::domino::morton4 candidate row struck
  (census history only).
- CLAUDE.md: symbiont removed from the binding-consumer lists via
  dated annotations.
- EPIPHANIES: cleanup recorded in the pin-ruling entry.

Audit also confirmed NO redo needed: A1's research report is clean
(0 citations), the Java/Panama arc never touched either (ndarray::simd
enforced), board history stays append-only.

Co-Authored-By: Claude <noreply@anthropic.com>

* RP-SEAL pass 1 consolidated: evidence matrix M1-M15, tiering, Appendix H

15-researcher independent pass complete (15/15, 0 errors, no cross-talk).
Consolidation per charter §12-§13: docs/lotus/RP-SEAL-CONSOLIDATION-PASS1.md
- DELETED: phase/comma coefficient schedule (3 independent routes), the
  unconditional SFC default, 'compaction = one publication', the naive
  query+repair one-grouping hope
- Found: b+1 coalescing amplification (65x at b=64) + non-amortizing FNV
  seal CPU (E2); 11 order leaks, L2 = replay order IS physical scan order
  (D2); five defective seal paths wearing strong names (C2/C3); Lance
  9.0.0 apparatus complete and unused (5 cells concordant)
- Survived: hash + row/column P+Q 64x64 seal baseline; compaction-time
  layout-key upstream seam; two novel candidates (joint locality question
  refined by the anti-synergy; STRICT/AWARE/RETRO reader-rung tiers)
- Appendix H: 15 reports + orchestrator pre-pass under docs/lotus/rp-seal-v1/
- Scope-pivot filter PASSED (zero rustynum/symbiont-sourced findings)
- Board: EPIPHANIES entry, AGENT_LOG entry (one-writer), STATUS_BOARD +
  plan header flips

Co-Authored-By: Claude <noreply@anthropic.com>

* §0 operator STORNO: canonical replay coordinates are core; compaction = optional economics

PHYSICAL ITERATION ORDER MUST NOT DEFINE SEMANTIC REPLAY ORDER — fixed
by canonical replay coordinates/order, never by physical reordering or
compaction. Permanent temporal replay additionally requires a
retention/tombstone policy under which historical knowledge states stay
reconstructible indefinitely. Desired model: arbitrary placement +
arbitrary fragmentation + historically addressable tombstones +
canonical semantic replay order + current visibility as a projection +
historical visibility as a QueryReference projection.

Consolidation doc gains §0 with in-place ⊘-marked corrections; Tier 1
reordered (canonical coordinates, retention/tombstone, QueryReference
projection fixes); Tier 2/3 layout/compaction items reclassified as the
optional storage-economics track; F-NOCOMPACT promoted to the default
operating description. EPIPHANIES ruling entry + STATUS_BOARD + plan
header amended.

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants