plan: rebuild substrate-comfort-zones §2 as a cross-swap, not a horse race

[AdaWorldAPI]

Why

Operator ruling, two messages:

1. the method is cross-swap and hypothesis testing, under the premise that the model CAPTURES the phenomenon but is NOT calibrated;
2. in science you hold variables constant to test the others — constancy is relative, so the design deliberately manufactures strong correlation differences, on the assumption those differences are fit to evaluate the hypothesis.

Against that, substrate-comfort-zones-v1 §2 was wrong in kind, not in detail. It raced four calibration arms on reconstruction RMSE. But under the stated premise a deliberately wrong calibration produces bad error by definition — so CAL-ABS-FOREIGN is the measurement condition, not a competitor that might lose. Racing a condition guarantees the answer and measures a tautology.

What changed

• §2 is now a 4×4 donor × target transfer matrix per geometry arm. Primary metric Spearman ρ (structure preservation); the quantity the hypothesis actually concerns is the derived transfer loss L[D][T] = ρ[T][T] − ρ[D][T]. RMSE/bias in Pa stay per cell as evidence the swap genuinely hurt — never as the verdict.
• CAL-ABS-OWN / CAL-ABS-FOREIGN collapse into one arm's diagonal and off-diagonal. Splitting them was the tell that the design was still a race.
• The dynamic arms' rows are flat by construction (no donor exists, so L ≡ 0) — which is the property under test, not an artifact. C2 now requires both halves: that it hold exactly, and that the identical code path be proven able to produce a non-flat row. Half one alone is the can-it-DIFFER defect W5 already paid for.
• C1b measures constancy (separation = between-box range / mean within-box σ ≥ 3) instead of claiming it.
• C1c gives the suitability ASSUMPTION a falsifier: the regimes must differ in autocorrelation decay and rank-distribution shape, not merely in |∇p| — otherwise the ladder is four copies of one condition, and a null there VOIDS the reading rather than weakening it (the spread would have been manufactured on the wrong axis).
• The crossover now scores against the diagonal on ρ — own-calibration, the hardest available opponent. The weak form against the swapped cell is reported separately and labelled evidence of wiring, not of merit.

What did NOT change

The §1 regime ladder and its three preflight corrections, the equal-budget discipline, the geometry axis, the C0 control gate, the store-the-operands output contract, and §5's non-claims. The scaffold was sound; the question was inverted.

Honesty notes

• Every D-CZ row was still Queued when this happened — only D-CZ-0 (the regime preflight) was DONE, and it is untouched. No measured result was reinterpreted. This is a pre-registration sharpened before any bar ran, which is precisely when a design is allowed to change.
• v1's framing is preserved in place as §2's correction note, not deleted — a future session can see what the design was and why it moved.
• Not a v2 file: same plan, same D-ids, so the board rows and the plan stay joined. The old→new bar mapping (C2→C4, C3→C3-reshaped, C4→C5, C5→C6) is recorded on STATUS_BOARD.
• Two new bars exist because "held constant" and "the manufactured spread is on the right axis" moved from assumed to measured.

Board hygiene

file	change	check
EPIPHANIES.md	E-A-HORSE-RACE-IS-NOT-A-CROSS-SWAP-1	prepend, suffix-verified (+4824 chars, old content an exact suffix of new)
INTEGRATION_PLANS.md	revision entry	prepend, suffix-verified (+3006 chars)
STATUS_BOARD.md	D-CZ rows re-cut + renumbering note	in-place (all rows Queued; dashboard, not an append-only ledger)

Zero code, zero probe runs, zero fetches — pre-registration only.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CcpLeEC3XK8Eye53GKBVvi

---

Generated by Claude Code

Summary by CodeRabbit

• Documentation
  • Updated experiment plans and status tracking to evaluate calibration through donor-to-target cross-swap matrices.
  • Added transfer loss and structure preservation as primary evaluation measures, with error and bias as supporting metrics.
  • Added controls for calibration consistency, regime suitability, degenerate transfer rows, occupancy, saturation, and crossover behavior.
  • Expanded reporting requirements to include matrix inputs, calibration boundaries, regime measurements, and diagnostic ratios.
  • Documented limitations and rationale for the revised evaluation approach.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 86c8713d-d2d1-46f8-9d1a-e4bf8a87554f

📥 Commits

Reviewing files that changed from the base of the PR and between 4304590 and 1756c51.

📒 Files selected for processing (4)

• .claude/board/EPIPHANIES.md
• .claude/board/INTEGRATION_PLANS.md
• .claude/board/STATUS_BOARD.md
• .claude/plans/substrate-comfort-zones-v1.md

---

📝 Walkthrough

Walkthrough

The experiment documentation now replaces calibration-arm RMSE comparisons with donor-target cross-swap matrices. It adds transfer-loss, regime-suitability, degeneracy, occupancy, saturation, output, and limitation requirements.

Changes

Calibration cross-swap redesign

Layer / File(s)	Summary
Cross-swap experiment definition .claude/plans/substrate-comfort-zones-v1.md	The plan defines 4×4 donor-target matrices, revised calibration arms, Spearman correlation, transfer loss, occupancy, saturation, and diagonal-based comparisons.
Validation and output requirements .claude/plans/substrate-comfort-zones-v1.md	The plan adds regime-separation checks, dynamic-row degeneracy verification, expanded matrix outputs, and limitations for interpreting transfer loss.
Plan and status records .claude/board/INTEGRATION_PLANS.md, .claude/board/STATUS_BOARD.md, .claude/board/EPIPHANIES.md	The project records document the cross-swap revision, new validation bars, updated deliverables, and the related finding.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: claude

Poem

I hop through matrices, neat and square,
Donor rows cross targets there.
Loss and rank now guide the way,
While flat rows prove what tests must say.
A rabbit stamps the plan: “Hooray!”

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_058281f5-2969-4f13-a3b4-50f8169fbed4)

[AdaWorldAPI]

@coderabbitai review

Requesting explicitly despite draft status. This PR is pre-registration prose only — no code, no probe runs — and the failure mode it is most exposed to is a stated relation that does not hold between its own operands (this arc has shipped that defect five times: a fused comparison in #930, a decimal-place claim refuted by its own table in #927, a wrong audit figure in #928, a dropped qualifier in #941, a prescribed-vs-done conflation in #943). A reviewer reading the bars against each other is worth more here than a reviewer reading a diff.

Specifically worth checking:

• L[D][T] = ρ[T][T] − ρ[D][T] — does the sign convention match every prose statement about it? "Low L means the substrate carried the structure" and "transfer loss shrinks with turbulence" must both be consistent with that definition and with C3's L̄[T] = mean_{D≠T} L[D][T].
• C4's direction. Δ[T] = ρ(CAL-RANK) − ρ(CAL-ABS, D=T) is asserted to be < 0 in calm and > 0 in storm. Since ρ is higher is better (unlike v1's RMSE, where the inequality ran the other way), the signs were flipped when the metric changed — please confirm they are flipped correctly and not half-converted.
• Bar numbering after the re-cut. C0, C1, C1b, C1c, C2, C3, C4, C5, C6 should each appear exactly once in the plan and map 1:1 onto the D-CZ-* rows in STATUS_BOARD.md, with the old→new mapping note accurate.
• C2's two halves. Half (i) is dynamic arms show L ≡ 0; half (ii) is CAL-ABS proven non-degenerate through the same code path. If (ii) reads as optional anywhere, that is the defect the bar exists to prevent.
• Any place the prose still implies a ranking of formulas rather than a transfer measurement — that framing is what this PR removes, and a leftover sentence would reintroduce it.

---

Generated by Claude Code

[coderabbitai]

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

for name in EPIPHANIES.md INTEGRATION_PLANS.md STATUS_BOARD.md; do
  echo "===== FILE: $name ====="
  fd -a -t f -g "$name" .
done

echo "===== OUTLINES / MATCH LOCATIONS ====="
rg -n -i -C 3 \
  'substrate-comfort-zones|D-CZ|CAL-ABS|CAL-RANK|transfer loss|L\[D\]\[T\]|L̄|rho|ρ|Spearman|C0|C1b|C1c|C2|C3|C4|C5|C6|horse race|rank(ing)? of formulas?' \
  EPIPHANIES.md INTEGRATION_PLANS.md STATUS_BOARD.md 2>/dev/null || true

echo "===== INTEGRATION PLAN: TARGET SECTION ====="
awk '
  /substrate-comfort-zones/ { found=1 }
  found { print NR ":" $0 }
  found && NR > start + 900 { exit }
  found && start == 0 { start=NR }
' INTEGRATION_PLANS.md 2>/dev/null || true

echo "===== STATUS BOARD: D-CZ ROWS ====="
rg -n -i -C 5 'D-CZ|old.*new|C2.*C4|C3.*reshap|C4.*C5|C5.*C6' STATUS_BOARD.md 2>/dev/null || true

Length of output: 1202

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "===== REPOSITORY ROOT ====="
pwd
printf '\n===== ROOT ENTRIES =====\n'
find . -maxdepth 1 -mindepth 1 -printf '%f\n' | sort | sed -n '1,200p'

printf '\n===== MARKDOWN FILES (FIRST 300) =====\n'
fd -a -t f -e md . | sort | sed -n '1,300p'

printf '\n===== GIT STATE =====\n'
git status --short --branch
git rev-parse --show-toplevel
git remote -v

Length of output: 8867

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="AdaWorldAPI/lance-graph"
pr="944"

echo "===== PR METADATA ====="
gh api "repos/$repo/pulls/$pr" \
  --jq '{number, state, draft, base: .base.ref, head: .head.ref, head_sha: .head.sha, changed_files, additions, deletions}'

echo "===== CHANGED FILES ====="
gh api --paginate "repos/$repo/pulls/$pr/files?per_page=100" \
  --jq '.[] | {filename, status, additions, deletions, patch}'

echo "===== PR DIFF ====="
gh api "repos/$repo/pulls/$pr" -H 'Accept: application/vnd.github.diff' \
  | sed -n '1,2400p'

Length of output: 60752

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="AdaWorldAPI/lance-graph"
ref="97307ebb00d825a4bf225dd0e69115d16e0ee78a"
path=".claude/plans/substrate-comfort-zones-v1.md"

gh api "repos/$repo/contents/$path?ref=$ref" --jq '.content' \
  | tr -d '\n' \
  | base64 --decode \
  | nl -ba \
  | sed -n '150,225p'

echo "===== RELATED COMPARISON WORDING IN PR-HEAD DOCUMENTS ====="
for file in \
  ".claude/plans/substrate-comfort-zones-v1.md" \
  ".claude/board/EPIPHANIES.md" \
  ".claude/board/INTEGRATION_PLANS.md" \
  ".claude/board/STATUS_BOARD.md"
do
  echo "----- $file -----"
  gh api "repos/$repo/contents/$file?ref=$ref" --jq '.content' \
    | tr -d '\n' \
    | base64 --decode \
    | rg -n -i -C 2 \
      'absolute, foreign|foreign donor|D ≠ T|weak form|hardest.*opponent|against the diagonal|which formula wins|comparison of formulas' \
    || true
done

Length of output: 1201

---

🏁 Script executed:

#!/bin/bash
set -euo pipefail

repo="AdaWorldAPI/lance-graph"
ref="97307ebb00d825a4bf225dd0e69115d16e0ee78a"
path=".claude/plans/substrate-comfort-zones-v1.md"

gh api "repos/$repo/contents/$path?ref=$ref" --jq '.content' \
  | tr -d '\n' \
  | base64 --decode \
  | awk '{ printf "%6d  %s\n", NR, $0 }' \
  | sed -n '150,225p'

echo "===== RELATED COMPARISON WORDING IN PR-HEAD DOCUMENTS ====="
for file in \
  ".claude/plans/substrate-comfort-zones-v1.md" \
  ".claude/board/EPIPHANIES.md" \
  ".claude/board/INTEGRATION_PLANS.md" \
  ".claude/board/STATUS_BOARD.md"
do
  echo "----- $file -----"
  gh api "repos/$repo/contents/$file?ref=$ref" --jq '.content' \
    | tr -d '\n' \
    | base64 --decode \
    | rg -n -i -C 2 \
      'absolute, foreign|foreign donor|D ≠ T|weak form|hardest.*opponent|against the diagonal|which formula wins|comparison of formulas' \
    || true
done

Length of output: 9397

---

@AdaWorldAPI, I found one blocking consistency issue.

• .claude/plans/substrate-comfort-zones-v1.md Line 196–202 defines dynamic own-window ρ versus absolute foreign-donor ρ as “the real comparison” and “the operator’s hypothesis.” This contradicts C4, the status board, and the integration entry. Those sections correctly define the diagonal comparison as the hypothesis. They define the foreign-donor comparison as the weak wiring check. Update or remove this §2.3 passage. Otherwise, the plan retains both incompatible verdict criteria, and the swapped-cell comparison again appears to establish merit.

The remaining requested checks are consistent:

• Low L follows from L[D][T] = ρ[T][T] − ρ[D][T]. C3 correctly requires lower mean off-diagonal loss in R4 than in R1.
• C4 correctly uses < 0 in calm and > 0 in storm because higher ρ is better.
• C0, C1, C1b, C1c, C2, C3, C4, C5, and C6 each map once to D-CZ-*. The old-to-new mapping is accurate.
• C2 makes both halves mandatory. It requires exact dynamic degeneracy and a non-degenerate CAL-ABS row through the same code path.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[AdaWorldAPI]

Fixed in 1756c518. The finding is correct and it is the exact defect this PR exists to remove, so it is worth being precise about what went wrong.

Mechanism. §2.3's summary passage was written before C4 was sharpened. When C4 moved to scoring against the diagonal, the sharpening was not propagated backwards. Left standing, the plan carried two incompatible verdict criteria with the weaker one holding the headline — and C2(i) makes the dynamic arm win the foreign comparison almost by construction (an encoding with no anchor cannot be out-transferred by one whose anchor is deliberately wrong), so the swapped cell would have been establishing merit. That is precisely the tautology §2 was rebuilt to delete.

§2.3 now reads against the diagonal, states why (hardest available opponent), and explicitly demotes the foreign comparison to the weak form labelled evidence of wiring rather than merit. The correction is recorded beside it rather than silently patched.

The part worth keeping beyond this PR — added to the E-A-HORSE-RACE-IS-NOT-A-CROSS-SWAP-1 entry:

The defect is sticky. It survived the rewrite built to remove it, and a reviewer caught it, not the author. This is the fifth instance in this arc of a claim that is internally consistent with its own operands but inconsistent with a sibling claim (#930's fused relation, #927's decimal claim, #928's audit figures, #941's dropped qualifier, now this).

The generalization is sharper than any of the individual fixes: when a design is revised, the summary sentences are the last thing to be updated and the first thing a reader believes. A revision is not complete when the new machinery is right — it is complete when every sentence that names the conclusion has been re-derived from the new machinery. Self-review keeps missing it because the author re-reads the part that changed, not the part that merely still sounds right.

Also verified independently while checking: all four remaining foreign mentions in the plan are correct in their contexts (mechanism description, the negative-L case, the weak-form demotion, the correction note), and no other section states the hypothesis against a swapped cell.

The other four checks you confirmed — the L sign convention, C4's inverted inequality, the C-bar → D-CZ 1:1 mapping, C2's both-halves requirement — are unchanged.

One item added in 522efab4 that predates your review and is worth flagging since it touches the same arithmetic: L is not guaranteed non-negative. A box whose own min/max is set by one outlier spreads its 256 levels badly, so a donor with a wider range can legitimately beat the diagonal. Pre-registered as informative rather than as a bug, with the consequence stated — if it occurs, ρ[T][T] is not a valid reference point for that box's L̄[T], and C3's trend must be re-read against the best-available cell with the substitution said out loud.

---

Generated by Claude Code