self-healing: transient merge failures never recovered on a renamed board (twenty-third sweep) by gsxdsm · Pull Request #2916 · Runfusion/Fusion

gsxdsm · 2026-07-31T00:31:29Z

recoverTransientMergeFailures refunds the retry budget for a merge that failed for a transient reason and burned all its retries. The literal read meant that on a renamed board the refund never happened, so a card that failed on a network blip or a provider fault stayed failed permanently — visibly failed to the operator, with no visible cause.

Two lane guards, and the test caught the second

I converted the read and the slim-snapshot filter, and the new case still failed. There is a second guard: after the full row is re-read, task.column !== "in-review" re-checks the selector. Converting only the first left it rejecting every renamed-board card the widened query had just found.

That is the "convert the pair or neither" rule from the sibling doc, and it was caught by running the test, not by reading the function — which is the argument for the test existing at all. A conversion that stops at the first guard looks complete and delivers nothing.

Fixture note

The error string is a real transient signature — "ACP turn failed", which classifyTransientMergeError matches — not invented prose. An unrecognised string is filtered out one line later, so the case would pass with the fix reverted. Same trap as the post-done wedge fixture earlier in this series.

Revert results

Each applied alone and the file re-run:

conversion	reverted →
the resolved read + both guards	fails — the card is never listed
the second guard alone	fails — the re-read row is rejected

The second row is the one worth having: it proves the half I nearly shipped unconverted.

A non-vacuous companion (transient failure on a wip card → no refund) rules out a read that returns everything; a card that never reached merge has no merge budget to refund.

Verification

pnpm test:gate 161 + 487 + 13 + 71, plus self-healing.test.ts 412; tsc engine clean; pnpm lint, check:changesets, census --strict and check-sql-column-literals clean, each run explicitly.

coderabbitai · 2026-07-31T00:31:38Z

Warning

Review limit reached

@gsxdsm, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 24 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 4034af96-ddd1-4ede-a66c-355d9a9dbcfd

📥 Commits

Reviewing files that changed from the base of the PR and between c6767cb and 41b2c51.

📒 Files selected for processing (4)

.changeset/self-healing-transient-merge-query.md
packages/engine/src/__tests__/self-healing-query-filter-blindness.test.ts
packages/engine/src/self-healing.ts
scripts/lib/lifecycle-column-census-baseline.json

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

… plus a ratchet SELF-AUDIT triggered by #2916, where the behavioural test caught a SECOND lane guard on a re-read row. I scanned the other sweeps I had converted and found five more in this one, all inside the loop body under the widened read. One of them matters a great deal: `if (task.column === "in-review")` decides whether a backward move needs its triple-proof. Left literal, a renamed review card admitted by the widened read reads as NOT-in-review, so the proof gate is skipped and the card is moved back without it — a safety check silently bypassed BY the conversion. The other four gate the resume-limbo path the same way. MEASURED: self-healing.ts column guards 86 -> 81. Adds a ratchet for the class, because a scan found it and a scan can keep finding it. It DERIVES the sweep list (a sweep is converted when its body calls resolveProjectColumnsForRoles) rather than hardcoding one, so it is correct on every branch and cannot go stale. Three refinements it needed, each from a real false positive: - strip comments: the FNXC notes quote the old form to explain its removal - excuse the FALLBACK arm of a resolved ternary (`own.length > 0 ? own.includes(c) : c === "in-review"`) — the correct shape, and the first version reported all three such lines as defects - allow the one documented literal in clearStaleBlockedBy's log-dedup closure Two positive controls, because both halves fail silently: a broken regex finds no offenders, and a broken derivation iterates nothing — an empty for-loop registers no tests and reads as green. Revert measured: restoring any one of the five fails the ratchet, naming the sweep and the exact line. Fusion-Task-Id: KB-SELF-HEALING-QUERIES

… board (twenty-fourth sweep) recoverStaleIncompleteReviewTasks requeues a review card whose STEPS are not finished — it reached review on a graph failure, not on completed work. The literal read meant that on a renamed board it was never requeued, so the card sat in review claiming to be done while its own steps said otherwise. Checked for a second pair deliberately: the triple-proof here is NOT lane-gated, so unlike #2916 (a second guard on a re-read row) and #2879 (five in the loop body) there is nothing else to convert. The audit ratchet added in #2879 confirms it. The requeue keeps its literal `todo` — recoveryRehome: true, one of the 22 documented escapes. Reverts measured, each alone: - literal read restored -> fails, the card is never listed - verdict back to `task.column === "in-review"` -> fails, the renamed review lane is filtered out Fusion-Task-Id: KB-SELF-HEALING-QUERIES

… plus a ratchet SELF-AUDIT triggered by #2916, where the behavioural test caught a SECOND lane guard on a re-read row. I scanned the other sweeps I had converted and found five more in this one, all inside the loop body under the widened read. One of them matters a great deal: `if (task.column === "in-review")` decides whether a backward move needs its triple-proof. Left literal, a renamed review card admitted by the widened read reads as NOT-in-review, so the proof gate is skipped and the card is moved back without it — a safety check silently bypassed BY the conversion. The other four gate the resume-limbo path the same way. MEASURED: self-healing.ts column guards 86 -> 81. Adds a ratchet for the class, because a scan found it and a scan can keep finding it. It DERIVES the sweep list (a sweep is converted when its body calls resolveProjectColumnsForRoles) rather than hardcoding one, so it is correct on every branch and cannot go stale. Three refinements it needed, each from a real false positive: - strip comments: the FNXC notes quote the old form to explain its removal - excuse the FALLBACK arm of a resolved ternary (`own.length > 0 ? own.includes(c) : c === "in-review"`) — the correct shape, and the first version reported all three such lines as defects - allow the one documented literal in clearStaleBlockedBy's log-dedup closure Two positive controls, because both halves fail silently: a broken regex finds no offenders, and a broken derivation iterates nothing — an empty for-loop registers no tests and reads as green. Revert measured: restoring any one of the five fails the ratchet, naming the sweep and the exact line. Fusion-Task-Id: KB-SELF-HEALING-QUERIES

… board (twenty-fourth sweep) recoverStaleIncompleteReviewTasks requeues a review card whose STEPS are not finished — it reached review on a graph failure, not on completed work. The literal read meant that on a renamed board it was never requeued, so the card sat in review claiming to be done while its own steps said otherwise. Checked for a second pair deliberately: the triple-proof here is NOT lane-gated, so unlike #2916 (a second guard on a re-read row) and #2879 (five in the loop body) there is nothing else to convert. The audit ratchet added in #2879 confirms it. The requeue keeps its literal `todo` — recoveryRehome: true, one of the 22 documented escapes. Reverts measured, each alone: - literal read restored -> fails, the card is never listed - verdict back to `task.column === "in-review"` -> fails, the renamed review lane is filtered out Fusion-Task-Id: KB-SELF-HEALING-QUERIES

…ard (twenty-third sweep) recoverTransientMergeFailures refunds the retry budget for a merge that failed for a TRANSIENT reason and burned all its retries. The literal read meant that on a renamed board the refund never happened, so a card that failed on a network blip or a provider fault stayed failed permanently — visibly failed to the operator, with no visible cause. TWO lane guards here, not one, and the test caught the second. I converted the read and the slim-snapshot filter, and the new case still failed: a SECOND guard re-checks `task.column !== "in-review"` on the full row after the re-read. Converting only the first left it rejecting every renamed-board card the widened query had just found — the "convert the pair or neither" rule, caught by running the test rather than by reading the function. Fixture note: the error string is a REAL transient signature ("ACP turn failed", which classifyTransientMergeError matches), not invented prose. An unrecognised string is filtered out one line later, and the case would then pass with the fix reverted. Reverts measured, each alone: - literal read + both guards restored -> fails, the card is never listed - the SECOND guard alone restored -> fails, the re-read row is rejected Fusion-Task-Id: KB-SELF-HEALING-QUERIES

gsxdsm · 2026-07-31T01:34:37Z

Superseded by #2944 — folded into batch-self-healing-renamed-boards with the other 22 renamed-board sweeps. Same root cause, same file; 23 CI runs for one file was the queue jam.

The conversion and its revert measurements are carried over in the commit message. Nothing here is dropped.

… plus a ratchet SELF-AUDIT triggered by #2916, where the behavioural test caught a SECOND lane guard on a re-read row. I scanned the other sweeps I had converted and found five more in this one, all inside the loop body under the widened read. One of them matters a great deal: `if (task.column === "in-review")` decides whether a backward move needs its triple-proof. Left literal, a renamed review card admitted by the widened read reads as NOT-in-review, so the proof gate is skipped and the card is moved back without it — a safety check silently bypassed BY the conversion. The other four gate the resume-limbo path the same way. MEASURED: self-healing.ts column guards 86 -> 81. Adds a ratchet for the class, because a scan found it and a scan can keep finding it. It DERIVES the sweep list (a sweep is converted when its body calls resolveProjectColumnsForRoles) rather than hardcoding one, so it is correct on every branch and cannot go stale. Three refinements it needed, each from a real false positive: - strip comments: the FNXC notes quote the old form to explain its removal - excuse the FALLBACK arm of a resolved ternary (`own.length > 0 ? own.includes(c) : c === "in-review"`) — the correct shape, and the first version reported all three such lines as defects - allow the one documented literal in clearStaleBlockedBy's log-dedup closure Two positive controls, because both halves fail silently: a broken regex finds no offenders, and a broken derivation iterates nothing — an empty for-loop registers no tests and reads as green. Revert measured: restoring any one of the five fails the ratchet, naming the sweep and the exact line. Fusion-Task-Id: KB-SELF-HEALING-QUERIES

…olds 23 PRs) (#2944) **Consolidation of 23 open PRs into one.** Every one shared a single root cause and mostly touched a single file; 23 CI runs for that was indefensible. Folds and supersedes: #2867 #2869 #2876 #2879 #2883 #2891 #2899 #2901 #2902 #2905 #2906 #2914 #2916 #2918 #2919 #2920 #2922 #2927 #2929 #2932 #2934 #2937 #2939. (#2865, #2882, #2897, #2909, #2912 already merged and are not re-folded.) ## The root cause A self-healing sweep selects its work with `listTasks({ column: "in-review" })`. On a board whose lanes are renamed that returns **nothing**, so the sweep never runs — no error, no log line, no failed task. Several sweeps had already had their *predicates* converted to resolved lanes, which dropped a census count and changed nothing, because the query above the loop had already returned an empty list. **26 sweeps converted.** Each one: read the project's columns for the role, then decide each card against **its own** workflow, with the legacy ids unioned so a board mid-rename is never skipped. ## What each sweep stops silently failing to do | | | | --- | --- | | stale merger status | one finished card held the **merge queue** for everything behind it | | stale `blockedBy` / completed-task release | dependents stayed blocked on work that had already finished — the board stops moving | | workspace partial lands | a task left with **some repos merged and some not** | | mid-merge retry stamp | the card stalled *and* the operator's manual Retry was gated by the same stamp | | in-progress limbo / no-progress failures | dead cards held a work slot forever | | partial-progress retry | real work parked failed with its **retry budget unspent** | | orphaned-execution signal | visibility only — the one signal pointing at an orphan went silent | | zero-commit audit | went **half-blind**: the error arm kept working, the lane arm did not | Plus: ghost review cards, transient merge failures, misclassified failures, branch misbinding, missing-worktree failures, merged-but-unfinished finalization, done-metadata repair, self-owned branch conflicts, orphan-only scope violations, post-done wedges, idle assigned agents, PR-conflict worktree ownership, and orphaned workspace worktrees. ## Two defects the conversion itself introduced, both caught and fixed 1. **Missed pairs.** Widening a read without converting the guards beneath it is *worse than not converting*: the sweep starts admitting renamed-board cards and then mis-decides every one. Review caught a second guard on a re-read row; the audit that triggered found **five more**, one of which gates the `reviewProof` triple-proof — a renamed review card would have been moved backward with the safety check silently skipped. Column guards 86 → 81. 2. **Duplicate processing.** The literal reads were disjoint by construction; resolved reads are not, so a column carrying two role flags put one card in two buckets — duplicate moves, duplicate audit rows, inflated counts. Both now have ratchets. `self-healing-converted-sweeps-have-no-literal-lane-guards.test.ts` **derives** its sweep list (a sweep counts as converted when its body calls `resolveProjectColumnsForRoles`), so it cannot go stale, and it carries two positive controls because a broken regex finds no offenders and a broken derivation iterates nothing — an empty loop registers no tests and reads green. ## Deliberately unchanged - 22 `moveTask` destinations carrying `recoveryRehome: true` — `moves.ts` exempts these so a card stranded in an undeclared column stays rescuable. - One literal in `clearStaleBlockedBy`'s log-dedup closure (allowed by name in the ratchet, with the reason). - `surfaceInReviewStalls` — hot list-read path, needs a batched prefetch; that is a performance design decision, not a conversion. - `scheduler.ts` and `replan-target.ts` — built on `resolveTaskWorkflowIrSync`, which returns the default IR for every task in production. Converting there produces inert code. ## The fold itself is worth one note All 23 branches appended to the **same test file at the same anchor**, so every automatic strategy — git 3-way, `merge-file --union`, and three hand-written resolvers — interleaved them mid-block. Two attempts committed conflict markers before I caught it. The file is therefore **reconstructed**: head authored once, body assembled as the union of each branch's own intact top-level segments keyed by test title, with the nested `already-merged hard blocker` describe appended whole (flattening it orphaned its helper). Verified by *parsing after every step* rather than trusting the merge — which is how each interleaving was caught. ## Verification `pnpm test:gate` 161 + 487 + 13 + 71. Scoped suites 592 passed (self-healing, the blindness suite at 68 cases, the ratchet, and the notification suite). `tsc` engine clean; `pnpm lint`, `check:changesets`, `lifecycle-column-census --strict` and `check-sql-column-literals` all clean, each run explicitly. Each folded conversion was individually revert-proven on its original branch — the read reverted alone, and the per-card verdict reverted alone — and those measurements are recorded in the commit messages carried into this branch. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

gsxdsm force-pushed the shq34 branch from 343899b to 0a73580 Compare July 31, 2026 01:23

gsxdsm mentioned this pull request Jul 31, 2026

batch: self-healing sweeps were blind on renamed boards (26 sweeps, folds 23 PRs) #2944

Merged

gsxdsm force-pushed the shq34 branch from 0a73580 to 41b2c51 Compare July 31, 2026 01:34

gsxdsm closed this Jul 31, 2026

Navigation Menu

Uh oh!

self-healing: transient merge failures never recovered on a renamed board (twenty-third sweep) - #2916

self-healing: transient merge failures never recovered on a renamed board (twenty-third sweep)#2916
gsxdsm wants to merge 1 commit into
mainfrom
shq34

gsxdsm commented Jul 31, 2026

Uh oh!

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading

Review limit reached

Uh oh!

gsxdsm commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

gsxdsm commented Jul 31, 2026

Two lane guards, and the test caught the second

Fixture note

Revert results

Verification

Uh oh!

coderabbitai Bot commented Jul 31, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

gsxdsm commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Jul 31, 2026 •

edited

Loading