⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.
Context
src/selfhost/ai.ts:936-939 computes const level = input.finalAttempt === false ? "warn" : "error";, with a doc comment explaining the contract: "only the FINAL attempt of a caller's own
retry loop is Sentry-visible (error); a retried attempt logs at warn ... a single-shot caller that
never sets this field keeps today's always-loud behavior."
src/services/ai-slop.ts:163, src/services/ai-review.ts:1228,2013, and
src/services/issue-plan-draft.ts:212 all correctly compute finalAttempt: attempt === LAST && modelIndex === LAST_MODEL on their multi-attempt ai.run() calls, honoring the contract.
src/services/ai-e2e-test-gen.ts:203-207 and src/services/linked-issue-satisfaction-run.ts:89-93
both run the identical 2-model × 3-attempt retry loop shape but never set finalAttempt in the
options object passed to ai.run. Per the contract above, an unset finalAttempt is reserved for
genuinely single-shot callers and always logs loud (error) — but neither of these two functions is
single-shot; both retry up to 6 times per invocation. As a result, every transient failure that
eventually succeeds on a later retry still logs a Sentry-visible error for each failed earlier
attempt in these two functions specifically, contrary to the documented intent that a retried
attempt should log at quiet warn.
This is an observability/noise bug (inflated Sentry error volume specifically for these two
features), not a functional one — the underlying retry/fallback behavior itself works correctly.
Requirements
- Add
finalAttempt: attempt === LAST_ATTEMPT && modelIndex === LAST_MODEL_INDEX (or the equivalent
computation matching this file's own loop variables) to both ai-e2e-test-gen.ts's and
linked-issue-satisfaction-run.ts's ai.run() call options, mirroring the pattern already used by
ai-slop.ts/ai-review.ts/issue-plan-draft.ts exactly.
Deliverables
All four Deliverables are required in the same PR.
Test Coverage Requirements
src/** is measured by codecov/patch (99%+ target, branch-counted). The new tests must exercise
both the non-final (warn) and final (error) branches for both functions, matching
ai-slop.ts's existing test pattern for the same distinction.
Expected Outcome
ai-e2e-test-gen.ts and linked-issue-satisfaction-run.ts correctly log a retried (non-final)
attempt's failure at quiet warn, matching every other multi-attempt AI-calling function in the
codebase, eliminating inflated Sentry error-volume noise for these two features specifically.
Links & Resources
src/selfhost/ai.ts:936-939 (the documented finalAttempt contract)
src/services/ai-e2e-test-gen.ts:203-207, src/services/linked-issue-satisfaction-run.ts:89-93
(the two functions to fix)
- Already-correct siblings:
src/services/ai-slop.ts:163, src/services/ai-review.ts:1228,2013,
src/services/issue-plan-draft.ts:212
Context
src/selfhost/ai.ts:936-939computesconst level = input.finalAttempt === false ? "warn" : "error";, with a doc comment explaining the contract: "only the FINAL attempt of a caller's ownretry loop is Sentry-visible (error); a retried attempt logs at warn ... a single-shot caller that
never sets this field keeps today's always-loud behavior."
src/services/ai-slop.ts:163,src/services/ai-review.ts:1228,2013, andsrc/services/issue-plan-draft.ts:212all correctly computefinalAttempt: attempt === LAST && modelIndex === LAST_MODELon their multi-attemptai.run()calls, honoring the contract.src/services/ai-e2e-test-gen.ts:203-207andsrc/services/linked-issue-satisfaction-run.ts:89-93both run the identical 2-model × 3-attempt retry loop shape but never set
finalAttemptin theoptions object passed to
ai.run. Per the contract above, an unsetfinalAttemptis reserved forgenuinely single-shot callers and always logs loud (
error) — but neither of these two functions issingle-shot; both retry up to 6 times per invocation. As a result, every transient failure that
eventually succeeds on a later retry still logs a Sentry-visible
errorfor each failed earlierattempt in these two functions specifically, contrary to the documented intent that a retried
attempt should log at quiet
warn.This is an observability/noise bug (inflated Sentry error volume specifically for these two
features), not a functional one — the underlying retry/fallback behavior itself works correctly.
Requirements
finalAttempt: attempt === LAST_ATTEMPT && modelIndex === LAST_MODEL_INDEX(or the equivalentcomputation matching this file's own loop variables) to both
ai-e2e-test-gen.ts's andlinked-issue-satisfaction-run.ts'sai.run()call options, mirroring the pattern already used byai-slop.ts/ai-review.ts/issue-plan-draft.tsexactly.Deliverables
ai-e2e-test-gen.ts'sai.run()call setsfinalAttemptcorrectly for every attempt/modelcombination in its retry loop.
linked-issue-satisfaction-run.ts's equivalent call has the same fix.env.AI.runto fail once then succeed, and asserts thefailing call's logged level is
warn(i.e.finalAttempt: falsewas passed) — pinning thesame convention
ai-slop.ts's existing test already pins.exhausted) logs at
error(finalAttempt: true).All four Deliverables are required in the same PR.
Test Coverage Requirements
src/**is measured bycodecov/patch(99%+ target, branch-counted). The new tests must exerciseboth the non-final (
warn) and final (error) branches for both functions, matchingai-slop.ts's existing test pattern for the same distinction.Expected Outcome
ai-e2e-test-gen.tsandlinked-issue-satisfaction-run.tscorrectly log a retried (non-final)attempt's failure at quiet
warn, matching every other multi-attempt AI-calling function in thecodebase, eliminating inflated Sentry error-volume noise for these two features specifically.
Links & Resources
src/selfhost/ai.ts:936-939(the documentedfinalAttemptcontract)src/services/ai-e2e-test-gen.ts:203-207,src/services/linked-issue-satisfaction-run.ts:89-93(the two functions to fix)
src/services/ai-slop.ts:163,src/services/ai-review.ts:1228,2013,src/services/issue-plan-draft.ts:212