⚠️ Definition of Done: this issue must be completed in full, in a single PR. Do not split this
work across multiple PRs, and do not defer any Deliverable below to a follow-up issue. A PR that
satisfies only some of the Deliverables, stubs a required test, or leaves a checkbox
partially-done does NOT resolve this issue and will be closed.
Context
src/services/ai-e2e-test-gen.ts:195's own doc comment says it "mirrors runWorkersSlopOpinion's
exact shape." Its per-model retry loop's catch block (lines 200-214, specifically line 210,
catch { /* retry / fall through to fallback */ }) has no rate-limit check at all —
isRateLimitError is never imported anywhere in this file.
Every other near-identical per-model retry loop in the codebase correctly short-circuits on a rate
limit:
src/services/ai-slop.ts:169-174 — } catch (error) { if (isRateLimitError(error)) break; }
src/review/planner.ts:137
src/services/issue-plan-draft.ts:219
src/services/linked-issue-satisfaction-run.ts:98
isRateLimitError's own doc comment (src/services/ai-review.ts:1109-1120) states it exists
specifically so "every independent AI-calling retry loop ... can share this one definition," and
that retrying the same model against a live 429 "has near-zero chance of success."
Consequence: when the default AI provider 429s while generating an E2E test,
ai-e2e-test-gen.ts burns all E2E_TEST_GEN_ATTEMPTS_PER_MODEL (3) retries against the same
rate-limited model — using half of its total pre-budgeted E2E_TEST_GEN_MAX_CALLS (6) on attempts
documented elsewhere in this same codebase as having zero chance of success — before ever reaching
the fallback model, adding avoidable latency and wasted quota.
Requirements
- Import
isRateLimitError into ai-e2e-test-gen.ts and add the same break on a detected rate
limit that every sibling retry loop already has, mirroring ai-slop.ts:169-174 exactly.
- Do not change behavior for any non-rate-limit failure (those should continue to retry as today).
Deliverables
All three Deliverables are required in the same PR.
Test Coverage Requirements
src/** is measured by codecov/patch (99%+ target, branch-counted). Both new tests must exercise
the rate-limit and non-rate-limit branches directly, matching ai-slop.ts's existing test pattern
for the same distinction.
Expected Outcome
ai-e2e-test-gen.ts's retry behavior on a rate-limited model matches every other AI-calling retry
loop in the codebase: immediate fallback on a detected rate limit, full retry budget preserved for
genuinely retryable failures.
Links & Resources
src/services/ai-e2e-test-gen.ts:195-214 (the retry loop to fix)
src/services/ai-slop.ts:169-174 (the already-correct pattern to mirror)
src/services/ai-review.ts:1109-1120 (isRateLimitError, the shared helper to reuse)
- Other correct siblings:
src/review/planner.ts:137, src/services/issue-plan-draft.ts:219,
src/services/linked-issue-satisfaction-run.ts:98
Context
src/services/ai-e2e-test-gen.ts:195's own doc comment says it "mirrorsrunWorkersSlopOpinion'sexact shape." Its per-model retry loop's
catchblock (lines 200-214, specifically line 210,catch { /* retry / fall through to fallback */ }) has no rate-limit check at all —isRateLimitErroris never imported anywhere in this file.Every other near-identical per-model retry loop in the codebase correctly short-circuits on a rate
limit:
src/services/ai-slop.ts:169-174—} catch (error) { if (isRateLimitError(error)) break; }src/review/planner.ts:137src/services/issue-plan-draft.ts:219src/services/linked-issue-satisfaction-run.ts:98isRateLimitError's own doc comment (src/services/ai-review.ts:1109-1120) states it existsspecifically so "every independent AI-calling retry loop ... can share this one definition," and
that retrying the same model against a live 429 "has near-zero chance of success."
Consequence: when the default AI provider 429s while generating an E2E test,
ai-e2e-test-gen.tsburns allE2E_TEST_GEN_ATTEMPTS_PER_MODEL(3) retries against the samerate-limited model — using half of its total pre-budgeted
E2E_TEST_GEN_MAX_CALLS(6) on attemptsdocumented elsewhere in this same codebase as having zero chance of success — before ever reaching
the fallback model, adding avoidable latency and wasted quota.
Requirements
isRateLimitErrorintoai-e2e-test-gen.tsand add the samebreakon a detected ratelimit that every sibling retry loop already has, mirroring
ai-slop.ts:169-174exactly.Deliverables
ai-e2e-test-gen.ts's per-model retry loop breaks out of retrying the same model immediatelyon a detected rate-limit error, moving to the fallback model instead of burning all 3
attempts.
env.AI.runto throw a rate-limit-shaped error for the first model on everycall, and asserts the first model is called at most once before the second (fallback) model is
attempted — mirroring
ai-slop.ts's existing rate-limit test.count for the same model (proving the fix doesn't over-broaden the break condition).
All three Deliverables are required in the same PR.
Test Coverage Requirements
src/**is measured bycodecov/patch(99%+ target, branch-counted). Both new tests must exercisethe rate-limit and non-rate-limit branches directly, matching
ai-slop.ts's existing test patternfor the same distinction.
Expected Outcome
ai-e2e-test-gen.ts's retry behavior on a rate-limited model matches every other AI-calling retryloop in the codebase: immediate fallback on a detected rate limit, full retry budget preserved for
genuinely retryable failures.
Links & Resources
src/services/ai-e2e-test-gen.ts:195-214(the retry loop to fix)src/services/ai-slop.ts:169-174(the already-correct pattern to mirror)src/services/ai-review.ts:1109-1120(isRateLimitError, the shared helper to reuse)src/review/planner.ts:137,src/services/issue-plan-draft.ts:219,src/services/linked-issue-satisfaction-run.ts:98