Copilot harness retries max-turns/invocation-cap exhaustion against a saturated pooled budget (guaranteed instant re-fail; should fast-fail)

[shawnHartsell]

Summary

When a run hits the api-proxy per-run LLM invocation cap (CAPIError: 429 Maximum LLM invocations exceeded (N/N)), the Copilot CLI harness treats it as a retriable partial execution and re-invokes with --continue. But the invocation budget is pooled across all attempts in the Execute step, so it is already saturated at N/N. Every retry therefore re-fails instantly, on its first call, with 0B of new output — the retries can never make progress. The harness should detect this terminal condition and fail fast instead of looping.

Why this is a distinct bug

This is not a request to raise maxRuns or reduce fan-out (that's the workflow author's job, tracked in #39199 / #39751). It's about the harness retry policy: retrying a saturated pooled cap is provably useless work. It:

• wastes wall-clock time (multiple retry attempts that each abort in ~1 call),
• produces confusing logs (attempts 2-N show 429 (N/N) with 0B output, masking that attempt 1 is where the real work — and the real exhaustion — happened),
• and, when a job is left running, presents as an endless retry loop that a human has to cancel manually.

Both #39199 and #39751 noted in passing that "the retry loop cannot recover because the cap is per-run and already saturated," but were closed as not_planned and scoped to cap-sizing + a classifier flag. The retry-behavior fix itself was never tracked.

Concrete example

Workflow: capacity-service-discovery in github/api-platform (Copilot CLI engine, max-turns: 25).

Run 29105746208:

• Attempt 1 ran ~6m4s and legitimately consumed all 25 invocations (25/25), producing partial output (~5990B) before exhausting the budget.
• Attempts 2-4 (--continue) each hit CAPIError: 429 Maximum LLM invocations exceeded (25/25) on their first call, each emitting 0B of new output.
• The job presented as a retry loop and had to be cancelled manually.

Earlier run 29102641110 (same workflow, max-turns: 20) showed the identical pattern at 20/20.

Proposed remediation

1. Fast-fail on the terminal invocation-cap condition. When an attempt returns 429 Maximum LLM invocations exceeded (N/N) and the step's pooled budget is already at N/N, stop retrying — the outcome is deterministic. Surface attempt 1's partial output as the run result rather than burying it under 0B retry aborts.
2. Classify it distinctly (aligns with [aw-failures] P1: Code Simplifier exhausts api-proxy invocation cap (maxRuns 50/50) → CAPIError 429, 6/6 fail + unclassified #39199's ask): add a dedicated flag (e.g. GH_AW_INVOCATION_CAP_EXCEEDED) so this stops being bucketed as a generic exit-1, and so the failure summary can point the author at cap-sizing rather than at a transient error.
3. Log clarity: in the failure summary, distinguish "attempt 1 exhausted the cap after real work" from "attempts 2-N re-failed instantly against the saturated pooled budget," so investigators aren't misled into thinking every attempt did work.

Success criteria

• A run that saturates the pooled invocation cap on attempt 1 does not spawn further --continue retries against the same saturated budget.
• The failure is surfaced with a dedicated invocation-cap classification and attempt-1 partial output preserved.
• No run presents as a manually-cancellable retry loop for this condition.

References

• Related: [aw-failures] P1: Code Simplifier exhausts api-proxy invocation cap (maxRuns 50/50) → CAPIError 429, 6/6 fail + unclassified #39199, [aw-failures] P1: Copilot CLI aborts at CAPI layer — model-not-supported (4 daily workflows, regression) + max-LLM-invocations ( [Content truncated due to length] #39751 (both closed not_planned; cap-sizing + classifier flag, not retry policy)
• Example runs: github/api-platform 29105746208, 29102641110