Skip to content

[Code Quality] Distinguish pre-agent vs long-run 0-turn copilot CLI failures for smarter retry #46506

Description

@github-actions

Description

The Agentic Workflow Audit repeatedly observes two distinct classes of copilot Execute-CLI 0-turn failures that are currently treated identically:

  1. Pre-agent startup failures — agent dies before doing any work (0 tokens, negligible cost)
  2. Long-run then exit failures — agent completes substantial work (30+ minutes, 119–210 AIC) but the CLI wrapper exits non-zero, discarding all completed output

Class 2 wastes significant resources with no recovery path.

From the Agentic Workflow Audit — 2026-07-18 #46492:

"3 of today's fails lost 30–31 min of real agent work + 119–210 aic to a post-work CLI exit. Distinguishing pre-agent startup fails (0-tok) from longrun-then-exit fails, and retrying only the latter, would recover completed work."

Suggested Changes

  • In the copilot engine driver (likely pkg/workflow/engines/copilot/ or similar), add token-usage detection to classify failures:
    • If TokenUsage == 0 at exit: classify as startup failure (no retry needed)
    • If TokenUsage > threshold at non-zero exit: classify as long-run failure (safe-outputs may still be recoverable)
  • Log the distinction in the failure summary for audit visibility.
  • Optionally: on long-run exit, attempt to read any partial safe-output declarations before discarding.

Files Affected

  • pkg/workflow/engines/copilot/ — CLI driver exit handling
  • pkg/workflow/ — run lifecycle / failure classification

Success Criteria

  • Long-run failures (>10K tokens) are distinguishable from startup failures in run logs.
  • Audit reports can report accurate wasted-AIC by failure class.
  • Existing tests pass; no behavior change for pre-agent startup failures.

Source

Extracted from Agentic Workflow Audit — 2026-07-18 #46492

Priority

High — recovers 119–210 AIC per occurrence on long-run failures; directly impacts fleet cost efficiency.

🔍 Task mining by Discussion Task Miner - Code Quality Improvement Agent · 308.6 AIC · ⌖ 5.31 AIC · ⊞ 7K ·

  • expires on Jul 19, 2026, 5:31 PM UTC-08:00

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions