Skip to content

[aw-failures] safe_outputs job hard-fails entire batch on one non-retryable error #53263

Description

@github-actions

Problem

Stop letting one bad safe-output item sink an entire batch — two workflows this window had a safe_outputs job fail outright even though most of the batch's outputs had already succeeded.

Affected workflows and runs

  • PR Sous Chef: §31980573442 (2026-08-16 23:57 UTC) — update_pull_request on a stacked PR failed with Updating a stacked PR's branch via this endpoint is not supported (GitHub API, non-retryable). Other outputs in the same batch (issue create, comment post) succeeded before the job was marked failed.
  • Smoke Copilot - AOAI (apikey): §31974722434 (2026-08-16 21:51 UTC) — dispatch_workflow targeting haiku-printer failed with Required input 'message' not provided. 11/13 other outputs in the same batch succeeded before the job was marked failed.

Probable root cause

Both are deterministic, non-retryable per-item errors (a GitHub API limitation on stacked/dependent PR branch updates, and a missing required workflow-dispatch input) that the safe_outputs processor correctly flags as non-retryable, but still propagates as an overall job failure rather than a partial-success outcome. This is the same shape as the resolve_pull_request_review_thread hard-fail already flagged for PR Sous Chef in the prior 6h window's report (#53129) — one bad item failing the whole job is a recurring pattern across output types, not isolated to one tool.

Proposed remediation

  1. For update_pull_request: detect stacked-PR branch-update rejections and downgrade to a warning (skip just the branch-update sub-action) instead of failing the job, since the rest of the update (title/body/labels) can still apply independently.
  2. For dispatch_workflow: validate required inputs for the target workflow before attempting dispatch, and surface a clear pre-flight error to the calling agent instead of a hard API failure after the fact.
  3. More generally: audit whether safe_outputs should report partial_success (already computed in the step summary) without also setting a hard job failure, when at least one output in the batch succeeded.

Success criteria / verification

Generated by 🔍 [aw] Failure Investigator (6h) · agent · 237.9 AIC · ⌖ 8.49 AIC · ⊞ 5.6K ·

  • expires on Aug 23, 2026, 5:32 PM UTC-08:00

New corroborating instance — 2026-08-17 18:08 UTC

Treat this as a 4th confirmed instance of the same bug, not a new issue — the trigger differs but the failure shape is identical: one non-retryable per-item error hard-fails the whole safe_outputs job despite most of the batch succeeding.

  • PR Sous Chef: §32052943209 (2026-08-17 18:08 UTC) — add_comment failed with 503 Service Unavailable: No server is currently available to service your request (transient GitHub API outage, ~17:45–18:19 UTC). 8/9 safe-output messages succeeded (Status: partial_success) before the job was still marked failed.

Stacked-PR rejection, missing dispatch input, and now a plain GitHub API 503 all hit the identical hard-fail path — the bug is trigger-agnostic. Prioritize the partial_success-without-job-failure fix already proposed here; it closes all three trigger variants at once.

Related transient-outage siblings in the same 17:45–18:19 UTC window (not separately actionable — same external root cause, no gh-aw bug involved): Copilot Agent PR Analysis §32054193614, Ponytail Reviewer §32051718180.> Generated by 🔍 [aw] Failure Investigator (6h) · agent · 157.7 AIC · ⌖ 9.12 AIC · ⊞ 5.8K ·


Update — 2026-08-19 (6h window)

Still broken — 2 more occurrences today, same safe_outputs / "Process Safe Outputs" signature, no fix landed yet (no commits touching safe-outputs processing code since this issue opened):

Keeping open — priority unchanged.> Generated by 🔍 [aw] Failure Investigator (6h) · agent · 167.2 AIC · ⌖ 10.4 AIC · ⊞ 5.9K ·


Confirmed — 2 more PR Sous Chef runs hit the same unhandled approve_workflow_run type

One-sentence rationale: the safe_outputs job still hard-fails with "No handler loaded" for approve_workflow_run, exactly matching this issue.

Fresh occurrences (2026-08-20): PR Sous Chef — §32389691755, §32381373575

No new information beyond confirming recurrence — keep existing priority and remediation plan.> Generated by 🔍 [aw] Failure Investigator (6h) · agent · 238.8 AIC · ⌖ 7.28 AIC · ⊞ 5.9K ·


Fresh occurrence — ESLint Miner hits the same safe_outputs hard-fail shape

One-sentence rationale: the safe_outputs job failed outright on a per-item error while other outputs in the batch had already succeeded — same hard-fail shape this issue tracks.

  • ESLint Miner — §32563331906 (2026-08-22 08:50 UTC) — Process Safe Outputs step failed with process_safe_outputs_items_failed > 0 alongside non-zero code_push_failure_count / create_discussion_error_count outputs, indicating a per-item failure (code push and/or discussion creation) hard-failed the whole batch.

No fix has landed yet (no commits touching safe-outputs batch processing since this issue opened 2026-08-17). Keeping open — priority unchanged. Apply the partial_success-without-job-failure fix already proposed here; it would have prevented this occurrence too.

References:

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions