Skip to content

[aw-failures] [P1] PR Sous Chef: Start DIFC Proxy step fails with no captured root cause #52502

Description

@github-actions

Fix PR Sous Chef's DIFC Proxy startup — it's failing with no visible error, blocking every PR-gate run it touches

Affected workflow and run

  • Workflow: PR Sous Chef (.github/workflows/pr-sous-chef.lock.yml)
  • Failed run: §31697879776 (2026-08-13 11:58 UTC), job 94439975204, failed step Start DIFC Proxy
  • Isolated occurrence so far (1/1 in this window) — no existing open agentic-workflows issue mentions "DIFC Proxy" or this step name.

Probable root cause

Unknown — root-cause visibility is currently blocked by the log-capture bug tracked in #52501: the only captured log tail for this run is post-failure cleanup (artifact upload, git config unset, job-output evaluation), with no error line from the Start DIFC Proxy step itself.

Proposed remediation

  1. Once [aw-failures] [P0] Failure-Investigator log capture grabs post-job cleanup tail, not the actual error #52501 ships, re-run and capture the real Start DIFC Proxy error text to pin the actual cause.
  2. In parallel, don't wait on that fix alone — add explicit failure logging and a retry/backoff around the DIFC proxy startup command so the underlying error (timeout, port conflict, missing credential, etc.) surfaces directly in the step output regardless of tail-capture tooling.
  3. If this recurs before root cause is known, capture the full raw log via the Actions UI (not just the tail) to unblock diagnosis immediately.

Success criteria / verification

  • Root cause identified from real error text (not inferred from cleanup logs).
  • PR Sous Chef passes Start DIFC Proxy on its next 3 scheduled/PR-triggered runs after remediation.

Parent report: #52395
Related: #52395
Related to #52395

Generated by 🔍 [aw] Failure Investigator (6h) · agent · 190.4 AIC · ⌖ 56.1 AIC · ⊞ 5.3K ·

  • expires on Aug 20, 2026, 5:25 AM UTC-08:00

Fix PR Sous Chef's DIFC Proxy startup — it's timing out pulling its own container image from ghcr.io

Affected workflow and run

  • Workflow: PR Sous Chef (.github/workflows/pr-sous-chef.lock.yml)
  • Failed run: §31697879776 (2026-08-13 11:58 UTC), job 94439975204, step Start DIFC Proxy, exit code 125
  • Isolated occurrence so far (1/1 in this window).

Root cause (confirmed via agentic-workflows audit)

Starting DIFC proxy container: ghcr.io/github/gh-aw-mcpg:v0.4.9
Unable to find image 'ghcr.io/github/gh-aw-mcpg:v0.4.9' locally
docker: Error response from daemon: Head "https://ghcr.io/v2/github/gh-aw-mcpg/manifests/v0.4.9":
  Get "https://ghcr.io/token?scope=repository:github%2Fgh-aw-mcpg:pull&service=ghcr.io":
  net/http: request canceled (Client.Timeout exceeded while awaiting headers)
##[error]Process completed with exit code 125.

start_difc_proxy.sh does a cold docker run with no locally cached image, no retry, and no explicit pull timeout override. A ~15s ghcr.io auth/manifest fetch stall (transient registry latency, not a policy or auth failure — the request was progressing, just slow) is enough to fail the whole job outright.

Proposed remediation

  1. Add retry-with-backoff around the docker run/docker pull for the DIFC proxy image in start_difc_proxy.sh, matching the pattern needed for the AWF/threat-detect installer in [aw-failures] [P0] Harden AWF/threat-detect binary installers against transient GitHub Releases connection failures #52397.
  2. Pre-pull or cache ghcr.io/github/gh-aw-mcpg:v0.4.9 in a setup step (or warm the runner image) so the proxy-start step never depends on a cold registry fetch during the timing-sensitive part of the job.
  3. Consider raising the Docker client's pull timeout for this specific image if backoff alone isn't enough to absorb ghcr.io latency spikes.

Success criteria / verification

  • PR Sous Chef passes Start DIFC Proxy on its next 3 scheduled/PR-triggered runs.
  • No exit code 125 / Client.Timeout exceeded while awaiting headers recurrence against ghcr.io/github/gh-aw-mcpg over the following 7 days.

Parent report: #52395
Related: #52395, #52397 (same class of fix — retry/backoff around external image/binary pulls)> Generated by 🔍 [aw] Failure Investigator (6h) · agent · 190.4 AIC · ⌖ 56.1 AIC · ⊞ 5.3K ·

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions