[aw-failures] Smoke Gemini agent step fails on every PR — `fetch failed` to Gemini API (generateContentStream / generateJson)

[github-actions]

Problem statement

Smoke Gemini PR runs fail at the agent step on every PR with TypeError: fetch failed sending request from @google/gemini-cli. Two distinct call paths produce identical errors per run:

exception TypeError: fetch failed sending request
  at Models.generateContentStream (.../`@google/gemini-cli`/bundle/chunk-FNPZLVJU.js:37043:16)
  at Turn.run (.../`@google/gemini-cli`/bundle/chunk-UIBQS45C.js:312945:24)

and

exception TypeError: fetch failed sending request
  at Models.generateContent (.../`@google/gemini-cli`/bundle/chunk-FNPZLVJU.js:36994:16)
  at BaseLlmClient.generateJson (.../`@google/gemini-cli`/bundle/chunk-UIBQS45C.js:275465:21)
  at NumericalClassifierStrategy.route (.../`@google/gemini-cli`/bundle/chunk-UIBQS45C.js:335311:28)

The agent JSON result is {"type":"result","status":"error","error":{"type":"unknown","message":"[API Error: exception TypeError: fetch failed sending request]"}} with zero tokens consumed across both gemini-2.5-flash-lite and gemini-3.1-pro-preview models.

Affected workflows and run IDs

• Smoke Gemini — chronic failure on every PR-triggered run for at least 3 days:
  • §25689434393 2026-05-11 18:30 UTC (gemini-client-error-Turn.run-sendMessageStream + gemini-client-error-generateJson-api)
  • §25649467849 2026-05-11 03:56 UTC (identical pair of error files)
  • §25641551143 2026-05-10 22:30 UTC
  • §25605079586 2026-05-09 15:45 UTC
  • §25590903324 2026-05-09 03:49 UTC
• Per-run auto-tracker (latest occurrence): [aw] Smoke Gemini failed #31565

Probable root cause

Firewall analysis from §25689434393 shows heavy BLOCKED traffic against localhost:8080 (98 blocks) and the firewall sidecar 172.30.0.30:10003 (15 blocks), while play.googleapis.com:443 was allowed (20). The Gemini CLI almost certainly tries to reach generativelanguage.googleapis.com or another googleapis.com host that is not on GH_AW_ALLOWED_DOMAINS. The current allow-list (visible in 8_Process Safe Outputs.txt) includes play.googleapis.com, storage.googleapis.com, and www.googleapis.com, but not generativelanguage.googleapis.com or *.googleapis.com.

Alternative hypotheses to rule out:

• Gemini CLI proxy misconfiguration (the CLI routes through localhost:8080 which is being blocked, suggesting MITM/proxy is intercepting and failing).
• Gemini API auth/quota error surfacing as a transport-level fetch failed rather than an HTTP response.

Proposed remediation

1. First fix to try: Add the Gemini API endpoint host(s) to GH_AW_ALLOWED_DOMAINS for the Smoke Gemini workflow. The Gemini CLI 0.x uses generativelanguage.googleapis.com for v1beta and aiplatform.googleapis.com for Vertex; whichever the smoke test invokes must be whitelisted. Verify by re-running and confirming the firewall log shows ALLOWED traffic on that host with non-zero request count.
2. If firewall already allows the relevant host, capture the underlying cause of the fetch failed (Node 24 surfaces it as error.cause — needs to be logged by the Gemini CLI wrapper or via NODE_OPTIONS='--unhandled-rejections=warn' plus a small wrapper).
3. If the proxy (localhost:8080) is the firewall, surface the proxy's reject reason in the agent step log so future investigations don't require firewall log correlation.

Success criteria / verification

• Smoke Gemini agent step completes successfully on a fresh PR.
• No gemini-client-error-*.json artifacts produced.
• Firewall log shows ALLOWED traffic on the Gemini API host(s) used by the engine, with a non-zero successful request count.
• Chronic pattern across 5+ consecutive PR runs ends; auto-tracker [aw] Smoke Gemini failed #31565 stops re-creating.

Parent / context

• Parent failure report: [aw-failures] [aw] Failure Report 2026-05-11 (6h): aw-portfolio-yield workflow defect blocks compile + own runtime + recurrences [Content truncated due to length] #31455
• Related auto-tracker (current run): [aw] Smoke Gemini failed #31565
• Source evidence: gemini-client-error-Turn.run-sendMessageStream-2026-05-11T18-55-35-985Z.json and gemini-client-error-generateJson-api-2026-05-11T18-51-50-527Z.json artifacts in §25689434393; identical pair in §25649467849.

Confidence and unknowns

• High confidence the failure recurs every PR run — identical error files across 5 runs spanning 3 days.
• Medium confidence the root cause is firewall-denied Gemini API host. Strong supporting signal (localhost:8080/172.30.0.30:10003 blocks dominate traffic), but the firewall log does not show the failing host name in the current artifacts.
• Unknown: which exact Gemini API host(s) the engine is targeting in this version of the CLI. Setting DEBUG=gemini-cli:* or capturing error.cause would confirm.

Generated by [aw] Failure Investigator (6h) · ● 16M · ◷

• expires on May 18, 2026, 7:29 PM UTC

[github-actions]

🍪 Issue Monster selected this for Copilot

I've identified this issue as a good candidate for automated resolution and requested assignment to the Copilot coding agent.

If assignment succeeds, the Copilot coding agent will analyze the issue and create a pull request with the fix.

Om nom nom! 🍪

🍪 Om nom nom by Issue Monster · ● 58.5K · ◷

[lpcox]

🔗 AWF tracking issue: https://github.com/github/gh-aw-firewall/issues/github/gh-aw-firewall#2984

Generated by Firewall Issue Dispatcher · ● 938.8K · ◷

[github-actions]

Agent job 25756984886 failed.

⚠️ Engine Failure: The gemini engine terminated unexpectedly.

Last agent output:

    at process.processTicksAndRejections (node:internal/process/task_queues:103:5)
    at async Models.generateContentStream ((redacted)
    at async (redacted)
    at async (redacted)
    at async retryWithBackoff ((redacted)
    at async GeminiChat.makeApiCallAndProcessStream ((redacted)
    at async GeminiChat.streamWithRetries ((redacted)
    at async Turn.run ((redacted)
    at async GeminiClient.processTurn ((redacted)
{"type":"result","timestamp":"2026-05-12T19:55:27.672Z","status":"error","error":{"type":"unknown","message":"[API Error: exception TypeError: fetch failed sending request]"},"stats":{"total_tokens":0,"input_tokens":0,"output_tokens":0,"cached":0,"input":0,"duration_ms":0,"tool_calls":0,"models":{"gemini-2.5-flash-lite":{"total_tokens":0,"input_tokens":0,"output_tokens":0,"cached":0,"input":0},"gemini-3.1-pro-preview":{"total_tokens":0,"input_tokens":0,"output_tokens":0,"cached":0,"input":0}}}}

Generated from Smoke Gemini · ◷