Skip to content

Smoke Test: OTEL Backends - 25863489297 #32133

Description

@github-actions

Executive Summary: FAIL

The local OTEL emit side shows spans for this run but recorded 2 OTLP export errors. Both Sentry and Grafana backends failed with auth errors (403/401 respectively), so end-to-end visibility cannot be confirmed.

Overall verdict: FAIL


Checklist

  • Local send verification — INCONCLUSIVE (spans present, 2 export errors)
  • Sentry query verification — FAIL (HTTP 403 — permission denied)
  • Grafana query verification — FAIL (HTTP 401 — unauthorized)

Evidence

Local OTEL Emission

  • OTEL_EXPORTER_OTLP_ENDPOINT: set
  • OTEL_EXPORTER_OTLP_HEADERS: set
  • GH_AW_OTLP_ENDPOINTS: set
  • OTEL_SERVICE_NAME: gh-aw
  • COPILOT_OTEL_FILE_EXPORTER_PATH: /tmp/gh-aw/copilot-otel.jsonl
  • /tmp/gh-aw/otel.jsonl: 1 line, span gh-aw.agent.setup with run_id=25863489297 confirmed
  • /tmp/gh-aw/copilot-otel.jsonl: 13 lines, Copilot-instrumented spans for traceId=c6d9cf819c6c1ecce4d5317c903307d1
  • /tmp/gh-aw/otlp-export-errors.count: 2 ← export errors present

Decision: send_status = inconclusive — span for current run exists locally but 2 OTLP export errors indicate the remote push may have partially or fully failed.


Sentry

  • MCP bridge connection: OK (HTTP 200 on initialize)
  • sentry find_organizations: HTTP 403 — You do not have permission to perform this action
  • No project lookup possible; no span query possible
  • No fallback query attempted (blocked by same 403)

Decision: sentry_status = fail — Sentry MCP is reachable but the configured token lacks find_organizations permission.


Grafana

  • MCP bridge connection: OK (HTTP 200 on initialize)
  • grafana list_datasources: HTTP 401 — Unauthorized
  • No tracing datasource discovered; no span query possible
  • No fallback query attempted (blocked by same 401)

Decision: grafana_status = fail — Grafana MCP is reachable but the configured service account token is invalid or missing.


Blockers

🔴 Sentry — HTTP 403 on find_organizations

Problem: The SENTRY_ACCESS_TOKEN secret does not have sufficient permissions to list organizations via the Sentry API.

Action: Grant the Sentry token org:read scope (minimum) on the github organization, or rotate SENTRY_ACCESS_TOKEN with a token that has that scope.

🔴 Grafana — HTTP 401 on list_datasources

Problem: The GRAFANA_SERVICE_ACCOUNT_TOKEN secret is either missing, expired, or revoked — the Grafana MCP returns 401 Unauthorized.

Action: Rotate GRAFANA_SERVICE_ACCOUNT_TOKEN in the repository secrets with a valid Grafana service-account token that has Viewer role or higher.

🟡 OTLP Export Errors (send side)

Problem: /tmp/gh-aw/otlp-export-errors.count = 2. Spans were generated locally but at least 2 batches failed to export to the remote OTLP endpoint.

Action: Check whether GH_AW_OTLP_ENDPOINTS destinations are reachable from the runner. Verify the value of OTEL_EXPORTER_OTLP_HEADERS has not expired or been rotated.


Run URL

https://github.com/github/gh-aw/actions/runs/25863489297

Generated by Smoke OTEL Backends for issue #32126 · ● 3M ·

  • expires on May 14, 2026, 3:54 PM UTC

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions