Skip to content

[awf] api-proxy: modelFallback rewrites Copilot model names, obscuring retired/restricted model errors #3987

Description

@lpcox

Problem

When a gh-aw workflow sets engine.id: copilot with a retired or organization-restricted model, the AWF api-proxy sidecar's modelFallback logic can silently rewrite the model name, and the resulting failure surface is a provider authentication error rather than a model-availability message.

Context

Upstream report: github/gh-aw#35032

Key AWF-side factors:

  • apiProxy.modelFallback.strategy = middle_power (default) rewrites models not in the cached catalog. For the Copilot engine, the rewritten name may not match any server-side deployment, causing a 404 DeploymentNotFound (also observed in gh-aw-firewall#3975 for BYOK Azure).
  • gh-aw passes COPILOT_MODEL=gpt-5-codex raw to the Copilot CLI. The alias map is emitted to apiProxy.models, but a concrete retired model string is never resolved against that map before launch.
  • The AWF sidecar's internal BYOK sentinel path (COPILOT_DUMMY_BYOK) can cause authentication-style errors that obscure the actual model-availability root cause.

Root Cause

apiProxy.modelFallback treats the Copilot engine the same as BYOK providers: it rewrites unknown model names without knowing that Copilot deployment names are user-opaque. The sidecar does not distinguish between a gh-aw alias key (resolvable) and a concrete retired model string (should fail fast).

Proposed Solution

  1. In containers/api-proxy/, when engine=copilot and COPILOT_PROVIDER_BASE_URL is not set (i.e. standard Copilot, not BYOK), skip or disable modelFallback rewriting — the Copilot CLI is authoritative for its own model selection.
  2. Expose a apiProxy.modelFallback.excludeEngines config option (or honour modelFallback.enabled: false scoped to the Copilot engine) so gh-aw can suppress rewriting without patching the lock file.
  3. Improve error messaging: when the Copilot engine's pre-flight returns a model-unavailable code, emit a model-specific error rather than a generic provider-auth failure.

Generated by Firewall Issue Dispatcher · sonnet46 2.5M ·

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions