Skip to content

Retry with backoff on 429 and 5xx responses #12

Description

@TheCoder30ec4

Motivation

Jev and OpenRouter can rate-limit (HTTP 429) or fail briefly (502/503). Right now one hiccup fails route() straight away.

Proposal

  • Retry 429, 500, 502, 503 and 504 up to N times (default 2) with exponential backoff plus jitter
  • Honor the Retry-After header when present
  • Never retry other 4xx codes
  • Make it configurable: Router(..., max_retries=2)

Stdlib only (time.sleep). No new dependencies.

Where

src/model_router/_http.py, plus passing the setting through from Router.

Done when

  • Retries are tested with a mocked urlopen and a mocked sleep, so tests stay fast
  • README documents the default

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: httpNetwork calls, errors, timeouts and retriesenhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions