Skip to content

feat: Automatic retries with backoff and Retry-After handling #8

Description

@yukik8

Is your feature request related to a problem?

The Jev API documents rate limits (1,200 requests/min, 250k tokens/sec) and returns HTTP 429 when they are exceeded; transient 5xx responses and network timeouts also happen in production. Today JevClient.evaluate() / evaluateAsync() treat any non-200 response as terminal — a single 429 or a momentary blip immediately surfaces as JevApiException with no attempt to recover. That makes the SDK fragile under exactly the high-volume, repeated-decision workloads Jev is built for.

Proposed Solution / Feature

Add lightweight automatic retries:

  • Retry on HTTP 408 / 429 / 5xx, connection failures, and timeouts.
  • Exponential backoff (e.g. 500ms → 1s → 2s) with small jitter and a cap, and a small default retry count (e.g. 2).
  • Honor a server-supplied Retry-After (and retry-after-ms) header when present, up to a sane maximum.
  • Apply to both evaluate() and evaluateAsync().

The configuration surface should stay minimal — a sensible default that just works, with a simple opt-out. No need for the full policy-object knobs that larger SDKs expose.

Example API Usage

// Default: retries happen automatically, nothing to configure.
JevClient jev = JevClient.create(apiKey);

// Opt out if the caller wants to handle retries themselves:
JevClient jev = JevClient.builder()
    .apiKey(apiKey)
    .maxRetries(0)
    .build();

Additional Context

A cheap companion improvement while touching the error path: include the API request id (x-typesafe-request-id) and the body's error / message field in JevApiException's message, which makes failures much easier to debug and report.

I'm happy to follow up with a PR containing a minimal implementation and tests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions