Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,21 @@ All notable changes to this project are documented here. The format follows [Kee

## [Unreleased]

## [0.2.0] - 2026-09-25

### Added
- `await router.aroute(task, limits=None)`: async version of `route()` for asyncio apps (#18).
- `Router(timeout=60)`: configurable HTTP timeout in seconds for catalog downloads and routing calls (#13).

### Fixed
- `request_json` now raises `RouterError` (with the original exception chained) for non-JSON responses, read timeouts and connection errors, so `except RouterError:` fallbacks work as documented.
- `request_json` now raises `RouterError` (with the original exception chained) for non-JSON responses, read timeouts and connection errors, so `except RouterError:` fallbacks work as documented (#5).
- A JSON response that isn't an object (e.g. `null` or a list), or a model list without `data`, now raises `RouterError` instead of `AttributeError`/`KeyError`.
- `Limits` rejects invalid values (`output_tokens < 1`, negative or NaN `max_cost_usd`, booleans) with `ValueError` (#9).
- `Router` rejects a non-positive or non-numeric `timeout` with `ValueError`.
- Duplicate model ids or providers are removed, keeping first-seen order, so Jev never sees the same model twice (#8).

### Internal
- The examples run offline in CI with mocked network calls (#20).

## [0.1.0] - 2026-09-23

Expand All @@ -18,5 +31,6 @@ All notable changes to this project are documented here. The format follows [Kee
- `router.api_key_for(model_id)` returns your key for the chosen model's provider.
- Examples for basic routing and agents (plan-and-execute, multi-turn chat, fallback).

[Unreleased]: https://github.com/TheCoder30ec4/model_router_python/compare/v0.1.0...HEAD
[Unreleased]: https://github.com/TheCoder30ec4/model_router_python/compare/v0.2.0...HEAD
[0.2.0]: https://github.com/TheCoder30ec4/model_router_python/compare/v0.1.0...v0.2.0
[0.1.0]: https://github.com/TheCoder30ec4/model_router_python/releases/tag/v0.1.0
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,10 +174,11 @@ These are real outputs from `examples/basic.py`.

| Call | What it does |
|---|---|
| `Router(*, jev_api_key=None, openrouter_api_key=None, providers=None, models=None, limits=Limits(), models_per_provider=None, timeout=60)` | All arguments are keyword-only. Loads live prices, context sizes and output limits (cached for 24h, no key needed). `timeout` is the HTTP timeout in seconds for catalog downloads and Jev/OpenRouter routing calls (e.g. `timeout=5`); it is passed to `urllib.request.urlopen`, not a total routing deadline. Raises `UnknownModelError` for unknown model ids or providers. |
| `Router(*, jev_api_key=None, openrouter_api_key=None, providers=None, models=None, limits=Limits(), models_per_provider=None, timeout=60)` | All arguments are keyword-only. Loads live prices, context sizes and output limits (cached for 24h, no key needed). `timeout` is the HTTP timeout in seconds for catalog downloads and Jev/OpenRouter routing calls (e.g. `timeout=5`); it is passed to `urllib.request.urlopen`, not a total routing deadline, and must be a finite number > 0 (`ValueError` otherwise). Raises `UnknownModelError` for unknown model ids or providers. |
| `refresh_catalog()` | Clears the cached model list, so the next `Router` downloads fresh prices. |
| `router.api_key_for(model_id) -> str \| None` | The key you passed in `providers={...}` for this model's provider. |
| `router.route(task, limits=None) -> str` | Returns the best model id for `task`. |
| `await router.aroute(task, limits=None) -> str` | Async version of `route()` for asyncio apps. Runs `route()` in a worker thread, so the event loop isn't blocked; same results and errors. |
| `router.fitting(task, limits=None) -> list[ModelInfo]` | Returns the models that pass the limits, without calling Jev (free). |
| `Limits(output_tokens=1024, max_cost_usd=None)` | The output size you expect and an optional cost cap for each call. |

Expand All @@ -188,7 +189,7 @@ Booleans and NaN are rejected. Invalid limits raise `ValueError` with the field
Routing errors (all subclasses of `RouterError`):
- `NoModelFitsError`: no model passes the limits. The message gives the reason for each model.
- `UnknownModelError`: a model id isn't on OpenRouter.
- `RouterError`: no routing key, a network or HTTP failure, or an error returned by Jev (e.g. a rate limit).
- `RouterError`: no routing key, a network or HTTP failure, a timeout, a response that isn't valid JSON, or an error returned by Jev (e.g. a rate limit).

### Using it in an agent

Expand Down
3 changes: 2 additions & 1 deletion docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -616,8 +616,9 @@ <h2 class="sec">API reference</h2>
<table>
<thead><tr><th>Call</th><th>What it does</th></tr></thead>
<tbody>
<tr><td><code>Router(*, jev_api_key=None, openrouter_api_key=None, providers=None, models=None, limits=Limits(), models_per_provider=None)</code></td><td>Keyword-only. Loads live model data (cached for 24h) and works out the candidate list. Raises <code>UnknownModelError</code> for unknown model ids or providers.</td></tr>
<tr><td><code>Router(*, jev_api_key=None, openrouter_api_key=None, providers=None, models=None, limits=Limits(), models_per_provider=None, timeout=60)</code></td><td>Keyword-only. Loads live model data (cached for 24h) and works out the candidate list. <code>timeout</code> is the HTTP timeout in seconds for each request. Raises <code>UnknownModelError</code> for unknown model ids or providers.</td></tr>
<tr><td><code>router.route(task, limits=None) → str</code></td><td>The best model id for <code>task</code>.</td></tr>
<tr><td><code>await router.aroute(task, limits=None) → str</code></td><td>Async <code>route()</code> for asyncio apps; doesn't block the event loop.</td></tr>
<tr><td><code>router.fitting(task, limits=None) → list[ModelInfo]</code></td><td>The models that pass the limits, without calling Jev (free).</td></tr>
<tr><td><code>router.api_key_for(model_id) → str | None</code></td><td>The key you passed in <code>providers={...}</code> for that model's provider.</td></tr>
<tr><td><code>Limits(output_tokens=1024, max_cost_usd=None)</code></td><td>The output size you expect and an optional cost cap for each call.</td></tr>
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "model-router-python"
version = "0.1.0"
version = "0.2.0"
description = "Pick the right LLM for every task: filter by context, output and cost limits, then let Jev choose the most efficient model."
readme = "README.md"
requires-python = ">=3.12"
Expand Down
5 changes: 4 additions & 1 deletion src/model_router/_http.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def request_json(url, api_key=None, body=None, timeout=60):
req = urllib.request.Request(url, data=data, headers=headers)
try:
with urllib.request.urlopen(req, timeout=timeout) as r:
return json.load(r)
res = json.load(r)
except urllib.error.HTTPError as e:
raise RouterError(f"{url} -> HTTP {e.code}: {e.read().decode(errors='replace')[:500]}") from e
except urllib.error.URLError as e:
Expand All @@ -25,3 +25,6 @@ def request_json(url, api_key=None, body=None, timeout=60):
raise RouterError(f"{url} -> {e}") from e
except (json.JSONDecodeError, UnicodeDecodeError) as e:
raise RouterError(f"{url} -> response was not valid JSON: {e}") from e
if not isinstance(res, dict):
raise RouterError(f"{url} -> expected a JSON object, got {type(res).__name__}")
return res
6 changes: 5 additions & 1 deletion src/model_router/catalog.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import time

from ._http import request_json
from .errors import RouterError
from .models import ModelInfo

MODELS_URL = "https://openrouter.ai/api/v1/models"
Expand All @@ -17,9 +18,12 @@ def fetch_catalog(max_age=CATALOG_TTL_SECONDS, *, timeout=60):
"""
# ponytail: in-process cache only; persist to disk if many short-lived processes each pay the fetch
if _cache["data"] is None or time.time() - _cache["at"] > max_age:
data = request_json(MODELS_URL, timeout=timeout).get("data")
if not isinstance(data, list):
raise RouterError(f"{MODELS_URL} -> response has no 'data' list")
_cache["data"] = {
m["id"]: ModelInfo.from_openrouter(m)
for m in request_json(MODELS_URL, timeout=timeout)["data"]
for m in data
if "text" in ((m.get("architecture") or {}).get("output_modalities") or ["text"])
}
_cache["at"] = time.time()
Expand Down
6 changes: 6 additions & 0 deletions src/model_router/router.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,12 @@ def __init__(
uses every current model from those providers (or the newest `models_per_provider`)
models exact OpenRouter-style ids, e.g. ["anthropic/claude-opus-5.5"]; overrides the auto-pick
"""
if (
isinstance(timeout, bool)
or not isinstance(timeout, (int, float))
or not 0 < timeout < float("inf")
):
raise ValueError("timeout must be a finite number of seconds > 0")
if jev_api_key:
self._choose = partial(jev.choose_via_jev, jev_api_key, timeout=timeout)
elif openrouter_api_key:
Expand Down
8 changes: 7 additions & 1 deletion tests/test_catalog.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import unittest
from unittest.mock import patch

from model_router import ModelInfo, catalog
from model_router import ModelInfo, RouterError, catalog


def raw(id, output=("text",), **extra):
Expand Down Expand Up @@ -49,6 +49,12 @@ def test_custom_max_age_zero_always_refetches(self, req):
catalog.fetch_catalog(max_age=0)
self.assertEqual(req.call_count, 2)

def test_response_without_data_list_is_router_error(self):
for res in ({}, {"error": {"message": "down"}}, {"data": None}):
with self.subTest(res=res), patch("model_router.catalog.request_json", return_value=res):
with self.assertRaises(RouterError):
catalog.fetch_catalog()


def info(id, created=0, expires=None):
return ModelInfo(id, 1000, None, 0.0, 0.0, "", created=created, expires=expires)
Expand Down
9 changes: 9 additions & 0 deletions tests/test_http.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,15 @@ def test_connection_reset_becomes_router_error(self):
request_json("https://x.test/a")
self.assertIsInstance(ctx.exception.__cause__, ConnectionResetError)

def test_json_that_is_not_an_object_becomes_router_error(self):
for body in (b"null", b"[1, 2]", b'"ok"'):
with self.subTest(body=body):
resp = MagicMock()
resp.__enter__.return_value = io.BytesIO(body)
with patch("urllib.request.urlopen", return_value=resp):
with self.assertRaises(RouterError):
request_json("https://x.test/a")


if __name__ == "__main__":
unittest.main()
5 changes: 5 additions & 0 deletions tests/test_router.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,11 @@ def make_router(models=("cheap/small", "big/smart"), limits=None, **kw):


class ConfigTest(unittest.TestCase):
def test_invalid_timeout_is_rejected(self):
for bad in (0, -1, None, "5", True, float("nan"), float("inf")):
with self.subTest(timeout=bad), self.assertRaises(ValueError):
make_router(timeout=bad)

def test_timeout_reaches_both_routing_backends(self):
for backend, payload in [
("jev_api_key", {"code": 0, "data": {"decision": "cheap/small"}}),
Expand Down
2 changes: 1 addition & 1 deletion uv.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading