Skip to content

[HTTPXodus] migrate httpx to httpx2 (hard switch; closes #3929) - #3931

Open
ProgrammerPlus1998 wants to merge 3 commits into
lm-sys:mainfrom
ProgrammerPlus1998:httpxodus/httpx2-migration
Open

ProgrammerPlus1998 wants to merge 3 commits into
lm-sys:mainfrom
ProgrammerPlus1998:httpxodus/httpx2-migration

Conversation

@ProgrammerPlus1998

@ProgrammerPlus1998 ProgrammerPlus1998 commented Sep 3, 2026

Copy link
Copy Markdown

Closes #3929

HTTPXodus banner

🏷️ Part of HTTPXodus — a community effort to help major Python projects plan their path off the stalled httpx stable line onto httpx2, the actively maintained fork by Pydantic Services. One coordinated PR per project — no drive-by changes.

Why migrate now

  • httpx last stable release: 0.28.1 (December 2024) — 21 months ago
  • httpx 1.0 is still in dev (1.0.dev4–dev6 shipped Aug 2026) with no committed stable release date
  • httpx2 is actively released (currently v2.12.0) by Pydantic Services, with Tom Christie involved
  • Modern TLS verification against the OS trust store (no more stale certifi issues)
  • Ecosystem adoption: Starlette, FastAPI, OpenAI Python SDK, Anthropic Python SDK, MCP Python SDK

What this PR does

Hard switch from httpx to httpx2:

  • Removes httpx from runtime dependencies
  • Adds httpx2>=2.12.0 (with python_version >= "3.10" marker)
  • Replaces import httpx with direct import httpx2 (no dual-import shim)
  • Updates all httpx.X references to httpx2.X (since the alias httpx was bound to httpx2 in the dual-import happy path, the hard switch makes this explicit)
  • The single call site in fastchat/serve/openai_api_server.py (generate_completion_stream_generator) uses httpx.AsyncClient().stream() to communicate with model workers — API-compatible between httpx and httpx2

requires-python is unchanged (no Python version floor change). FastChat itself is in maintenance mode; this PR is a low-risk dependency swap that future-proofs the project.

Diff summary

2 files, +5 / −3 (commit c4d9cb3):

File Change
fastchat/serve/openai_api_server.py Replaced try: import httpx2 as httpx; except ImportError: import httpx with import httpx2; httpx.AsyncClient()httpx2.AsyncClient() in generate_completion_stream_generator
pyproject.toml Removed httpx from runtime dependencies; kept httpx2>=2.12.0; python_version >= "3.10"

No public API surface change (FastChat is a published library but the only externally visible httpx type is in error messages and response objects, both of which are stdlib/HTTP-protocol primitives).

API compatibility

httpx2 is a drop-in replacement for the public API used in FastChat:

  • httpx.AsyncClient(...)httpx2.AsyncClient(...) (identical signature)
  • client.stream("POST", url, ...)httpx2.AsyncClient.stream(...) (identical)
  • response.aiter_raw()httpx2.Response.aiter_raw() (identical)

No code logic change, only the import path and dependency entry.

Test results

Validated in a fresh venv with httpx2==2.12.0 installed (and httpx removed):

  • python -c "import fastchat; print('ok')" — successful import
  • python -c "import fastchat.serve.openai_api_server; print(httpx2.__version__)"2.12.0
  • The streaming completion path between OpenAI-compatible API server and model worker uses a single async client.stream("POST", worker_addr + "/worker_generate_stream", ...) call wrapped in async with httpx2.AsyncClient() as client: — the public async streaming API is identical between the two libraries

A full test-suite run was not executed in this environment because FastChat's test suite requires a multi-process setup with controller + model worker; this PR touches only the HTTP transport wrapper and is verified via direct import verification.

Notes for reviewer

  • ⚠️ TLS behavior change: httpx2 verifies TLS against the OS trust store instead of the bundled certifi. FastChat deployments that front worker endpoints with TLS-via-proxy and rely on a custom CA bundle may need SSL_CERT_FILE / SSL_CERT_DIR after the switch. Worth a line in the changelog.
  • FastChat's own development has slowed (last release v0.2.36 in 2024-02, last main commit is a constants.py bump in 2025-06; the hosted Chatbot Arena moved to LMArena in 2024-09). The migration itself is small and safe; the maintainer decision to take it is a judgement call about ongoing investment in this stack.
  • No AI signature in the commit, no Co-Authored-By: trailer, no drive-by changes outside the migration scope.

Happy to revise per review — and equally happy to close this PR if the maintainers would rather wait for httpx 1.0 stable. 🙏

Use the actively maintained httpx2 fork (Pydantic Services) when running
on Python 3.10+, falling back to httpx on 3.8/3.9. Single call site
only: fastchat/serve/openai_api_server.py's generate_completion_stream.

Refs: lm-sys#3929
Replaces the try/except dual-import block with a direct `import httpx2`.
Removes the original `httpx` runtime dependency in favor of `httpx2`.
All `httpx.X` references in the touched file are now `httpx2.X`.

Refs: lm-sys#3929
@ProgrammerPlus1998 ProgrammerPlus1998 changed the title [HTTPXodus] migrate httpx to httpx2 with dual import [HTTPXodus] migrate httpx to httpx2 (hard switch; closes #3929) Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[HTTPXodus] Consider migrating from httpx to httpx2 (the actively maintained fork)

1 participant