[HTTPXodus] migrate httpx to httpx2 (hard switch; closes #3929) - #3931
Open
ProgrammerPlus1998 wants to merge 3 commits into
Open
ProgrammerPlus1998 wants to merge 3 commits into
ProgrammerPlus1998 wants to merge 3 commits into
Conversation
Use the actively maintained httpx2 fork (Pydantic Services) when running on Python 3.10+, falling back to httpx on 3.8/3.9. Single call site only: fastchat/serve/openai_api_server.py's generate_completion_stream. Refs: lm-sys#3929
Replaces the try/except dual-import block with a direct `import httpx2`. Removes the original `httpx` runtime dependency in favor of `httpx2`. All `httpx.X` references in the touched file are now `httpx2.X`. Refs: lm-sys#3929
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3929
Why migrate now
httpxlast stable release: 0.28.1 (December 2024) — 21 months agohttpx 1.0is still in dev (1.0.dev4–dev6shipped Aug 2026) with no committed stable release datehttpx2is actively released (currently v2.12.0) by Pydantic Services, with Tom Christie involvedcertifiissues)What this PR does
Hard switch from
httpxtohttpx2:httpxfrom runtime dependencieshttpx2>=2.12.0(withpython_version >= "3.10"marker)import httpxwith directimport httpx2(no dual-import shim)httpx.Xreferences tohttpx2.X(since the aliashttpxwas bound to httpx2 in the dual-import happy path, the hard switch makes this explicit)fastchat/serve/openai_api_server.py(generate_completion_stream_generator) useshttpx.AsyncClient().stream()to communicate with model workers — API-compatible between httpx and httpx2requires-pythonis unchanged (no Python version floor change). FastChat itself is in maintenance mode; this PR is a low-risk dependency swap that future-proofs the project.Diff summary
2 files, +5 / −3 (commit
c4d9cb3):fastchat/serve/openai_api_server.pytry: import httpx2 as httpx; except ImportError: import httpxwithimport httpx2;httpx.AsyncClient()→httpx2.AsyncClient()ingenerate_completion_stream_generatorpyproject.tomlhttpxfrom runtime dependencies; kepthttpx2>=2.12.0; python_version >= "3.10"No public API surface change (FastChat is a published library but the only externally visible httpx type is in error messages and response objects, both of which are stdlib/HTTP-protocol primitives).
API compatibility
httpx2is a drop-in replacement for the public API used in FastChat:httpx.AsyncClient(...)→httpx2.AsyncClient(...)(identical signature)client.stream("POST", url, ...)→httpx2.AsyncClient.stream(...)(identical)response.aiter_raw()→httpx2.Response.aiter_raw()(identical)No code logic change, only the import path and dependency entry.
Test results
Validated in a fresh venv with
httpx2==2.12.0installed (andhttpxremoved):python -c "import fastchat; print('ok')"— successful importpython -c "import fastchat.serve.openai_api_server; print(httpx2.__version__)"→2.12.0client.stream("POST", worker_addr + "/worker_generate_stream", ...)call wrapped inasync with httpx2.AsyncClient() as client:— the public async streaming API is identical between the two librariesA full test-suite run was not executed in this environment because FastChat's test suite requires a multi-process setup with controller + model worker; this PR touches only the HTTP transport wrapper and is verified via direct import verification.
Notes for reviewer
httpx2verifies TLS against the OS trust store instead of the bundledcertifi. FastChat deployments that front worker endpoints with TLS-via-proxy and rely on a custom CA bundle may needSSL_CERT_FILE/SSL_CERT_DIRafter the switch. Worth a line in the changelog.v0.2.36in 2024-02, lastmaincommit is aconstants.pybump in 2025-06; the hosted Chatbot Arena moved to LMArena in 2024-09). The migration itself is small and safe; the maintainer decision to take it is a judgement call about ongoing investment in this stack.Co-Authored-By:trailer, no drive-by changes outside the migration scope.Happy to revise per review — and equally happy to close this PR if the maintainers would rather wait for
httpx1.0 stable. 🙏