Conversation
The HTTP transport applied the chunk timeout to the raw response byte stream before framing. Framing drops SSE comment keepalives and other empty events, so any such byte reset the stall timer and a provider could hold a stalled generation warm forever (anomalyco#43519). This matched the native path but not the AI SDK path, whose wrapSSE resets only on parsed events. Bound the timeout on framed progress instead. A stream that only sends keepalives now still trips the chunk timeout, while real events keep resetting it.
Contributor
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
Contributor
|
The following comment was made by an LLM, it may be inaccurate: |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related: #43618 (merged, AI SDK path) and #51879 (open, v1 + SDK path). This fixes the remaining native @opencode/ai HTTP transport, which still measured the stall on raw bytes after #49229.
Issue for this PR
Closes #43519
Type of change
What does this PR do?
chunkTimeoutwas applied to the raw response byte stream before framing inpackages/ai/src/route/transport/http.ts.Framing.ssedrops SSE comments(
: keepalive), empty events anddata: null, but the stall timer reset on anybyte, so a provider or proxy that keeps a stalled generation warm with comment
heartbeats never tripped the timeout (#43519).
This moves the timeout onto the framed stream so it resets on emitted events
only. A keepalive produces no frame and no longer counts as progress, so the
request fails with the existing
Transport{ operation: "read", code: "Timeout" }error and follows the retry policy. The AI SDK path already workedthis way (
wrapSSEinpackages/core/src/aisdk.tsresets only on parsedevents); this makes the native path match it.
Why it works:
Stream.timeoutOrElsearms its deadline per pull, and framingconsumes and drops non-event bytes within a single pull without emitting an
empty chunk, so dropped bytes cannot reset the deadline.
How did you verify your code works?
ignores SSE comment keepalives when bounding a stalled streamtopackages/ai/test/http-timeout.test.ts: a server sends one content chunk,then a
: keepalivecomment 10s later; withchunkTimeout: 20_000therequest must fail with the read timeout. Against the current
v2http.tsthe test times out instead of failing, so it is a real regression test.
bun typecheckinpackages/ai: passes.bun testinpackages/ai: 1730 pass, 28 skip, 0 fail.Screenshots / recordings
N/A (no UI change).
Checklist