Repository navigation
fix(responses): consume hosted-tool streams before billing metadata - #560
Merged
Merged
Conversation
Live Copilot and Web IQ probes exposed a continuation deadlock: billing metadata resolves only after its event stream is consumed. Read each continuation stream first, then accumulate its billed usage. Cover all tool declaration carriers with deferred metadata across three turns.
Use one stream-consumption boundary for every hosted-tool turn. Read billing metadata after iterator completion or cancellation, retaining observed usage when a stream fails. Verify failed first and continuation turns preserve both the stream error and accumulated cost.
3 tasks done
Menci
marked this pull request as ready for review
September 30, 2026 05:56
yyyr-p
added a commit
to yyyr-p/Floway
that referenced
this pull request
Oct 4, 2026
Integrates upstream commits e286553..c7e4d78: - refactor(responses): lower agent_message in a request middleware (Menci#576) - fix(claude-code): update subscription request defaults (Menci#571) - fix(compaction): use the context compaction shim by default (Menci#572) - fix(codex): derive quota expiry from exhausted windows (Menci#570) - fix(tls): support IP-literal certificate identities (Menci#569) - fix(web-search): resolve the OpenAI search passthrough lazily (Menci#568) - fix(provider-codex): update stable client identity for newer models (Menci#567) - fix(provider-codex): preserve Responses Lite wire semantics (Menci#543) - feat(web): adopt the blue Floway logo (Menci#563) - feat(codex): show and redeem ChatGPT reset cards (Menci#528) - fix(responses): answer Codex WebSocket prewarms locally (Menci#552) - fix(responses): handle positional tool declarations in middleware (Menci#558) - fix(gateway): preserve first-token timing across tool continuations (Menci#561) - fix(responses): consume hosted-tool streams before billing metadata (Menci#560) - fix(codex): restore output omitted from terminal snapshots (Menci#551) - fix(codex): preserve model catalog context windows (Menci#559) - fix(web): disable inactive upstream disclosure (Menci#534) - fix(translate): isolate translated request payloads (Menci#557) - feat(models): support Claude Sonnet 5.5 (Menci#555) - feat(models): support GPT-6.1 Sol (Menci#553) - fix(web): allow imported credentials on upstream creation (Menci#556) - fix(gateway): preserve hosted tool selection across aliases and continuations (Menci#549) - fix(translate): restore Responses callable identities (Menci#547) - perf(translate): bound namespace tool name allocation work (Menci#548) - fix(translate): preserve namespace descriptions on child tools (Menci#545) - feat(requests): show each request's time to first token (Menci#550) - fix(translate): reject ambiguous flat callable kinds (Menci#542) - fix(gateway): preserve client tool identity in hosted dispatch (Menci#538) - fix(web): mask credential headers in request record exports (Menci#539) Conflict resolutions: - upstreams/access-control.tsx: keep the fork's parameterized title, description, and error props (title/description ?? defaults, plus defaultOpen/revealOn) alongside upstream Menci#534's disclosureDisabled={!override}. - upstreams/access-control_test.tsx: union the Control helper to accept both error and initialOverride, and keep both Menci#534's disabled-disclosure case and the fork's validation-error cases. - control-plane/routes.ts: union the schemas.ts named imports -- the fork's OAuth2 admin bodies plus upstream Menci#528's codexRateLimitResetCreditsBody and codexRateLimitResetConsumeBody; both sides' route blocks merged cleanly. - requests/ttft_test.tsx (Menci#550): pass the fork's required exchangeStreams={[]} to the RequestDetailPanel cases that construct it with no collected streams. Verified with pnpm run verify (typegen, lint, typecheck, 611 test files / 6644 tests, test:installers, check:agents-md, check:generated-assets, check:verify-parity, build:web).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hosted Responses tools make internal model continuation calls. Waiting for a continuation's billing metadata before reading its event stream prevents the stream from producing that metadata, so a completed web search can leave the final answer waiting indefinitely.
Consume each model turn before awaiting its deferred metadata through one shared generator boundary. Collect observed billing in the finalizer so successful, failed, and cancelled streams retain the costs already reported by upstream. Regression tests cover usage across two searches and an answer, plus failures in the first and later turns.
Test Plan
gpt-6-lunaand Microsoft Web IQ configuration copied into local SQLite.