fix(provider): bound a stalled provider stream body with a configurable idle deadline - #4531
Conversation
|
Warning Review limit reachedNext included review available in 42 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (8)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
📦 Client bundle boundary
A server module in a client graph aborts hydration in the browser. New leaks fail CI; known leaks are tracked in |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1287706c60
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
…t of the log Two P1 findings from the Codex review on #4531. `loadEnv` copies project `.env` entries into the real process environment, so the `getHostEnv` default this resolver used handed back a project-controlled value -- exactly what the doc comment claimed it prevented. A repository could ship `VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS=0` and disable the host's safety bound, restoring the unbounded stalled stream this PR exists to prevent. Default to `getHostEnvExcludingEnvFile`, which consults the provenance `loadEnv` records. The malformed-override warning serialized the rejected value. `.env` expansion substitutes host process values into an entry, so `VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS=$DATABASE_PASSWORD` reached that branch and wrote the credential to the log on every stream request. The warning now names the key and the accepted range only. The logger's own credential scrubber is not a defence here: it matches `sk-`, `ghp_`, `xoxb-` and JWT shapes, while expansion can pull in any host variable. Both cases are covered by tests confirmed to fail without the fixes.
Independent review: 🟡 minor fixes firstRisk if merged as-is: medium | Reviewer confidence: high Look at this firstRead src/agent/runtime/chat-stream-handler.ts:89-90 and :1214/:1223 on origin/main before reading anything in the diff. They declare STREAM_START_IDLE_MS=60_000 and STREAM_OUTPUT_IDLE_MS=15_000 and apply them with no VF_STREAM_LIFECYCLE_MODE gate, which means Blocking
Non-blocking
Where the reviewers disagreedFour real conflicts; I adjudicated each against origin/main rather than averaging.
Provenance: independent automated review (Claude Code) via @kojiwakayama's token. Tooling output, not a human approval. Three reviewers examined this diff through separate lenses (correctness, security/authz, contract & test quality); a synthesis pass adjudicated their disagreements against |
…e env knob end to end Four blocking findings from an independent audit of #4531. All four are evidence defects, not behaviour defects: the code is unchanged apart from comments. The scope claim was false. This branch said `veryfront dev` chat and hosted runs had no bound, and the design note explained away the 60s/15s figures as belonging to `streaming/lifecycle/policy.ts`, which is mode-gated. It omitted that `chat-stream-handler.ts:89-90` declares its own copy of those two numbers and applies them at `:1214`/`:1223` with no `VF_STREAM_LIFECYCLE_MODE` condition, so every streaming chat turn has always been bounded at 15-60s. The issue triage comment had this right and the branch overruled it. The genuinely unwatched caller is the non-streaming drain -- `agent.generate` through `buildGenerateResultFromStream`, which `veryfront-cloud` routes every gateway model into. Say that instead, in the note and in the CHANGELOG, where the false version was about to ship as release-note text. Drop the `DEFAULT_PROVIDER_STREAM_IDLE_TIMEOUT_MS === DEFAULT_CHAT_STREAM_IDLE_TIMEOUT_MS` assertion from the guard test. This partially walks back the previous commit's decision to pin all three relations, so: that decision's purpose was to stop the note drifting into an untrue claim, and it is better served by removing the numeric claim from the note. The equality is two independent choices landing on the same round number, and the constant governs hosted chat runs through `createChatStreamWatchdog`, not the caller the note names. The `> 45s` and `< 300s` relations carry design content and stay pinned. Cover the environment knob through `requestStream`. Every existing env test called `resolveProviderStreamIdleTimeoutMs` directly with an injected reader, so replacing the resolver call inside `requestStream` with a plain default-or-normalize expression left all eight of them green while the knob -- the whole answer to the issue's "configurable" item -- stopped working for every caller the CHANGELOG names. Verified: with that substitution the old suite still reports `ok | 1 passed`; with the new test the run stops on it with `error: Promise resolution is still pending`. It mutates the host environment for the same reason the `.env`-provenance test does, and `lint:test-semantic-dispositions` still passes. Scope the keepalive rationale to its evidence. The `ping`-frame and 15s-gateway-keepalive argument covers Anthropic and the Veryfront Cloud gateway; `ext-llm-openai` and `ext-llm-google` call `requestStream` with no `idleTimeoutMs` and decode no heartbeat. The conclusion holds because `streamWithCleanup` disarms on raw bytes before `provider-sse.ts` parses anything, so any SSE traffic re-arms the deadline whether or not the extension decodes that event -- but say that, rather than implying a documented interval that those two transports do not publish. Also correct the call-site claim that "nothing retries automatically" on `retryable: true`. `ext-llm-anthropic/src/anthropic-provider.ts:99`/`:751` is a mid-turn replay loop keyed on exactly this flag. It cannot duplicate output because `yieldedThisAttempt` short-circuits its catch before the predicate is consulted; name that guard as the reason the flag is safe. Refs veryfront/veryfront-issue-inbox#1465
|
Pushed 1. The scope claim was false, and it was about to ship as release notes. Fixed.Verified on
The genuinely unwatched caller is the non-streaming drain: Corrected in the CHANGELOG entry, in the design note at 2. Dropped the equality assertion in the guard test.Removed Flagging this explicitly because it partially walks back commit 3. Added the missing end-to-end coverage for the environment knob.This was the sharpest finding. The knob is this PR's entire answer to the issue's "configurable" acceptance item, and nothing exercised it through Confirmed the gap by mutation. Replacing const idleTimeoutMs = resolveProviderStreamIdleTimeoutMs(options.idleTimeoutMs);with a plain New test "bounds a stalled body at the host environment override" sets the variable to 4. Scoped the keepalive rationale to its evidence.The I did not change the conclusion, because the mechanism holds independently: Also corrected (was non-blocking, but it was a false claim)The call-site note said Declined / deferred
Gates
One thing I did not touchGitHub reports this branch as |
|
CI on Full One process note worth someone's attention: the
|
Once response headers arrived, `requestStream` released its only timer and handed the body to `streamWithCleanup`, whose `pull()` awaited `reader.read()` with no deadline at all. A provider that went silent mid-response blocked its consumer until the consumer cancelled. Only `veryfront eval` was protected, by `--record-timeout`; `agent.generate` has no stream watchdog, and every Veryfront Cloud gateway model routes generate through `doStream`, so an unbounded drain of an unbounded body was the default path. Arm a deadline around each pending body read and disarm it as soon as bytes land, so it bounds provider silence rather than response length. On expiry take the same path as a failed read -- abort the request, error the stream, cancel the upstream reader -- so the connection is released and the caller sees a retryable `ProviderRequestError` naming the idle deadline. The 120s default sits above the gateway's 15s SSE keepalive and above the hosted child-fork and chat stream watchdogs, which know which turn stalled and should keep reporting first. `idleTimeoutMs` overrides it per request; `0` restores the old unbounded body. Refs veryfront/veryfront-issue-inbox#1465
… end to end Review follow-up on the body idle deadline. Three things were wrong with the first commit. The deadline was not configurable in any way a shipped caller could reach. `idleTimeoutMs` is a `requestStream` argument, and none of ext-llm-anthropic, ext-llm-openai or ext-llm-google forwards it, so `veryfront dev` chat, hosted agent runs and library use of `agent.generate` / `agent.stream` could not widen or disable the new 120s bound without patching veryfront. Resolve it instead through `VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS`, read per request from the host environment -- not `getEnv`, so a project `.env` cannot widen a bound the operator set. An explicit `idleTimeoutMs` still wins, and a malformed override warns and falls back to the default rather than failing every request. The design note above the default was wrong about the consumer watchdogs. The 60s/15s windows it cited are the strict lifecycle policy's, which only apply under `VF_STREAM_LIFECYCLE_MODE=shadow|active`; the legacy chat watchdog's idle window is `DEFAULT_CHAT_STREAM_IDLE_TIMEOUT_MS`, exactly 120s, and its tool-running window is 300s. Say what is true, explain why an equal or smaller window does not pre-empt those -- they count semantic chunks, this counts bytes on the wire, and a provider running a server-side tool keeps sending pings and gateway keepalives -- and pin all three relations in the guard test, which previously only asserted the 45s one it was not named for. The `agent.generate` regression test was vacuous: it hand-built a `ProviderRequestError` and asserted `generateText` rejected with that same object, exercising none of the change. Replace it with a test that drives the real `requestStream` over a stalled fetch body through the bridge's drain loop, with no deadline configured so the shipped 120s default is what fires, using `FakeTime` to keep it fast. Both new deadline tests hang when `armIdleDeadline` is neutered; neither mutates the host environment, which the unit-hermeticity audit forbids. Also record why the timeout keeps `retryable: true` after partial output, and what a future automatic retry loop must check before replaying. Refs veryfront/veryfront-issue-inbox#1465
…ing for undefined `streamWithCleanup` has exactly one caller, which always passes both `onFinish` and `idle`, so the optional markers described a caller that does not exist and left `armIdleDeadline` with a dead `idle === undefined` branch (CodeQL alert 409). Make both parameters required so the signature matches reality. Dropping only the `undefined` comparison, as the bot suggested, would have left `idle` optional and `idle.timeoutMs` unguarded.
…t of the log Two P1 findings from the Codex review on #4531. `loadEnv` copies project `.env` entries into the real process environment, so the `getHostEnv` default this resolver used handed back a project-controlled value -- exactly what the doc comment claimed it prevented. A repository could ship `VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS=0` and disable the host's safety bound, restoring the unbounded stalled stream this PR exists to prevent. Default to `getHostEnvExcludingEnvFile`, which consults the provenance `loadEnv` records. The malformed-override warning serialized the rejected value. `.env` expansion substitutes host process values into an entry, so `VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS=$DATABASE_PASSWORD` reached that branch and wrote the credential to the log on every stream request. The warning now names the key and the accepted range only. The logger's own credential scrubber is not a defence here: it matches `sk-`, `ghp_`, `xoxb-` and JWT shapes, while expansion can pull in any host variable. Both cases are covered by tests confirmed to fail without the fixes.
…e env knob end to end Four blocking findings from an independent audit of #4531. All four are evidence defects, not behaviour defects: the code is unchanged apart from comments. The scope claim was false. This branch said `veryfront dev` chat and hosted runs had no bound, and the design note explained away the 60s/15s figures as belonging to `streaming/lifecycle/policy.ts`, which is mode-gated. It omitted that `chat-stream-handler.ts:89-90` declares its own copy of those two numbers and applies them at `:1214`/`:1223` with no `VF_STREAM_LIFECYCLE_MODE` condition, so every streaming chat turn has always been bounded at 15-60s. The issue triage comment had this right and the branch overruled it. The genuinely unwatched caller is the non-streaming drain -- `agent.generate` through `buildGenerateResultFromStream`, which `veryfront-cloud` routes every gateway model into. Say that instead, in the note and in the CHANGELOG, where the false version was about to ship as release-note text. Drop the `DEFAULT_PROVIDER_STREAM_IDLE_TIMEOUT_MS === DEFAULT_CHAT_STREAM_IDLE_TIMEOUT_MS` assertion from the guard test. This partially walks back the previous commit's decision to pin all three relations, so: that decision's purpose was to stop the note drifting into an untrue claim, and it is better served by removing the numeric claim from the note. The equality is two independent choices landing on the same round number, and the constant governs hosted chat runs through `createChatStreamWatchdog`, not the caller the note names. The `> 45s` and `< 300s` relations carry design content and stay pinned. Cover the environment knob through `requestStream`. Every existing env test called `resolveProviderStreamIdleTimeoutMs` directly with an injected reader, so replacing the resolver call inside `requestStream` with a plain default-or-normalize expression left all eight of them green while the knob -- the whole answer to the issue's "configurable" item -- stopped working for every caller the CHANGELOG names. Verified: with that substitution the old suite still reports `ok | 1 passed`; with the new test the run stops on it with `error: Promise resolution is still pending`. It mutates the host environment for the same reason the `.env`-provenance test does, and `lint:test-semantic-dispositions` still passes. Scope the keepalive rationale to its evidence. The `ping`-frame and 15s-gateway-keepalive argument covers Anthropic and the Veryfront Cloud gateway; `ext-llm-openai` and `ext-llm-google` call `requestStream` with no `idleTimeoutMs` and decode no heartbeat. The conclusion holds because `streamWithCleanup` disarms on raw bytes before `provider-sse.ts` parses anything, so any SSE traffic re-arms the deadline whether or not the extension decodes that event -- but say that, rather than implying a documented interval that those two transports do not publish. Also correct the call-site claim that "nothing retries automatically" on `retryable: true`. `ext-llm-anthropic/src/anthropic-provider.ts:99`/`:751` is a mid-turn replay loop keyed on exactly this flag. It cannot duplicate output because `yieldedThisAttempt` short-circuits its catch before the predicate is consulted; name that guard as the reason the flag is safe. Refs veryfront/veryfront-issue-inbox#1465
…ng tests The JSDoc on resolveProviderStreamIdleTimeoutMs said "the one test that does mutate it is the .env-provenance case". A second env-mutating test was added later in this branch (the requestStream environment-override case), so the count was stale. Refs veryfront/veryfront-issue-inbox#1465
VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS had no entry in the configuration guide. The only durable artifact was a generated api-reference row, which states neither the default, nor the 0-disables rule, nor the host-vs-.env rule. Mirrors the shape the guide already uses for VERYFRONT_FILE_CACHE_L1_*.
772b30f to
7c4d264
Compare
State of this PR for a reviewerAn independent audit rated this minor-fixes-first with four blocking items. Closed since the audit
Also fixed in this round
Deliberately deferredThe audit's remaining non-blocking items are the two logging-policy gaps, now The What I need from youDecision 1 — the resolver's logging policy. Pick one; I can implement
My read is (C) if you want it closed in this PR and (A) if you would rather Decision 2 — ratify the corrected scope. This PR reached its original Decision 3 — draft status and merge. Leaving this as a draft and not VerificationRun locally on the pushed head, against the rebased tree:
No test was weakened and no assertion was relaxed to pass anything. The one CI on I am therefore not claiming a fully green CI run — please confirm the Sonar last reported Quality Gate passed (97.7% coverage on new code, 0 |
|
Review score: 93/100\n\nReviewed head: \n\nThe provider stream body now has a bounded per-read idle deadline with explicit per-request and host-level configuration, aborts the upstream request on timeout, and preserves retryability/error context. The implementation correctly excludes project .env values from the host safety override and avoids logging malformed configured values. The latest source comments and tests also correct the earlier scope claim about existing chat watchdogs.\n\nValidation:\n- Focused provider HTTP and runtime bridge tests pass: 2 suites, 160 steps\n- CI, coverage, Sonar, CodeQL, lint, format, and typecheck pass\n- The two prior findings are fixed in source, but their review threads still need resolution\n- The PR is still draft, so the Automated review gate is waiting on readiness\n\nScore is 93/100. Resolve the addressed threads and promote to ready for review. |
Review score correction: 93/100Reviewed head: The provider stream body has a bounded per-read idle deadline with explicit per-request and host-level configuration, aborts the upstream request on timeout, and preserves retryability and error context. The implementation excludes project .env values from the host safety override and avoids logging malformed configured values. Validation:
The PR scores 93/100. Resolve the addressed threads and promote to ready for review. |
|
You have reached your Codex usage limits for security reviews. Please try again later. |
There was a problem hiding this comment.
Your trial has ended. Reactivate Greptile to resume code reviews.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
@codex review |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |



What was broken
Once response headers arrived, a provider response body had no deadline at all.
requestStreamdisposed its header deadline and handed the body tostreamWithCleanup, whosepull()awaitedreader.read()with nothing boundingit (
src/provider/runtime-loader/provider-http.ts:987before this change). Amodel stream that went silent mid-response blocked its caller until the caller
cancelled.
Scope, corrected. An earlier revision of this description claimed
veryfront devchat and hosted runs were unbounded. That was wrong, and itoverruled the issue triage comment that had it right. On
origin/main:src/agent/runtime/chat-stream-handler.ts:89-90declaresSTREAM_START_IDLE_MS = 60_000andSTREAM_OUTPUT_IDLE_MS = 15_000andapplies them at
:1214/:1223with noVF_STREAM_LIFECYCLE_MODEcondition.
src/agent/runtime/index.ts:3841passes no override, so everystreaming chat turn has always been bounded at 15-60s.
child-fork-execution-runner.ts:77bounds hosted child forks at 45s.createChatStreamWatchdog(
DEFAULT_CHAT_STREAM_IDLE_TIMEOUT_MS, and a 300s tool-running window).veryfront evalgained--record-timeoutin feat(eval): show live progress and add --record-timeout #4508.The genuinely unbounded path is the non-streaming drain:
agent.generategoesthrough
buildGenerateResultFromStreaminruntime-bridge.ts(drain at:968)with no timer at all, and
veryfront-cloud/provider.ts:51sets_generateViaStreamfor every gateway model. That caller, and libraryembedders on the same path, are what this change exists for. The safety margin
is therefore wider than the earlier description claimed, not narrower: a
stricter consumer bound fires first everywhere a consumer exists.
What this changes
src/provider/runtime-loader/provider-http.ts-streamWithCleanuparms adeadline around each pending body read and disarms it as soon as bytes land, so
it bounds provider silence rather than response length, and a consumer that
stops pulling is never timed out for its own backpressure. On expiry it takes
the same path as a failed read - abort the request, error the stream, cancel the
upstream reader - so the connection is released rather than left open, and the
caller sees a retryable
ProviderRequestErrornaming the deadline that fired.VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS, read from the host environment onevery stream request, falling back to a 120s default.
0disables it. Anexplicit
idleTimeoutMsargument still wins for custom provider extensions. Itis read via
getHostEnvExcludingEnvFile, notgetEnvor plaingetHostEnv, soa loaded project's
.envcannot widen or switch off a safety bound the hostoperator set; a malformed value warns and uses the default instead of failing
every provider request.
docs/guides/configuration.md- a "Provider stream idle deadline"section under "Environment variables", stating the default, the
0-disablesrule and the host-vs-
.envrule. The generated api-reference row states noneof those, so without this the knob had no durable user-facing documentation.
Regression tests, and confirmation they fail without the fix
src/runtime/runtime-bridge.test.ts- "bounds a stalled provider body reachedthrough generate": drives the real
requestStreamover a fetch whose bodydelivers one chunk and then goes quiet, through the buffering drain loop that
generateruns. Nothing configures the deadline, so the shipped 120s default iswhat fires.
FakeTimekeeps it instant.src/provider/runtime-loader/provider-http.test.ts- "bounds a stalled body atthe default deadline with no configuration" (same, at the
requestStreamlevel, and asserts the request is not aborted at 119_999ms), plus the
existing explicit-
idleTimeoutMscases for re-arming, disabling, aborting therequest, and rejecting invalid values before a request is issued.
src/provider/runtime-loader/provider-http.test.ts- "bounds a stalled body atthe host environment override", new in the latest revision: sets
VERYFRONT_PROVIDER_STREAM_IDLE_TIMEOUT_MS=5000in the real host environmentand drives
requestStreamwith no explicitidleTimeoutMs, asserting therequest is not aborted at 4_999ms and that the error names a 5000ms deadline.
This closes a real coverage gap: every other environment test called
resolveProviderStreamIdleTimeoutMsdirectly with an injected reader, soreplacing the resolver call inside
requestStreamwith a plaindefault-or-normalize expression left all of them green while the knob stopped
working for every caller this description names. Verified: with that
substitution applied, the previously existing suite still reports
ok | 1 passed, and with the new test present the run stops on it witherror: Promise resolution is still pending but the event loop has already resolved.Earlier, verified by neutering
armIdleDeadline()in the working tree andre-running: both files go from
ok | 1 passedtoEXIT 1, stopping on the newtest with the same pending-promise hang - i.e. exactly the hang this fixes.
Restored before committing.
Gates run
deno task test:file src/provider/runtime-loader/provider-http.test.tsdeno task test:file src/runtime/runtime-bridge.test.tssrc/provider/**/*.test.ts+ anthropic/openai/google provider testsdeno fmt --check docs/guides/configuration.mddeno check --no-lockon the changed.tsfilesdeno task lintdeno fmt --checkon the changed filesdeno task lint:test-typecheckdeno task lint:test-semantic-dispositionsdeno task lint:testing-front-doordeno task lint:client-bundledeno task lint:anti-slop,lint:style,lint:module-boundaries,lint:barrel-jsdoc,lint:check-awaits,lint:cwd-relative-test-reads,lint:skipped-tests,lint:dependency-boundariesdeno task docs+docs:api-reference:checklint:imports(86 findings),lint:ban-deep-imports(3) andlint:platform(119) fail, but fail identically on
origin/mainwith the same counts and noentries in the files this PR touches - pre-existing, not gating in the
lintchain.
lint:test-semantic-dispositionsinitially failed on an earlier revision ofthis branch because the configurability tests mutated the host environment with
setEnv/deleteEnv. Rather than growscripts/test/test-semantic-audit-migration.ts, the resolver now takes aninjectable
readEnvlookup that defaults to the real one. The two tests thatstill mutate the host environment - the
.env-provenance case and the newrequestStreamoverride case - are the two where the seam under test is thedefault reader, so injecting past it would assert on the mock; the audit passes
with both present.
Review findings addressed
agent.generateregression test was vacuous. Correct, and confirmed:it hand-built a
ProviderRequestError, errored a fake model stream with it,and asserted
generateTextrejected with that same object - zero lines of thechange. Replaced with the end-to-end test described above, which goes through
the real
requestStream, and which covers the default path rather than anexplicit short timeout.
revision of it was false in a new way. The original note claimed a margin
below the chat watchdog. The replacement fixed that but asserted the 60s/15s
figures "belong to
lifecycle/policy.ts, which only applies underVF_STREAM_LIFECYCLE_MODE=shadow|active" - which quietly omitted thatchat-stream-handler.tsdeclares its own ungated copy of those same twonumbers and applies them to every streaming chat turn. The note now states
the real layering (see "What was broken"), names
agent.generateas theunwatched caller, and drops the claim that the 120s equality with
DEFAULT_CHAT_STREAM_IDLE_TIMEOUT_MSis deliberate.grep -rn idleTimeoutMs extensions/confirms no shipped provider forwards it. Fixed with the environment knob
above, and the CHANGELOG migration advice rewritten to point at it. The new
requestStreamoverride test is what makes that claim checkable.DEFAULT_PROVIDER_STREAM_IDLE_TIMEOUT_MS === DEFAULT_CHAT_STREAM_IDLE_TIMEOUT_MSassertion. The equality is two independent choices landing on the same round
number, not a designed relation, and the constant it pinned governs hosted
chat runs through
createChatStreamWatchdograther than the caller the notenames. The
> 45sand< 300srelations do carry design content and staypinned. This is a partial walk-back of the previous revision's "pin all three"
decision; the purpose of that decision - the note must not be able to drift
into stating something untrue - is preserved by removing the numeric claim
from the note rather than by pinning it.
provider. The
ping-frame and 15s-gateway-keepalive evidence coversAnthropic and the Veryfront Cloud gateway.
ext-llm-openai(:1119,:1213)and
ext-llm-google(:618) callrequestStreamwith noidleTimeoutMsanddecode no heartbeat. The conclusion still holds -
streamWithCleanupdisarmson raw bytes before
provider-sse.tsparses anything, so any keepalive, SSEcomment line or progress event re-arms the deadline whether or not the
extension has a case for that event - but the note and the CHANGELOG now say
that for a directly-configured OpenAI or Google model the argument rests on
the transport emitting something rather than on a documented interval, and
point at the environment knob.
Review findings declined
tool_runningwindow forprovider-executed tools. The two deadlines do not measure the same thing: the
chat watchdog counts semantic chunks, this one counts bytes on the wire,
and it disarms before any SSE parsing. A provider running a server-side tool
(
web_search,web_fetch,code_execution, the MCP connector) holds the HTTPresponse open but is not silent on the wire. Making the provider deadline
phase-aware, or raising it above 300s, would also put it outside the 60-120s
the issue asks for. See finding 5 above for the scope limit now recorded.
setTimeout/clearTimeoutchurn (minor, perf). Measured at~1.15us per arm/disarm pair on this runtime (200k pairs in 230ms), so a
5,000-chunk SSE response spends under 6ms on timers across a response lasting
tens of seconds. A polling interval over a
lastChunkAttimestamp would tradethat for deadline precision and a timer outliving the read it guards. Declined
as a rewrite, but the measurement is now recorded in the comment so it is not
re-litigated.
retryable: trueon a post-partial-output timeout (minor). Kept, but thestated reason was wrong and is corrected. The earlier note said "nothing
retries automatically".
extensions/ext-llm-anthropic/src/anthropic-provider.tsdoes:
:99(isReplayableAnthropicStreamFailure) and the catch at:751area mid-turn replay loop keyed on exactly this flag. It cannot duplicate output
because
yieldedThisAttemptshort-circuits that catch before the predicate isconsulted, so a timeout that fired after the first chunk rethrows. The call-site
comment now names that guard as the reason the flag is safe, rather than
claiming no retry loop exists.
Known gap, filed separately rather than fixed here
Commit a534666 found that
getHostEnvreturns valuesloadEnvcopied out of aproject
.env, and fixed only its own new read.src/security/http/outbound-fetch.ts:246(
VERYFRONT_HOST_ALLOW_INTERNAL_EGRESS) and:295(
HOST_ALLOWED_INTERNAL_PROVIDER_ORIGINS) still read through plaingetHostEnvwhile
:400in the same file already usesgetHostEnvExcludingEnvFile. Thosetwo keys gate whether run-scoped inference credentials may leave over a
non-HTTPS, non-loopback origin. Different file, different subsystem, untouched by
this diff - so it is filed on its own rather than held here, as
veryfront/veryfront-issue-inbox#1623, with the reachability argument
(
src/utils/env-loader.tscopies every.envkey into the processenvironment, and the
if (existing && !override) continueguard does notprotect a gate that is off by default, because off-by-default means unset).
Deliberately not addressed here
Two logging-policy gaps in
resolveProviderStreamIdleTimeoutMs, left openbecause the sensible fixes trade against each other and against text already
in this CHANGELOG. They need a maintainer's call, not a patch:
dedupe, so one typo in a deployment's environment emits a warning per model
call for the life of the process.
loadEnvcopied out of a project.envisdiscarded silently - no log at any level. A
veryfront devuser whowidens the bound in
.envgets the 120s default with no signal that theirsetting was ignored.
Caching the resolved value at module scope would fix both at once, but it
contradicts the "read from the host environment on every stream request"
contract stated above and in the CHANGELOG, and the new
requestStreamoverride test depends on the per-call read. See the review comment on this PR
for the options.
Refs veryfront/veryfront-issue-inbox#1465