ai(claude): right-size ProForge response budgets and implement real end-to-end streaming

[qnbs]

Context

Execution owner for finding #2 from the 2026-09-12 /claude-api prompt-audit, under umbrella #704 and backed by prep/evidence PR #728.

Prep/evidence PR: #728 — docs(proforge): plan Claude max_tokens/timeout right-sizing

Plan document: docs/PROFORGE-CLAUDE-MAXTOKENS-CEILING-PLAN.md

Current main baseline at issue creation:

021b1bdc145f211cf4f4239db7ad26e44ca7fae2

The prep PR is planning evidence, not implementation authority. Re-fetch current source, current hosting constraints, current Anthropic API/model capabilities and #704's admitted model authority before implementing.

Related audit owners:

• ai(proforge): use native Claude structured outputs for schema-bound pipeline stages #729 + prep PR feat(proforge): plan Claude structured-outputs wiring #730 — native structured outputs;
• token-accounting sibling issue + prep PR docs(proforge): plan real token accounting for the Claude path #727 — usage telemetry needed to measure/tune this work;
• ai(lifecycle): unify Stop Generation, cancellation, supersession and duplicate-request semantics #714 — broader cancellation/supersession lifecycle; this issue owns only cancellation propagation required for correct Claude transport.

---

Audit finding

The Claude web/relay path and several ProForge stages are bounded by values and transport semantics that no longer match the current admitted Claude generation.

The audit identified:

api/_shared/claudeProxyCore.ts
  MAX_TOKENS_CEILING = 8192
  OUTBOUND_TIMEOUT_MS = 20_000
  one blocking upstream fetch
  full upstream body consumed before returning

services/aiProviderService.ts
  Claude delivery reads the final JSON response
  and surfaces the whole text as effectively one onChunk event

services/proForge/pipelineAgents/baseAgent.ts
  cfg.maxTokens ?? 8192

services/proForge/pipelineAgents/publishingAgent.ts
  Math.min(config.maxTokens, 4000)

The audit classified this as an API/architecture fossil: the app's current Claude catalogue targets much larger-capability models, while the web path remains optimized for small blocking completions.

The audit proposed a mechanical candidate of 32_000 max tokens, 55_000 ms timeout and a 16_000 publishing cap, but those numbers are not acceptance criteria. They must be requalified against the current provider, hosting/runtime constraints, workload evidence and real token telemetry.

---

Why this matters

Large ProForge artifacts can include:

structural plans with many edits
large prose/copy-edit batches
quality reports
publishing packages
other schema-bound long-form artifacts

Current behavior creates several risks:

1. application-level ceilings can truncate otherwise supported output;
2. a 20-second outbound timeout can terminate valid high-latency responses;
3. adaptive/reasoning behavior can consume meaningful latency before visible output;
4. the public streaming abstraction can imply incremental Claude delivery when the web Claude path is effectively buffered;
5. raising token ceilings without changing transport can increase timeout/memory pressure;
6. without real token accounting (docs(proforge): plan real token accounting for the Claude path #727 sibling), capacity tuning is based on guesses rather than observed provider usage.

---

Phase A — requalify and right-size the bounded non-streaming path

1. Inventory all output budgets and timeouts

Search current source for every independent value affecting Claude/ProForge output, including:

maxTokens
max_tokens
MAX_TOKENS_CEILING
OUTBOUND_TIMEOUT_MS
feature-specific Math.min(...) caps
InferenceGateway defaults
provider defaults
ProForge stage overrides
relay/host function limits

Classify every value as one of:

PROVIDER_HARD_LIMIT
MODEL_CAPABILITY
WORLDscript_ABUSE_SAFETY_LIMIT
HOSTING_RUNTIME_LIMIT
FEATURE_OUTPUT_BUDGET
UX_LATENCY_POLICY
HISTORICAL_ACCIDENTAL_CEILING

No magic number should remain unexplained.

2. Re-fetch provider capability

At implementation time, use #704's canonical model authority plus current official Anthropic docs.

Do not encode a permanent global 128K assumption merely because current high-end Claude models can support that scale.

The effective request allowance should eventually be bounded by the admitted model's current capability.

3. Re-fetch hosting/runtime limits

WorldScript's Claude web relay is deployed through supported hosted paths. Verify actual current execution/streaming/time limits for each supported host that serves this relay.

Do not choose a timeout that is impossible on one supported host.

Do not make GitHub Pages appear to support the Claude relay; it does not host arbitrary serverless relay behavior.

4. Choose a conservative interim budget

Before real streaming lands, choose a bounded interim ceiling/timeout that improves the known 4K/8K/20s mismatch without pretending the blocking path can safely consume the provider maximum.

Document evidence for the chosen values.

5. Publishing workload

Requalify the separate publishing cap from the actual publishingPackageSchema, representative output size and provider behavior.

Do not mechanically raise 4000 → 16000 if evidence supports a different bounded value.

---

Phase B — implement genuine Claude streaming end-to-end

Required invariant

Anthropic SSE event arrives
        ↓
relay forwards it without full-response buffering
        ↓
client parses incrementally
        ↓
onChunk receives real partial text
        ↓
completion and usage settle exactly once

A function that calls onChunk(fullFinalText) once after a complete JSON response is not streaming.

---

1. Explicit streaming request contract

Re-check current Anthropic Messages streaming API at implementation time.

The request mode must be explicit and testable.

Do not silently change ordinary non-stream requests if some call sites still intentionally require a single final response.

2. Relay streaming

Rework the platform-neutral Claude relay core and host adapters so a successful streaming Anthropic response is returned as a streaming Response/body and is not first consumed via .text()/.json().

Preserve all existing relay security/admission properties:

same-origin gate
model allowlist
request-body limit
message-count/content limits
rate limiting
API-key confidentiality
no prompt/response logging
cache-control: no-store
bounded error responses

Do not weaken ADR-0016/stateless privacy guarantees.

3. Client SSE parser

Implement a robust Claude client reader in services/aiProviderService.ts or the appropriate current provider adapter.

Handle at least:

arbitrary byte/chunk boundaries
partial SSE frames
multiple frames in one read
CRLF/LF normalization where relevant
text delta events
message start/stop
provider error events
usage events
stream close without expected terminal framing
malformed event payloads

Do not parse streaming JSON with a greedy brace regex.

Use current provider event semantics.

4. Incremental delivery contract

AIStreamCallbacks.onChunk must receive actual incremental text for Claude web requests.

onDone must settle once.

onError must not fire after a normal user cancellation unless the established API explicitly requires it.

No duplicate final text should be delivered after already-streamed deltas.

5. Cancellation propagation

Abort must propagate through:

feature caller
→ gateway/service
→ browser fetch
→ serverless relay request lifecycle
→ Anthropic upstream request / stream reader

Cancel should stop resource/network work and release readers/controllers.

Do not expand into #714's whole lifecycle state machine; integrate with it rather than duplicating it.

6. Backpressure / memory

The relay must not concatenate an unbounded long response merely to return it later.

The client may accumulate final text where existing consumers require it, but server-side buffering must not remain the architectural requirement for a streamed request.

7. Direct/native parity

Requalify Tauri/direct Anthropic behavior.

If native already streams differently, align the externally observable service contract. If host-specific transport differences remain, document and test them explicitly.

Do not route native Claude through the hosted web relay just for code reuse.

8. Structured-output interaction

Coordinate with #729/#730.

Structured output and streaming are separate provider capabilities that may interact. Re-fetch current Anthropic support constraints before assuming every structured-output request should stream identically.

Do not let this issue duplicate the schema plumbing owned by #729.

---

Error taxonomy

Differentiate enough errors for actionable behavior:

PROXY_ADMISSION_FAILURE
AUTH_FAILURE
MODEL_NOT_FOUND_OR_NOT_ADMITTED
RATE_LIMITED
UPSTREAM_TIMEOUT
HOST_TIMEOUT
NETWORK_FAILURE
STREAM_PROTOCOL_FAILURE
MID_STREAM_PROVIDER_ERROR
USER_ABORT
CAPABILITY_UNSUPPORTED

Preserve sanitized user-facing messages.

Never log API keys, manuscript prose, provider bearer tokens or full sensitive upstream responses.

---

Testing

Mandatory CI must stay deterministic and must not call billable Anthropic APIs.

Relay contract tests

Cover:

• output ceiling at boundary / above boundary;
• timeout classification;
• stream requested vs non-stream request;
• streaming headers/body passthrough;
• upstream pre-stream failure;
• mid-stream failure;
• abort;
• no-cache behavior;
• existing same-origin/rate/body/model admission unchanged.

SSE parser tests

Fixture cases:

one frame per chunk
multiple frames per chunk
one frame split across many chunks
Unicode split across byte boundaries
final frame without trailing newline
provider error event
message stop
usage-bearing events
malformed event
abort during read

ProForge qualification

Use synthetic large outputs to prove:

• no obsolete 4K/8K truncation where the selected feature/model budget permits more;
• publishing workload receives the intended budget;
• stream delivery is incremental;
• final assembled response equals the concatenation of delivered text deltas;
• cancellation stops generation cleanly.

Real-provider qualification

Under #704's opt-in/manual provider test mechanism, perform representative real Claude streaming qualification with external maintainer credentials.

Do not put secrets or billable calls into ordinary public CI.

---

Metrics / evidence

Coordinate with the token-accounting sibling backed by #727.

For tuning, capture privacy-safe aggregate evidence such as:

model
feature/stage
requested max output
provider-reported input/output usage when available
time to first text delta
total duration
completion vs truncation/error category
host path
application SHA

Do not persist prompts/manuscript text for telemetry merely to measure transport performance.

---

Non-goals

This issue does NOT own:

• native structured outputs themselves — ai(proforge): use native Claude structured outputs for schema-bound pipeline stages #729 / feat(proforge): plan Claude structured-outputs wiring #730;
• full AI model catalogue refresh — ai: requalify current model catalog, defaults and end-to-end AI functionality across all providers #704;
• universal provider streaming redesign unless required by a shared abstraction;
• cancellation/supersession UI/state machine — ai(lifecycle): unify Stop Generation, cancellation, supersession and duplicate-request semantics #714;
• token accounting architecture — sibling issue / docs(proforge): plan real token accounting for the Claude path #727;
• automatically exposing the provider's maximum output limit to every user request;
• weakening rate/body/model/origin security gates;
• adding server-side storage of prompts/responses;
• making mandatory CI depend on Anthropic.

---

Acceptance criteria

• Prep PR docs(proforge): plan Claude max_tokens/timeout right-sizing #728 and its plan are incorporated/requalified rather than re-audited from scratch.
• Every Claude/ProForge output ceiling and timeout has a classified authority.
• Current Anthropic output/stream capabilities are re-fetched at implementation time.
• Supported relay-host runtime constraints are explicitly verified.
• Obsolete low output ceilings are corrected with evidence-backed bounded values.
• Publishing cap is workload-qualified rather than mechanically increased.
• Claude web streaming becomes genuinely incremental end-to-end.
• Relay no longer must buffer a successful streamed response before forwarding.
• Client parser handles fragmented/multiple SSE events robustly.
• Cancellation propagates through client/relay/upstream for streaming.
• Existing relay security/privacy gates remain intact.
• Direct/native Claude path is behaviorally requalified for parity.
• Error classification distinguishes timeout/network/provider/abort/protocol failures.
• Deterministic streaming fixtures cover chunk-boundary and failure edge cases.
• Representative ProForge large-output tests prove the old truncation/timeout assumptions are no longer accidental limits.
• Real-provider streaming verification remains opt-in and privacy-safe under ai: requalify current model catalog, defaults and end-to-end AI functionality across all providers #704.
• Final evidence is linked back to ai: requalify current model catalog, defaults and end-to-end AI functionality across all providers #704 and docs(proforge): plan Claude max_tokens/timeout right-sizing #728.
• Exact-head CI, CodeQL, review/thread convergence and resulting-main verification complete before terminal closure.

---

Priority

P1.

This affects the correctness and reliability of an advertised cloud-AI transport and can truncate or time out legitimate ProForge work while presenting a misleading streaming abstraction. The architecture work should still respect the active #704/#708 ordering and must not preempt a higher P0 data-integrity issue.

[qnbs]

Post-#728 implementation checkpoint — 2026-09-16

Prep/evidence PR #728 is now terminally merged. This issue remains OPEN/P1 as the runtime owner for Claude response budgets, timeout truth, and genuine end-to-end streaming.

The converged #728 review corrected several assumptions that are important enough to make explicit here before implementation:

1. Raising only per-agent caps is effectively a no-op for the default user

The effective ProForge config.maxTokens comes from settings.advancedAi.maxTokens, whose current default is 4096. Existing agent code uses Math.min(config.maxTokens, featureCap). Therefore a mechanical 4000 → 16000 feature-cap change yields roughly min(4096, 16000) = 4096, not the intended large-output budget.

Phase A must requalify the effective settings/default budget together with feature-specific ceilings rather than changing only local literals.

2. BaseAgent.buildAiOpts() is provider-generic

Changing the shared ProForge max-token configuration can affect Gemini/OpenAI/Grok/Ollama/etc., not only Claude. Before mutation, decide from current architecture whether the interim budget correction is Claude-specific or intentionally provider-neutral; do not accidentally change every provider while claiming a Claude-only fix.

3. Correct current workload framing

The earlier plan over-described some inputs:

• proofAgent currently truncates its source input to roughly 12,000 characters;
• publishingAgent uses bounded excerpts rather than sending a whole manuscript.

Requalify output budgets against the actual current workload/schema, not the older “whole manuscript” shorthand.

4. The 20s relay timeout is security policy, not just a performance constant

OUTBOUND_TIMEOUT_MS = 20_000 is documented in the current security threat model as part of the public unauthenticated relay's CWE-400/resource-exhaustion controls. Raising it requires an explicit abuse/resource-cost reassessment plus synchronized threat-model documentation. Do not apply the historical 55s candidate mechanically.

5. Genuine streaming must cover both Anthropic transport paths

After #759, Anthropic transport lives in services/ai/providers/anthropicProvider.ts:

Tauri/direct → api.anthropic.com
web → WorldScript relay → Anthropic

Both are currently part of the product contract. Implementing SSE only in the web relay or assuming the desktop path already behaves equivalently would leave host divergence.

6. Do not “fix” Grok

The final review reverified that Grok already uses real SSE (stream: true + the OpenAI-compatible stream consumer). A bot claim that Grok merely faked streaming was false. This issue remains Claude-specific except for shared abstractions genuinely required by current architecture.

7. Coordination with current owners

• ai(lifecycle): unify Stop Generation, cancellation, supersession and duplicate-request semantics #714 owns the general cancellation/supersession lifecycle; this issue owns only the transport-specific propagation required for correct Claude streaming and must compose with ai(lifecycle): unify Stop Generation, cancellation, supersession and duplicate-request semantics #714.
• ai(proforge): replace character-count token metrics with provider usage telemetry #732 owns provider usage/token accounting and should supply evidence for budget tuning rather than this issue inventing a second usage contract.
• ai(proforge): use native Claude structured outputs for schema-bound pipeline stages #729 owns Claude structured-output schema plumbing; re-fetch current Anthropic support for structured-output + streaming interaction rather than assuming capability composition.

Current architecture has advanced through #748/#759/#761, so re-fetch live source before implementation and treat old aiProviderService.ts line references as historical.

Merging #728 was planning convergence only; none of the runtime acceptance criteria above are satisfied merely by that merge.