feat(stream): PartialOutput events for streaming structured output - #273
jaygupta17 wants to merge 1 commit into
Conversation
|
@jaygupta17 is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
|
Could a maintainer add the |
|
hey folks quick ping on #271 and #273 when you get a sec, been open since Aug 26. both are mergeable and green across 3.12-3.15 and Socket. the two reds are just fork perms (Vercel needs collaborator auth and changelog-label needs the |
804c02c to
797b315
Compare
|
Rebased onto latest main and ported onto the new _MessageHydrator: dropped my copy of _aggregate_event, the parse step now reads the hydrator message, telemetry fields come from the hydrator. Ruff, mypy, ty, pytest (188 across models/agents/types), and uv lock check all green locally. Same two reds as #271 need a maintainer (feature label + Vercel auth). |
With output_type set, streams only exposed text deltas; the parsed object appeared solely after StreamEnd via Stream.output. Consumers rendering progressive results (live forms, tables) had no way to see the object take shape. Streams now emit a synthetic PartialOutput event after each text delta that changes the parse of the accumulated JSON, plus expose the latest snapshot as Stream.partial_output. Parsing uses json-repair with stream_stable=True so truncated values stay verbatim strings instead of being creatively repaired mid-stream; non-dict parses (dangling keys) are skipped. Partials are plain dicts and deliberately unvalidated -- Stream.output remains the validated contract. Agent runs forward model stream events unchanged, so AgentStream consumers get partials for free; the UI adapter's event match ignores unknown kinds.
797b315 to
9071c60
Compare
|
Rebased onto latest main. I took the Vercel agent suggestion: synthetic PartialOutput events now carry the live message instead of the dummy default, with a test pinning it. Ruff, ty, mypy (src), and pytest are green locally. The two reds still need a maintainer: the feature label and Vercel auth. |
Closes #272
What
With
output_typeset,ai.stream()(andAgent.run()) now emit anai.events.PartialOutputevent after each text delta that changes the parse of the accumulated JSON. The latest snapshot is also available asstream.partial_output.PartialOutput.valueis a plaindict— the best-effort parse of the JSON so far, deliberately unvalidated.Stream.outputremains the validated contract once the stream ends.json-repairwithstream_stable=True: truncated values stay verbatim strings instead of being creatively repaired mid-stream, and successive snapshots only grow toward the final object.Stream.__anext__behind an internal queue, so message stamping and telemetry apply to partials exactly like provider events.Why json-repair
New core dependency (
json-repair==0.*, matching the repo's major-pin convention). Justification: MIT, zero required dependencies, ~50KB wheel, pure Python ≥3.10 — and itsstream_stablemode is purpose-built for this exact problem. It also gives the SDK a battle-tested repair primitive for future output-repair work. Without it we'd be maintaining a hand-rolled tolerant JSON parser (~150 lines) with a long truncation edge-case tail.Behavior notes
output_typenever emit partials; no changes for existing consumers that don't opt in\\(pinned by tests)AgentStreamconsumers get partials for free; the UI adapter's match ignores unknown kinds safelyValidation
uv run pytest: 766 passed — new tests cover the snapshot sequence vs deltas, dedupe of unchanged parses,partial_outputtracking, pinnedjson_repairtruncation behaviors (so a minor bump can't silently change semantics), no-partial-without-output_type, and agent-stream forwardingruff format --check/ruff check: cleanmypy: identical to baseline on main (pre-existing errors only)ty check: clean