Skip to content

feat(stream): PartialOutput events for streaming structured output - #273

Open
jaygupta17 wants to merge 1 commit into
vercel-labs:mainfrom
jaygupta17:feat/streaming-partial-output
Open

jaygupta17 wants to merge 1 commit into
vercel-labs:mainfrom
jaygupta17:feat/streaming-partial-output

Conversation

@jaygupta17

Copy link
Copy Markdown
Contributor

Closes #272

What

With output_type set, ai.stream() (and Agent.run()) now emit an ai.events.PartialOutput event after each text delta that changes the parse of the accumulated JSON. The latest snapshot is also available as stream.partial_output.

  • PartialOutput.value is a plain dict — the best-effort parse of the JSON so far, deliberately unvalidated. Stream.output remains the validated contract once the stream ends.
  • Parsing uses json-repair with stream_stable=True: truncated values stay verbatim strings instead of being creatively repaired mid-stream, and successive snapshots only grow toward the final object.
  • Emission lives in Stream.__anext__ behind an internal queue, so message stamping and telemetry apply to partials exactly like provider events.

Why json-repair

New core dependency (json-repair==0.*, matching the repo's major-pin convention). Justification: MIT, zero required dependencies, ~50KB wheel, pure Python ≥3.10 — and its stream_stable mode is purpose-built for this exact problem. It also gives the SDK a battle-tested repair primitive for future output-repair work. Without it we'd be maintaining a hand-rolled tolerant JSON parser (~150 lines) with a long truncation edge-case tail.

Behavior notes

  • Streams without output_type never emit partials; no changes for existing consumers that don't opt in
  • Dangling keys without values are dropped; keys with a colon but no value complete with \\ (pinned by tests)
  • Agent runs forward model stream events unchanged, so AgentStream consumers get partials for free; the UI adapter's match ignores unknown kinds safely

Validation

  • uv run pytest: 766 passed — new tests cover the snapshot sequence vs deltas, dedupe of unchanged parses, partial_output tracking, pinned json_repair truncation behaviors (so a minor bump can't silently change semantics), no-partial-without-output_type, and agent-stream forwarding
  • ruff format --check / ruff check: clean
  • mypy: identical to baseline on main (pre-existing errors only)
  • ty check: clean

@vercel

vercel Bot commented Aug 26, 2026

Copy link
Copy Markdown

@jaygupta17 is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

@socket-security

socket-security Bot commented Aug 26, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedpypi/​json-repair@​0.63.3100100100100100

View full report

@jaygupta17

Copy link
Copy Markdown
Contributor Author

Could a maintainer add the feature label? (External PRs can't set labels — same as #271.) The Vercel deploy check also needs collaborator status, so it'll stay red on this fork PR; the Python 3.12–3.15 test matrix is what matters here.

@jaygupta17

Copy link
Copy Markdown
Contributor Author

hey folks quick ping on #271 and #273 when you get a sec, been open since Aug 26. both are mergeable and green across 3.12-3.15 and Socket. the two reds are just fork perms (Vercel needs collaborator auth and changelog-label needs the feature tag which I can't set). happy to jump on any feedback fast. thanks!

@jaygupta17
jaygupta17 force-pushed the feat/streaming-partial-output branch from 804c02c to 797b315 Compare September 4, 2026 06:12
@jaygupta17

Copy link
Copy Markdown
Contributor Author

Rebased onto latest main and ported onto the new _MessageHydrator: dropped my copy of _aggregate_event, the parse step now reads the hydrator message, telemetry fields come from the hydrator. Ruff, mypy, ty, pytest (188 across models/agents/types), and uv lock check all green locally. Same two reds as #271 need a maintainer (feature label + Vercel auth).

Comment thread src/ai/models/core/api.py Outdated
With output_type set, streams only exposed text deltas; the parsed
object appeared solely after StreamEnd via Stream.output. Consumers
rendering progressive results (live forms, tables) had no way to see
the object take shape.

Streams now emit a synthetic PartialOutput event after each text delta
that changes the parse of the accumulated JSON, plus expose the latest
snapshot as Stream.partial_output. Parsing uses json-repair with
stream_stable=True so truncated values stay verbatim strings instead of
being creatively repaired mid-stream; non-dict parses (dangling keys)
are skipped. Partials are plain dicts and deliberately unvalidated --
Stream.output remains the validated contract.

Agent runs forward model stream events unchanged, so AgentStream
consumers get partials for free; the UI adapter's event match ignores
unknown kinds.
@jaygupta17
jaygupta17 force-pushed the feat/streaming-partial-output branch from 797b315 to 9071c60 Compare September 8, 2026 07:41
@jaygupta17

Copy link
Copy Markdown
Contributor Author

Rebased onto latest main. I took the Vercel agent suggestion: synthetic PartialOutput events now carry the live message instead of the dummy default, with a test pinning it. Ruff, ty, mypy (src), and pytest are green locally.

The two reds still need a maintainer: the feature label and Vercel auth.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Structured output streaming: no way to see the object while it generates

1 participant