fix(xai): emit one final user transcript per turn - #2238
Conversation
🦋 Changeset detectedLatest commit: 0f0dfee The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| export class RealtimeSession extends OpenAIRealtimeSession { | ||
| private pendingTranscription?: realtime.ConversationItemInputAudioTranscriptionCompletedEvent; | ||
| private responseSpoke = false; |
There was a problem hiding this comment.
🟡 Newly added public class and its overrides ship without API documentation
The new publicly exported xAI realtime session class and its methods are added without any TSDoc comments, which the repository's contribution rules require for every new class/method.
Impact: Generated API documentation for the xAI plugin will have an undocumented public class.
Rule reference
CONTRIBUTING.md: "If writing new methods/interfaces/enums/classes, document them. This project uses TypeDoc for automatic API documentation generation, and every new addition has to be properly documented." The new RealtimeSession class (plugins/xai/src/realtime/realtime_model.ts:64) and its public members (close, interrupt, session() on RealtimeModel at plugins/xai/src/realtime/realtime_model.ts:59) carry no doc comments.
Was this helpful? React with 👍 or 👎 to provide feedback.
e89bf78 to
f1d4957
Compare
Ports livekit/agents#6729 and folds in agents-js#1912.
xAI retranscribes its persistent input item from the beginning at every audio commit. Hold the latest committed transcript until a response produces output, then emit one final transcript for the turn. This also closes silently abandoned response streams, preserves held turns during chat-context reconciliation, handles unknown xAI item anchors, and settles rejected OpenAI chat-context events without waiting for the timeout.
Source diff coverage
livekit-plugins/livekit-plugins-openai/livekit/plugins/openai/realtime/realtime_model.py->plugins/openai/src/realtime/realtime_model.ts. Ported event-ID-correlated chat-context waiters, nonfatal rejection handling, settled-waiter guards, reconnect reset ordering, generation-close/subclass hooks, and audio debug suppression using TypeScript primitives. The rebased target now provides nativetoUpdatereconciliation, which supplies the modified-item behavior required by held turns.livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/realtime/realtime_model.py->plugins/xai/src/realtime/realtime_model.ts. Added the xAI realtime session subclass with deferred final transcripts, silent-response disposal, held-turn reconciliation, unknown-anchor fallback, reconnect/close flushing, and audio/text response boundaries.livekit-plugins/livekit-plugins-xai/livekit/plugins/xai/types.py->plugins/xai/src/realtime/realtime_model.ts, where agents-js definesGrokRealtimeModels. Addedgrok-voice-think-fast-2.0.tests/test_realtime/test_openai_realtime_model.py->plugins/openai/src/realtime/realtime_model.test.ts. Ported all five chat-context rejection/waiter scenarios.tests/test_realtime/test_xai_realtime_model.py->plugins/xai/src/realtime/realtime_model.test.ts. Ported the source turn-state, mirror integrity, reconciliation, anchor, and abandoned-response scenarios; the Python-only speech timestamp assertion is represented by the target contract that repeated same-item speech starts keep the held turn pending because agents-js does not track per-item speech onset timestamps.Validation
pnpm exec turbo run build --filter=@livekit/agents-plugin-openai... --filter=@livekit/agents-plugin-xai...env -u OPENAI_API_KEY pnpm test -- plugins/openai(106 passed, 7 credential-gated skipped)env -u XAI_API_KEY pnpm test -- plugins/xai(25 passed, 1 credential-gated skipped)Package-wide lint was also run, but current
mainhas unrelated Prettier failures in untouched OpenAI and xAI files.cue-cliruntime validation remains unavailable because the environment has LiveKit credentials but noXAI_API_KEY.