fix(elevenlabs): end server vad turns - #1745
Conversation
|
| if (this.#opts.serverVad !== null) { | ||
| this.queue.put({ type: stt.SpeechEventType.END_OF_SPEECH }); | ||
| this.#speaking = false; | ||
| } |
There was a problem hiding this comment.
🚩 #speaking state is not reset on WebSocket reconnect
When updateOptions({ serverVad: ... }) triggers a reconnect via #reconnectEvent (line 639-641 in the run method), the #speaking boolean is not reset. If a reconnect happens mid-speech (e.g., after a partial_transcript but before a committed_transcript), the new WebSocket connection starts with #speaking = true, which means a subsequent partial_transcript won't emit START_OF_SPEECH. The new code at line 807-810 actually improves this situation for the VAD case, since each committed transcript now resets #speaking to false. However, the general reconnect-without-reset pattern is still a potential pre-existing issue for manual mode.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
expected since the framework already observed the SOS. Otherwise we have to send a fake EOS before reconnecting.
There was a problem hiding this comment.
🚩 Shared #opts reference between STT and SpeechStream enables live updates but has a narrow race window
The SpeechStream stores a direct reference to the STT instance's #opts object (plugins/elevenlabs/src/stt.ts:488). This means STT.updateOptions() modifying this.#opts.serverVad at line 437 is immediately visible to SpeechStream.#processStreamEvent() at line 799 — even before the WebSocket reconnects with the new commit_strategy. During the brief window between updateOptions() and the reconnect completing, the stream may process events using new serverVad state while the server is still operating under the old commit strategy. This is a pre-existing pattern (the old code had the same shared reference) and the window is very narrow, but worth noting for robustness.
(Refers to lines 497-503)
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Factually correct, benign in both toggle directions
Squashed local branch work, reconciled with the latest remote state: - Refactor ToolContext to parity class taking a list of Tool | Toolset (#1517) - feat(agents): add Toolset support to ToolContext and AgentActivity (#1525) - feat(agents): add beta end call tool (#1474) - fix(agents): thread FlushSentinel through Agent.create llmNode types - fix(elevenlabs): end server vad turns (#1745) - Add Inworld delivery mode inference TTS option (#1749) - Don't retain recorded events when recording is disabled (#1750) - fix(deepgram): validate updated model with stored language on updateOptions
Ports livekit/agents#5872 to agents-js.
Summary
Testing