Conversation
capabilities.turn_detection tells the framework who owns the input audio buffer, but the openai plugin set it from create_response, which says who answers the turn. With ServerVad(create_response=False) the server keeps segmenting and committing every turn while the framework commits as well, so each turn logged input_audio_buffer_commit_empty. turn_detection goes back to meaning "the server segments and commits the input audio". The half it absorbed becomes auto_turn_reply_generation, which defaults to True, so only the openai plugin needs a change. Five of the seven reads that wanted the segmenting meaning become correct with no edit, and AgentActivity reads the new flag at the six places that want "the server answers". Endpointing for an explicit create_response=False now comes from the server, so a client turn_detection="vad" is ignored with a warning. Pass turn_detection=None to keep full client control.
longcw
marked this pull request as draft
August 24, 2026 06:24
davidzhao
approved these changes
Aug 24, 2026
Contributor
Author
|
closing and create a fix in openai realtime instead #6965 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
capabilities.turn_detectiontells the framework who owns the input audio buffer. The openai plugin sets it fromcreate_response, which says who answers the turn.With
turn_detection=ServerVad(create_response=False)the server still segments and commits every turn, but the framework reads the capability as client-side turn taking and commits as well. Each turn logsinput_audio_buffer_commit_empty, which #6642 suppressed rather than removed.Seven reads of this capability want the "the server segments" meaning, so barge-in gatekeeping and the default VAD were wrong for this configuration too.
Fix
turn_detectiongoes back to its first meaning: the server detects turns, and it segments and commits the input audio itself.The half it absorbed becomes
auto_turn_reply_generation, which isTrueby default. Only the openai plugin needs a change, and a plugin that does not know the field behaves as before.Five of the seven wrong reads become correct with no edit.
AgentActivitygains_rt_server_reply_enabledfor the six places that want "the server answers", and only the twocommit_audio()calls needed an explicit gate.Note: with an explicit
create_response=False, endpointing now comes from the server's own speech events, so a clientturn_detection="vad"is ignored with a warning. Passturn_detection=Noneto keep full client control. This ordering is what closes the gap whereresponse.createcould otherwise overtake the server's commit and answer a turn the conversation does not hold yet.