Conversation
A realtime model keeps a tool call open server-side. Gemini Live blocks the turn until every call it emitted is answered and offers no way to cancel one, so discarding a result the handler already returned strands the session: it stops responding and subsequent generate_reply() calls produce no generation. Mirror the pipeline path on the realtime path — commit the outputs of tools that finished despite the interruption, handoffs excluded so they stay retryable — and additionally sync them to the realtime session. Gemini answers a delivered result with speech, which is unwanted right after an interruption, so the response is sent with SILENT scheduling where the API honours it (NON_BLOCKING declarations on the Gemini API). Also stop setting tool_response_scheduling on Vertex AI, which does not support the field, and warn when it is configured there instead of dropping it silently.
u9g
reviewed
Jul 29, 2026
| self._session._tool_items_added(interrupted_fnc_outputs) | ||
|
|
||
| # unlike the pipeline, a realtime model holds the call open server-side | ||
| chat_ctx = self._rt_session.chat_ctx.copy() |
Contributor
There was a problem hiding this comment.
chat_ctx already returns self._chat_ctx.copy(), so this copies twice.
Contributor
Author
There was a problem hiding this comment.
I'd keep copy() here to make it explicit since the rt_session.chat_ctx is implemented per plugin and not documented to must be a copy.
This was referenced Aug 2, 2026
create_function_response already skips scheduling for vertex, so the call site does not need to repeat the rule.
…put-on-interrupt # Conflicts: # tests/test_plugin_google_realtime.py
chenghao-mou
reviewed
Aug 5, 2026
|
|
||
| def interrupt(self) -> None: | ||
| # recorded locally since this cannot reach Gemini under automatic activity detection | ||
| self._playout_interrupted = True |
Member
There was a problem hiding this comment.
Q: if _start_new_generation clears the flag and but immediately sets it to True with _handle_input_speech_started, we would have silent tools whenever the generation is not triggered by "generate_reply"?
| chat_ctx = self._rt_session.chat_ctx.copy() | ||
| chat_ctx.items.extend(interrupted_fnc_outputs) | ||
| try: | ||
| await self._rt_session.update_chat_ctx(chat_ctx) |
Member
There was a problem hiding this comment.
seems update might be handled differently, from Sol:
Ultravox converts that output to `agent_reaction="speaks"`, while Phonic starts a new assistant turn. Thus an interrupted tool can make non-Google agents speak again. Silent interrupted-result delivery needs provider-specific semantics or an explicit capability.
Contributor
Author
|
Superseded by #6785, which broadens this to every tool call rather than just Gemini's. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gemini Live blocks a turn until every tool call it emitted is answered, and offers no way to cancel one. If playout is interrupted from our side while a tool result is still owed, dropping that result leaves the server waiting forever: it stops responding and subsequent
generate_reply()calls produce no generation. Reported in #6569.When it happens
Only when the interrupt is client-initiated — a programmatic
session.interrupt(), a handoff or close, false-interruption resume, or local turn detection firing alone. A voice barge-in makes Gemini sendinterrupted, which releases its wait on its own.Fix
The realtime path now mirrors what #6349 did for the pipeline path: commit the outputs of tools that finished despite the interruption, handoffs excluded so they stay retryable. On top of that it syncs those outputs to the realtime session, which the pipeline has no need to do.
Delivering a result makes Gemini reply, which is unwanted right after an interruption, so it goes out with
SILENTscheduling. Gemini ignoresschedulingonBLOCKINGdeclarations (the default), so with blocking tools the reply is unavoidable — unblocking the turn is the better trade.