[bug] Hands-free hangs on "Speaking · tap to stop" with no audio — AudioBufferSourceNode onerror never resolves the playback promise, no speaking watchdog

[serge-ivo]

Summary

Two related symptoms reported together on the Chess Coder (technical/coding Coder instance) in the console voice UI:

1. Status pill shows "Speaking · tap to stop" but no audio is heard.
2. In hands-free mode it never ends — it hangs in the speaking state forever instead of returning to listening and the mic never reopens.

Both symptoms share one root cause: the speaking state is entered but never exited, because the internal promise that signals playback completion is awaited on an event that is never fired.

---

Repro

1. Open the Chess Coder instance in the console.
2. Switch to hands-free mode.
3. Send a message. Wait for a reply.
4. Observe: pill shows "Speaking · tap to stop", no audio plays, mode never returns to "Listening…".
5. Also reproducible in tap-to-talk/auto-speak mode: pill stays "Speaking · tap to stop" permanently.

---

Root-cause analysis

1. AudioBufferSourceNode.onerror is not handled — the playback promise never resolves

packages/sdk/src/voice/tts.ts:339-342:

await new Promise<void>((resolve) => {
    source.onended = () => resolve();
    source.start();
});

This Promise has exactly one resolution path: source.onended. There is no source.onerror handler and no timeout. The Web Audio AudioBufferSourceNode can emit an error event (e.g. the decode succeeded but the output device is lost, suspended, or the audio session is interrupted mid-playback on iOS). That event goes unhandled — it does not reject the Promise and does not call resolve(). The Promise hangs indefinitely.

The outer try/catch at tts.ts:344 only catches thrown exceptions propagating from await; an error event on the source node does not become a rejected Promise.

Effect: _speakOpenAI never returns → speak() never exits its try block → the finally at tts.ts:178-182 never runs → this.speaking stays true forever.

2. use-voice.ts speakAndResume sets setSpeaking(true) and waits for tts.speak() to return

packages/sdk/src/voice/use-voice.ts:809-833:

const speakAndResume = useCallback(async (text: string) => {
    ...
    setSpeaking(true);
    try {
        const tts = await ensureTts();
        await tts.speak(text);   // <-- hangs here forever
    } catch {}
    setSpeaking(false);          // <-- never reached
    speakEndedAtRef.current = Date.now();
    setPaused(false);
    if (convoOnRef.current) {
        await startListening();  // <-- never reached
    }
}, ...);

Because tts.speak() hangs, setSpeaking(false) and startListening() are never called. In hands-free mode this means:

• The React speaking state stays true → resolveVoiceStatus in convo.ts:141 returns { label: "Speaking · tap to stop", ... } permanently.
• pausedForThinkingRef stays true → canOpenMic() returns false → the mic never reopens → the conversation dies.

3. No speaking watchdog exists

The transcription path has TRANSCRIBE_WATCHDOG_MS (use-voice.ts:70) that force-ends a stuck transcription. There is no equivalent for the speaking state. Any hang in tts.speak() — from this bug or any future one — produces a permanent stuck state with no recovery path short of a page reload.

4. Technical cleanForSpeech narrows the window but doesn't cause it

For a technical agent (Coder/Repo Chat), cleanForSpeech(text, { technical: true }) keeps identifiers and file basenames but strips fenced code, URLs, and git hashes. A reply composed mostly of code fences and links could reduce to (code) a link (code) or similar — non-empty, passes the if (!clean) return guard at tts.ts:161, and enters the full OpenAI TTS proxy path. Under normal conditions the proxy returns valid audio. The onerror bug is the trigger regardless of content, but a near-empty cleaned string may compound it (some TTS providers handle trivial inputs differently).

The if (!text?.trim()) return guard at tts.ts:159 and if (!clean) return at tts.ts:161 both correctly short-circuit before setting speaking = true, so an empty-string case does NOT hang. The hang only occurs when a non-empty cleaned string enters _speakOpenAI and source.onended never fires.

5. Verification via durable error log

TTS failures log to client:voice-tts (readable via MCP list_errors for this instance). Pulling those logs for the Chess Coder instance at the time of the report will confirm which branch fired — a non-OK response, an empty body, a context state fault, or a source.onerror event (which currently leaves no log entry, confirming the missing handler). The absence of a voice-tts log row while the pill is stuck would be diagnostic of the onerror path.

---

Affected files

• packages/sdk/src/voice/tts.ts — lines 339-342 (missing source.onerror) and lines 178-183 (the finally that never runs)
• packages/sdk/src/voice/use-voice.ts — lines 809-833 (speakAndResume), lines 777-806 (speak); no speaking watchdog anywhere

---

Acceptance criteria

• source.onerror handled: _speakOpenAI's playback Promise resolves (not hangs) on any AudioBufferSourceNode error event, the same way _speakBrowser uses u.onerror = finish. Logging the event to client:voice-tts is preferred over silent recovery so the audio session loss is observable.
• Speaking watchdog added: a timeout (suggested ceiling: 120 s or maxChars * 80 ms + 5 000 ms, mirroring the formula used by _speakBrowser at tts.ts:274) fires if tts.speak() has not returned, calls tts.cancel(), sets setSpeaking(false), setPaused(false), and (in hands-free mode) calls startListening(). The watchdog is cleared when tts.speak() returns normally.
• Consistent behaviour across paths: speak() (manual tap-to-hear) and speakAndResume() (auto-speak + hands-free) both use the watchdog — neither can strand the speaking state.
• Unit test: a test in tts.test.ts covering the case where a bufferSource fires an error event instead of ended — verifies that tts.speaking is false after the call, not true forever. (The hangingSynth helper in the existing test file (tts.test.ts:182) is the pattern to follow.)
• No regression on the existing TTS test suite: the iOS AudioContext "interrupted" recovery tests and the global-exclusivity tests continue to pass.
• After fix: switching the Chess Coder instance to hands-free mode and receiving a reply results in audio playback (or silent graceful fallback), followed by the status pill returning to "Listening…" and the mic reopening, within the watchdog deadline even if audio fails.

---

Labels / priority

Labels: bug, voice
Priority: high — the stuck speaking state completely disables hands-free mode for any agent whose TTS hits this path; the Coder agent is the primary use case for hands-free (coding by voice).