[bug] Hands-free start has no feedback until the mic is live — and a failed start reverts silently

[serge-ivo]

The report

Pressing Hands-free feels like nothing happened — there is a visible delay before anything changes,
with no indication the tap registered.

Cause: every state flag flips only AFTER the whole startup completes

packages/sdk/src/voice/use-voice.ts:1080 — toggleConvo:

try { getAudioCtx().resume(); } catch {}
unlockSpeechSynthesis();                          // prime TTS for iOS/Safari
void ensureTts().then((t) => t.unlock());         // warm the TTS audio context
try {
    setPaused(false);
    sttRef.current = await makeStt();             // ← may fetch voice config
    await sttRef.current.start();                 // ← getUserMedia: device open + permission
    startAudioMonitor();                          // ← AudioContext + analyser
    setConvoOn(true);                             // ← FIRST visible change
    setSpeakOn(true);
    setMicOn(true);
    void acquireWakeLock();
    playListeningChime();
} catch { setConvoOn(false); }

setConvoOn / setMicOn / setSpeakOn are the last statements. Everything the UI keys off happens
after two awaits — makeStt() and start() (which is getUserMedia, opening a hardware device
and possibly checking permission). Until those resolve the button is in exactly the state it was
before the tap.

Confirmed: there is no starting / pending / busy state anywhere in the hook. The UI has no
value it could bind a spinner to even if it wanted one.

And the failure path is silent, which is worse than slow

} catch { setConvoOn(false); }

The catch swallows the error and resets to off. If the mic fails to open — device busy, permission
denied, another tab holding it — the user taps, waits, and the control simply stays off. No message,
no error, nothing distinguishing "still starting" from "failed" from "you didn't press it".

There is already classifyVoiceError / micUnavailableMessage for exactly this, used elsewhere in
the hook. This path does not reach them.

What it should do

• Expose a starting state, set synchronously at the top of toggleConvo and cleared in a
  finally. The mode control binds a spinner to it, so the press registers on the same frame.
• Keep the chime where it is — it confirms ready, not pressed, and those are different events.
• On failure, surface it through micUnavailableMessage rather than resetting silently. A user who
  denied permission months ago currently has no way to learn that is the problem.
• Worth measuring which await dominates. If makeStt() fetches voice config on every entry, that is
  cacheable and would cut the delay rather than just papering it with a spinner — the better fix
  where it applies.

Verification

• Tapping Hands-free changes the control's appearance within one frame, before the mic is live.
• The spinner clears exactly when listening actually begins (the chime and the spinner agree).
• A denied or busy mic produces a visible message, not a silent revert.
• Tapping again while starting does not open a second getUserMedia (there is an existing
  leaked-stream note at :1260 about ptt→handsfree doing precisely that).

[serge-ivo]

Fixed in 56ca1ae (on main; the Host Worker deploy for that commit succeeded, so it is live).

Your read of toggleConvo was exactly right, including that there was no starting/pending/busy state anywhere in the hook — the UI had nothing to bind a spinner to even if it wanted one.

The press is acknowledged on the same frame

starting is now real state, set synchronously at the top of toggleConvo (before the config read and before getUserMedia) and cleared in a finally, so a thrown start can never strand the spinner. The mode control binds to it: the Hands-free button shows a spinner and reads "Starting…" the moment you tap, in both the Assistant tab and the Coder Co-pilot. It is also a first-class VoicePhase and a resolveVoiceStatus state, so the pill agrees.

One subtlety worth recording, because it is easy to get wrong: starting had to be checked above the mode === "text" early return in both derivePhase and resolveVoiceStatus. convoOn has not flipped yet while the mic is opening, so the mode is still "text" — checked below that line, the state would have been unreachable for its entire lifetime and the press would still have looked unacknowledged. There is a test naming exactly that.

The chime stays where it is. As you said, it confirms ready, not pressed — the spinner covers the gap before it rather than replacing it, and clears exactly when listening begins, so the two agree.

The silent revert is gone

resolveToggleAction (pure, tested) makes a second tap while starting an ignore, not a stop — so it cannot run a second getUserMedia (the leaked-stream note at :1260), and an impatient tap cannot cancel the session it just asked for.

On failure it now reaches micUnavailableMessage. The reason that path "did not reach them" turned out to be a vocabulary mismatch: classifyVoiceError only ever knew the Web Speech codes (not-allowed, audio-capture), while this catch receives a getUserMedia DOMException named NotAllowedError / NotFoundError / NotReadableError — so it fell through to the generic "error" branch. Added normalizeMediaError to translate one dialect into the other, keeping a single classifier, plus a new audio-busy code for NotReadableError: the device-held-by-another-tab-or-app case has no Web Speech equivalent and is the one a user is least able to guess. So a mic denied months ago now says so.

The await you asked about

You were right to ask which one dominates, and right that it was cacheable. makeStt called invalidateVoiceConfig() and then await getVoiceConfig(...) on every entry — a full round trip to /v1/instances/:id/voice-settings, plus a second to /v1/keys/status whenever an OpenAI provider is configured, standing between the tap and getUserMedia.

That is now served from cache and revalidated behind the start (refresh: "background", with in-flight de-duping so a revalidate racing a cold read doesn't fan out). The obvious cost — a settings change landing one start late — is paid off rather than accepted: the settings writer calls invalidateVoiceConfig() on save and on "use my defaults", which is the one moment we know the cache is wrong. So the change still applies on the very next mic start, and the round trip is off the hot path entirely. The spinner covers what is left (getUserMedia itself, which is genuinely slow and genuinely can prompt).

iOS/Safari priming is intact

Deliberately verified, since this change inserts code ahead of it:

• getAudioCtx().resume(), unlockSpeechSynthesis(), and ensureTts().then(t => t.unlock()) are still in toggleConvo, still before the first await. The only things now preceding them are two synchronous statements (startingRef.current = true and a flushSync setState), so the call stack is still inside the user-gesture activation window.
• The "suspended" or "interrupted" resume logic lives in tts.ts / cues.ts, and neither file is in this diff — untouched.
• The re-entrancy guard returns before the priming, but only when a start is already in flight, in which case the priming already ran inside the first tap's gesture.
• setVoiceMode("ptt")'s own priming is unchanged.

Verification against your list

• ✅ tapping Hands-free changes the control within one frame, before the mic is live
• ✅ the spinner clears exactly when listening begins (spinner and chime agree)
• ✅ a denied or busy mic produces a visible message, not a silent revert
• ✅ tapping again while starting does not open a second getUserMedia
• ✅ the dominant await was measured and removed, not just covered

pnpm typecheck, the full unit suite (3660 tests), and biome check at its pre-existing baseline.