Voice: unmute command word — dictation-based wake-to-record with configurable keyword and start-recording signal

[serge-ivo]

Summary

When the mic is muted there is no hands-free way back. Saying "unmute" does nothing, so the user must tap the UI — which defeats hands-free, the one mode where mute is most likely to be used.

Why it does nothing today

The command vocabulary is a closed union of two values:

export type VoiceCommand = "repeat" | "mute";   // packages/sdk/src/voice/convo.ts:106

matchVoiceCommand (convo.ts:231) tests the repeat phrases, then the mute phrases, then returns null. The word "unmute" appears nowhere in the codebase except button tooltips and comments.

Verified on live instances: commandsEnabled: true, muteWords: [], repeatWords: [] — nothing is misconfigured, the feature simply does not exist.

It is a clean no-op rather than a misfire: phraseMatchesTranscript (convo.ts:214) requires a single-word phrase to BE the whole utterance, so "unmute" !== "mute" and saying it cannot accidentally re-mute.

Already done — do not rebuild

The background listener from the original criterion 2 shipped with #153. The always-on control listener runs whenever engaged && !mainRecording (use-voice.ts:1094), and muteFromCommand (:580) clears micOn while leaving convoOn set — so it is live for the entire muted period. Its own comment: "It runs while the agent speaks / thinks / is muted (mic closed)".

The browser is already listening the whole time you are muted and discarding the word purely because it is not in the vocabulary. This is a vocabulary + wiring change, not new audio plumbing.

Implementation map

Step	File	Where
Add "unmute" to the union	packages/sdk/src/voice/convo.ts	:106
Add UNMUTE_BY_LANG	convo.ts	next to MUTE_BY_LANG :119 — cover the same 10 languages
Match it before mute	convo.ts	matchVoiceCommand :231
unmute?: string[]	convo.ts	VoiceCommandWords :110
Ref + hydrate	packages/sdk/src/voice/use-voice.ts	:395 (ref), :421 + :813 (hydrate)
Pass to the 4 call sites	use-voice.ts	:616, :684, :703, :788
Handle the command	use-voice.ts	handleControlResult :604
Parse per-instance	packages/sdk/src/voice/config.ts	:31 type, :94 parse, :123 profile-fallback pair
Profile field voiceUnmuteWords	workers/api/src/lib/preferences.ts	:63 type, :112 default, :152 parse
Settings UI	store/console/src/components/VoiceFields.tsx	:40, :63, :272 (mirror the mute-words input)

Traps

1. Reuse the existing unmute path — do not write a new one. toggleMute (use-voice.ts:1121) already handles the stale-ref problem, with a comment recording the bug it fixed: setting only React state left startListening reading a stale mutedRef.current === true, so "unmute did nothing". A fresh implementation that sets state without flipping the ref reproduces exactly the symptom this ticket is about.
2. Only match while muted. Otherwise "unmute" said mid-sentence during a normal turn is a stray trigger. Gate on mutedRef.current.
3. Order matters. Check unmute before mute in matchVoiceCommand, so a user-configured phrase containing both words resolves the way they expect.
4. Custom words REPLACE built-ins (existing behaviour for repeat/mute) — keep that contract for unmute rather than merging.

Known limitation (accept, don't file later)

The control listener is a Web Speech dictation loop and is null on browsers without it — notably iOS Safari. Unmute-by-voice will not work there, the same limitation mute-by-voice already has. State it in the UI copy rather than treating it as a defect.

Acceptance criteria

• Saying a configured unmute word while muted re-opens the mic, hands-free, no tap.
• Saying it while NOT muted does nothing.
• A start-recording signal (audio cue and/or visible mic-active indicator) fires immediately on unmute — the only genuinely new UI in this ticket.
• voiceUnmuteWords on the profile + unmuteWords per-instance, with the per-instance value overriding a non-empty profile value (same precedence as mute; config.test.ts covers the pattern).
• Settings UI exposes the field with copy noting the iOS limitation.
• Unit tests in convo.test.ts (matching, ordering vs mute, whole-utterance precision) and config.test.ts (profile→instance merge).
• Voice settings remain reachable without opening job-application UI (see Profile page: voice commands are editable in two places (one silently ignored), and candidate fields show for every user #222 — they currently sit inside the Candidate Profile section).

Related

• Voice: mute command only active during active recording — must work at all times (during TTS, agent processing, co-pilot chat) #153 — shipped the always-on listener this depends on (closed)
• Voice: move hands-free control words to the global user profile #129 — mute command words, the mirror feature (closed)
• Voice: exit hands-free mode command word — return to text mode #165 — exit-hands-free command word; same matcher, same call sites — worth doing together
• Configurable voice control: hands-off command mode (next / play / record) #8 — broader hands-off command epic
• Profile page: voice commands are editable in two places (one silently ignored), and candidate fields show for every user #222 — where the settings field should live

[serge-ivo]

Confirmed still reproducing, and the work is smaller than this ticket assumes — one of the acceptance criteria is already satisfied.

Why "unmute" does nothing today. The command vocabulary is a closed union of two values:

export type VoiceCommand = "repeat" | "mute";   // packages/sdk/src/voice/convo.ts:106

matchVoiceCommand tests the repeat phrases, then the mute phrases, then returns null. There is no unmute anywhere in the codebase — only button tooltips and comments. Verified on live instances: commandsEnabled: true, muteWords: [], repeatWords: [], so built-in defaults apply and nothing is misconfigured.

It is a clean no-op, not a misfire: phraseMatchesTranscript requires a single-word phrase to BE the whole utterance, so "unmute" !== "mute" and saying it does not accidentally re-mute.

Criterion 2 is already done. "While muted, a background keyword listener is active (dictation-only — no LLM/STT pipeline)" — that shipped with #153. The always-on control listener runs whenever engaged && !mainRecording (use-voice.ts:1094), and muteFromCommand clears micOn while leaving convoOn set, so the listener is live for the entire muted period. Its own comment says so: "It runs while the agent speaks / thinks / is muted (mic closed)".

So the browser is already listening the whole time you're muted and discarding the word purely because it isn't in the vocabulary.

Remaining work is therefore:

1. Add "unmute" to VoiceCommand + an UNMUTE_BY_LANG set (unmute · unmute mic · start listening · resume listening, plus the other 9 languages already covered for mute).
2. Check it before mute in matchVoiceCommand, and only match while actually muted — otherwise "unmute" mid-sentence is a stray trigger.
3. handleControlResult → the existing unmute path at use-voice.ts:1123.
4. unmuteWords field in voice-settings + Settings → Voice UI (which per Profile page: voice commands are editable in two places (one silently ignored), and candidate fields show for every user #222 should move off the Candidate Profile section it currently sits inside).
5. The start-recording signal (criterion 4) — the only genuinely new UI.

One caveat worth recording: the control listener is a Web Speech dictation loop and is null on browsers without it (iOS Safari). So unmute-by-voice will not exist there — the same limitation mute-by-voice already has. Worth stating in the ticket rather than filing as a bug later.

[serge-ivo]

Correction to my earlier comment: the unmuteWords settings field belongs on /preferences (store/console/src/pages/Preferences.tsx → VoiceFields), which already exists and is the canonical home for voice command words. My earlier note pointed at the Profile page's block — that one is a dead legacy duplicate whose writes are never read; see #222. Add the field to VoiceFields.tsx only, and to users.preferences.voice via PUT /v1/preferences (workers/api/src/lib/preferences.ts), not to /v1/profile.

[serge-ivo]

Dependency: #228. Unmute will inherit the same bug verbatim if built first.

Mute is unreachable while the user is actively speaking a turn in Whisper/OpenAI STT mode (the default on real instances) — the control listener yields on micOn and the Whisper main path has no interim during capture. Unmute matters less in that specific window (you are already unmuted if you are recording), but the two commands share matchVoiceCommand, the same four call sites, and the same shouldRunControlListener gating. Fixing #228 first means unmute gets the corrected wiring for free; doing #152 first means implementing against a gate that is about to change.

Suggested order: #228 → #152 → #165.