[bug] A deaf Web Speech gate vetoes 10 turns the energy detector proved were speech — isAlive() answers "has ever run", not "is hearing now"

[serge-ivo]

A microphone-starved Web Speech gate vetoes turns the platform's own detector proved were speech

Live on the owner, 2026-08-12 08:50–08:57 UTC, hands-free + Whisper. Ten discards, each with the telemetry 97bf5a4 added today:

"voice turn discarded at end-of-turn — the live gate heard no words"
  path:"end"  peakLevel:0.664  noiseFloor:0  onsetFloor:0.1
  frames:244  gateAlive:true   gateHeardSpeech:false

The mic measured 0.664 against a 0.1 onset floor over 244 frames — unambiguous speech by vad.ts:99-101's own definition — and the turn was destroyed unheard. The mic then reopened with no cue, which is what the owner reported as "it transcribes, then doesn't send, it just starts listening again."

isAlive() answers the wrong question

packages/sdk/src/voice/gate.ts assigns alive = true in exactly two places — :160 (onresult, honest) and :221 (onend, commented "it started and ended → the engine is running") — and never assigns false after :143. reset() (:290-296) clears heard and the utterance but not alive.

A recognizer that dies on audio-capture still fires onend. So a gate that has never received one audio frame reports isAlive() === true permanently.

machine.ts:145-148 then reads it as authority to condemn:

if (gate?.isAlive && !gate.heardSpeech) return "discard";

isAlive() means "has this engine ever run". endOfTurnAction needs "is this gate hearing right now". Those are different questions and only the first is answered.

The discard ignores evidence it holds

turn.ts:166-167:

if (decision === "end") {
    return endOfTurnAction(s.gate) === "discard" ? { action: "discard", report: CLOSE_END_DISCARDED } : { action: "transcribe" };
}

s.peakLevel and s.voiceFloor are in scope and unused — the idle branch one line below uses both.

And path:"end" is itself proof of speech: vadStep returns "idle"/null while !seen (vad.ts:305-331); "end" is only reachable at :340/:343, after onset, which requires level > onsetFloor && speaking (vad.ts:322).

Nothing is uploaded. stopDiscard() sets _discard = true (stt.ts:315-319) and mediaRec.onstop returns before _transcribeWhisper (stt.ts:479-483). idleRecycleRef suppresses the listening chime (use-voice.ts:771), so the recycle is silent.

Three rules over one body of evidence, and the harshest runs first

• planClipGate (vad.ts:212-223) consults condemned (alive && !heard) only when frames === 0; with frames > 0 it falls through to hadSpeech(peakLevel, noiseFloor) at :220 and would have transcribed this clip.
• planTurnClose's end branch runs first and destroys it, so the forgiving rule is unreachable.
• planNoiseRejection is a third.

97bf5a4's own docstring states the principle as "a gate that never ran vouches for nothing" — but a gate refused the microphone did run, so it is trusted to condemn.

The upstream cause is measured

list_errors, same window:

row	created	last seen	count
client:voice-gate "speech gate refused the microphone: audio-capture" {fails:1,burstMs:0}	08:51:09	08:57:20	10
client:voice "discarded at end-of-turn"	08:50:18	08:57:20	10

Identical last-seen, identical count. Dedup is one row per message per ~30s (mic-retry.ts:17-21) so counts are a floor, not a 1:1 pairing — but the lockstep is not coincidence. audio-capture is neither benign (mic-retry.ts:88-90) nor a permission verdict (convo.ts:1034-1036), so the gate retries forever, never latches denied, and stays "alive". fails:1 on every report means the counter resets between failures, so #425's backoff never accumulates past 400 ms.

Not a regression from #510

git show 97bf5a4 -- use-voice.ts: the removed lines are the same gate discard. 97bf5a4 unified the two branches and added the telemetry; the behaviour predates it, and alive = true on onend dates to 15466e3, the gate's introduction.

Caveat, stated: the client:voice-gate rows are themselves new as of 4cf17eb today (reporting was previously gated on isMicPermissionDenied), so their appearance today is not evidence that contention began today. Inferred, not verified: 4cf17eb also added a 400 ms+ delayed re-arm (gate.ts:245-247) where the gate previously re-armed instantly, which would widen the alive-but-deaf window. Not separable from pre-existing contention with the data available.

Issue coverage

• [bug] The #490 noise gate is wrong in both directions — no_speech_prob is dead code on the default model, and the adaptive floor discards peaks the same file calls speech #511 (open) criterion 4 covers only the positive direction — gate evidence outranking an energy heuristic. This is the negative direction: gate silence outranking energy that proves speech. Not covered. Its criterion 1 (no_speech_prob dead code) is independent — that lives in a branch never reached here, because nothing is uploaded.
• [bug] Repeated microphone prompts: three recognizers share one mic and two re-arm forever — and a denial in either is completely unlogged, so mute-by-voice can die silently #425 (open) covers the three-recognizer contention, not the veto authority.
• [bug] #490 added three paths that discard captured audio with no report — the idle recycle eats the clip and the live words, and a backgrounded tab discards every clip #510 (closed) added the telemetry that exposed this.

Acceptance criteria

1. A gate that has not received audio for the current turn cannot condemn it. Distinguish "has ever run" from "is hearing now" — a per-turn liveness fact, not a session-lifetime one.
2. Where the energy detector has already proven onset (path:"end" is reachable only after onset), gate silence alone does not discard.
3. The three rules over this evidence agree, or their precedence is stated where they are defined and tested. Today the harshest runs first and the most forgiving is unreachable.
4. A recognizer failing audio-capture is reflected in what the gate claims about itself.
5. Tests: alive:true, heard:false, frames>0, peak>floor must transcribe; a genuine silent clip must still discard.

What could not be reproduced

No microphone available. One observation from the owner settles the remaining ambiguity — does he see his words on screen before it recycles?

• No visible words → confirms the above (gate has no device).
• Words appear, then vanish → onresult is firing, so the gate has the device and heard is false for a different reason: acceptSpeech() returning false, or isNoiseTranscript(delta) at gate.ts:196. Same discard line, materially different cause and fix.

Workarounds that work today, verified in code

• Tap-to-talk — shouldAutoDetectEndOfTurn (vad.ts:261-268) returns false when manualTalk is set (use-voice.ts:452, :1724), so planTurnClose never runs and the clip reaches planClipGate, which transcribes it. Requires the tab visible, or frames === 0 puts it back on the condemned branch (vad.ts:215-217).
• Dictation mode — the gate is only constructed when sttIsWhisperRef.current && speechGateAvailable() (use-voice.ts:317).

Hands-free + Whisper is the only combination that hits this.

[serge-ivo]

Fixed in 9fb7cd6, deployed to the console and live in production.

What changed

The question, not the call site. gate.ts now carries a per-TURN liveness fact, isHearing(), set by onaudiostart — the UA reporting the microphone actually open — or by any result arriving, and cleared by reset() at every turn boundary and by any non-benign recognizer error, audio-capture included (criterion 4). onend no longer touches it and keeps its honest session-lifetime meaning in isAlive, which is now read by nothing but the telemetry. no-speech and aborted deliberately do NOT clear hearing: no-speech is the recognizer reporting from an open microphone that nobody spoke, which is the evidence the veto is actually made of.

The three rules are one function. speechVerdict (machine.ts) is the single precedence rule, and planClipGate, planTurnClose and planNoiseRejection all consult it:

1. a vouching gate outranks the energy heuristic ([bug] The #490 noise gate is wrong in both directions — no_speech_prob is dead code on the default model, and the adaptive floor discards peaks the same file calls speech #511 criterion 4);
2. a measurement of speech outranks gate silence (criterion 2 — the direction that was missing);
3. a measurement of silence discards whatever the gate says ([bug] STT transcribes silence in hands-free mode — Whisper hallucinations posted as phantom user turns ("Pottery Barn", "Thank you for watching") — gate fails on iOS Safari and when SpeechRecognition stalls #490 stands);
4. with no measurement at all, only a hearing gate may condemn ([bug] #490 added three paths that discard captured audio with no report — the idle recycle eats the clip and the live words, and a backgrounded tab discards every clip #510).

That is planClipGate's rule generalised, so its verdicts are byte-identical and the other two moved to it — the forgiving rule is no longer unreachable from the path that mattered. endOfTurnAction is gone rather than repaired; its one-line body was the defect.

Both readings of the open question are handled, so it no longer needs answering by eye. A gate with no device (isHearing:false) and a gate that had words and declined them (acceptSpeech false, or isNoiseTranscript) both now transcribe on the end path, because onset was proven. A test replays the production row verbatim under both. And captureDiagnostics now carries gateHearing and gateSawWords, which separate the two causes in the log itself: gateHearing:false = no microphone; gateSawWords:true = words arrived and were declined.

What this knowingly gives up

On the end-of-turn path a loud non-speech transient that clears the adaptive onset floor (a door slam) is now uploaded instead of vetoed. That was the last place the veto bit. Keeping it would mean keeping a veto that fires on a gate whose liveness we have just established we cannot trust, and the identical transient has always been uploaded in tap-to-talk, which runs planClipGate — so this equalises the two modes at a level the system already accepts, behind the same downstream defences (planNoiseRejection, isNoiseTranscript against the bias prompt, isRepetitionLoop).

The idle path is deliberately untouched: "idle" means onset never fired in 15 s, which is the VAD stating silence, and only a vouching gate overrules it. Reading hadSpeech(peak, floor) there too was considered and rejected — a lone spike inside the noise-sampling window would have started uploading 15-second mostly-silent clips to Whisper, which is the #490 failure that path exists to prevent.

One #510 test was removed rather than fixed: "still discards an end-of-turn the live gate condemns — and now says so". The end branch no longer discards, so its premise is gone. Every other path that throws a recording away still reports, and a new test pins that no gate reading at all can discard an end — so if vadStep ever grows an exit to "end" that is not onset-proven, that test is what fails and makes someone decide again.

Verified

pnpm typecheck (11 projects) · pnpm vitest run (8,029 tests) · biome · design tokens · agents allowlist · qa config · file-size ratchet — use-voice.ts held at its 2017 pin (the gate snapshot grew a field, not a line) · migrations · test-isolation · docs:drift. Playwright chromium+webkit locally: 369 passed, with one timing-sensitive settings spec red under full-suite load and green 3/3 in isolation; CI ran the whole bar including test:e2e and was green on 9fb7cd6, and the Host Worker deploy succeeded.

Production, checked against the served bundle rather than asserted: proagentstore.online/console/ now serves planTurnClose as e === "end" ? "speech" : "silence" → speechVerdict, the gate's onaudiostart handler setting both flags, gateEvidence reading isHearing, and rows carrying gateHearing / gateSawWords.

Not verified, and deliberately not claimed

• Not reproduced end-to-end — no microphone here. The behavioural proof is the telemetry: a fixed build should stop producing "discarded at end-of-turn" rows with frames > 0 and peakLevel above onsetFloor. Any row that does appear from now on carries gateHearing/gateSawWords and says which cause it was.
• Not a regression from 97bf5a4 — confirmed: the discard is byte-identical before it, and alive = true on onend dates to 15466e3, the gate's introduction.
• Whether 4cf17eb's 400 ms delayed re-arm widened the alive-but-deaf window remains inferred and not separable from pre-existing contention with the data available. Nothing here treats it as established.

#425 stays open

Untouched, and it should stay open. Three recognizers over one device, with fails:1 on every report because the counter resets between failures, so the backoff never accumulates past 400 ms. This change narrows what a deaf gate may CLAIM; it does not stop the gate going deaf, and the client:voice-gate "audio-capture" rows are expected to continue. One small, safe improvement is visible from here and is NOT folded in: micFails is reset in onresult on the gate but the failure run is per-device, not per-recognizer-object — the same reasoning stt.ts already applies by holding _micFails on the instance rather than in _startBrowser's closure. That belongs on #425 with its own measurement, not in this commit.

[serge-ivo]

Post-deploy log read — a non-reproduction, and a sharper correlation

Deploy landed 09:31:47 UTC (Deploy Host Worker, 9fb7cd6, success). Log read at 09:41 and 09:44 UTC.

No confirmation is available yet, and I am not claiming one. The newest row in the entire error log from any source is 09:04:55 UTC — nothing has touched the microphone since ~30 minutes before the deploy landed. So: no "discarded at end-of-turn" row after the deploy, and no row carrying gateHearing/gateSawWords, both of which are consistent with the fix and equally consistent with nobody having spoken. It is a recorded non-reproduction.

The correlation is exact, and one better than the issue body measured. Both rows carry repeat_count 11 (not 10) and the same last-seen second:

row	first	last seen	n
client:voice "discarded at end-of-turn"	08:50:18	09:02:01	11
client:voice-gate "refused the microphone: audio-capture"	08:51:09	09:02:01	11

client:voice-gate then records not-allowed at 09:02:02, one second after the last discard — the run ends with the permission latch, which is #425's terminal path. Under the new build that whole sequence writes gateHearing:false (the deaf-gate case), not gateSawWords:true — so the traced mechanism is the one that fired, and the alternative reading is not what was happening here. The fix covers both regardless.

Why a new row of that message should now be impossible, checked against the served bundle rather than the source: planTurnClose maps decision === "end" to energy: "speech", and speechVerdict returns transcribe on energy === "speech" before consulting the gate at all. The discard branch is unreachable from the end path. Any future occurrence therefore dates the emitting tab to the old build.

To confirm it in one turn: reload the console tab (it is served cache-control: no-store, so a reload gets the new JS; an already-open tab keeps the old one — which is the most likely reason to see one more of these). Then take a hands-free Whisper turn. Success = the turn transcribes with no new client:voice row, while client:voice-gate audio-capture keeps accruing (#425 is untouched by design).

One thing to watch that is NOT this issue

collapseRepeat (workers/api/src/lib/error-log.ts:61-62) updates only repeat_count and last_seen_at — the context of every occurrence after the first is discarded, inside a 60-minute window keyed on source+message. That is why the row above holds one frames:244 reading for eleven turns. It also means the new gateHearing/gateSawWords diagnostics will be silently dropped whenever a new-build row folds into an old-build row with the same (deliberately fixed) message string. It does not bite this particular message any more, because that path can no longer emit; it will bite the noise and clip-gate rows. Flagging it rather than fixing it here — it is a separate defect in the error log, not in the voice gate, and it wants its own issue.

[serge-ivo]

Post-deploy confirmation — the veto has not fired again, and the window it did not fire in is real

Following up the recorded non-reproduction. There is now a measurement, and it is a genuine
negative rather than silence.

The one row after the deploy is a pre-fix tab, not a post-fix veto

Two "voice turn discarded at end-of-turn" rows exist in the entire 23-day log. The second was
created 09:58:49Z, 27 minutes after 9fb7cd6 deployed (Host deploy finished 09:31:47Z), and
repeats to 10:26:59Z:

path:"end"  peakLevel:0.3808  noiseFloor:0  onsetFloor:0.1  frames:541
gateAlive:true  gateHeardSpeech:false

It does not carry gateHearing or gateSawWords. Those fields exist unconditionally in the
fixed build — a row from 11:26 carries all four — so the emitting tab was still running the cached
pre-fix bundle. Inferred from the field set, not proved from a build stamp: build stamping only
began at 11:01:08Z (cfe8ff79), so everything before that is build: null for a reason that says
nothing. There is independent support for the mechanism: a client:voice-control row at 03:11:06Z
carries the pre-4cf17eb context shape two minutes after that deploy landed. Cached tabs lag.

The absence afterwards is meaningful, and that had to be checked

Voice was demonstrably in heavy use afterwards on a build-stamped post-fix bundle
(cfe8ff7926ac): noise rejections 11:26:53 → 12:18:53, client:voice-transcript 11:40:23 →
12:07:11, mic-control 11:28:32 → 11:39:04, and voice-carrying user messages through 12:21:08Z.

So: ~52 minutes of confirmed post-fix voice use produced zero deaf-gate vetoes, where the
preceding hour produced eleven. That is a real negative, not "nobody spoke".

One post-fix row worth reading, because it shows the new diagnostics working

12:07:00Z  build cfe8ff7926ac
"voice turn rejected as noise — the live capture was kept"
{"transcript":"Thanks for watching!", "peakLevel":0.700, "frames":1668,
 "gateAlive":true, "gateHearing":true, "gateHeardSpeech":true, "gateSawWords":true}

The gate was hearing AND had words, and a different layer rejected the turn. That is #511's
mechanism, cleanly separated from this one by exactly the two fields this fix added — which is what
they were for.

What is still not verified

• Not reproduced end-to-end; no microphone here. The evidence is telemetric.
• Voice has been unused since 2026-08-12 12:21Z, so no further data will accrue until it is used
  again. The 52-minute window is what this rests on, and it is what it is.
• #425 remains open and untouched by this, as designed: client:voice-gate audio-capture rows
  continue. See that ticket for the contention measurement.

Closing. Criterion 1-5 are met in code and the behavioural prediction — that the row can no longer be
emitted from the end path — has held over the only post-fix traffic that exists.

[serge-ivo]

CI green on 7521eed7. Closing on the post-deploy measurement above: ~52 minutes of build-stamped post-fix voice traffic with zero deaf-gate vetoes, where the preceding hour produced eleven. Not reproduced with a real microphone, and stated as such.