[bug] temperature=0 removed Whisper's repetition/truncation fallback — a clause the user spoke is missing from the transcript that reached the agent

[serge-ivo]

temperature=0 removed Whisper's repetition/truncation mitigation — clauses the user spoke are missing from the transcript

The owner reports voice sometimes not transcribing. Two recorded messages show a different failure from a dropped clip: the clip transcribed, and a whole trailing clause he spoke inside the recorded window is absent.

The evidence

dictation is persisted only when the live capture and the final transcript diverge (packages/sdk/src/voice/machine.ts:599-604), and heard is stamped at end-of-turn (machine.ts:410) — so anything in heard was spoken inside the clip. Two messages on Chess coder 2 (bd43f4de-ef35-4051-bdec-43f8571414a1), 2026-08-11:

UTC	sent	heard (live capture)
23:04:28	…and file issues that come out of it.	…file issues that come out of it **don't wait for the night rebuild we have to run**
23:00:49	…Headless, headless browser, but they should run.	…**also run Browser** but they should run **are they running an HP**

And a repetition loop that was sent as a real user turn at 21:44:05: "apps chess academy, chess academy, chess academy, …" ×6. A second, longer loop was caught and ignored at 01:33:37 ("chess-academy" ×14, client:voice "result ignored").

Mechanism — INFERRED, and the experiment that settles it

1449ee1 (#490) added form.append("temperature", "0") to all requests, including the streaming default model — packages/sdk/src/voice/stt.ts:401.

OpenAI's reference Whisper decoder uses a temperature fallback ladder (0 → 0.2 → 0.4 → … → 1.0), triggered on compression-ratio and average-logprob thresholds, and it exists specifically because pure greedy decoding produces repetition loops and premature truncation. Pinning temperature to 0 with no fallback removes that mitigation.

I cannot claim it introduced the repetitions. A drifting (not looping) repetition appears pre-#490 at Aug 10 02:11:24. What is established is that the documented mitigation is now gone, and pure loops appear after.

Settled by: unset temperature for a day and count repetition/truncation rows against the same window. That is a one-line change and a query, not a design argument.

Why it matters beyond tidiness

A dropped clip is visible — the user says it again. A silently truncated clip is not: the agent receives a grammatical, plausible instruction that is missing its operative clause. In the 23:04:28 case the lost words were "don't wait for the night rebuild we have to run" — an instruction with real consequences, spoken and never delivered.

Acceptance criteria

1. Decide, with reasons written down, whether temperature=0 should apply to the streaming model at all — it was added for the phantom-turn problem, which no_speech_prob was also meant to address (and does not, on this model — see the gate-calibration issue).
2. If greedy decoding is kept, add the repetition/compression-ratio check the ladder provided, or detect a repetition loop client-side before it is sent as a user turn (one was, at 21:44:05).
3. Where the live capture and the final transcript diverge by more than a threshold, that divergence is surfaced rather than silently resolved in favour of the transcript — the data is already persisted (machine.ts:599-604).
4. A test with a known repetition-loop transcript asserting it is not sent as a user turn.

[serge-ivo]

Fixed in de02397 (on main; live — verified below). Part of the stated mechanism is inverted,
so it is not implemented as written. Detail first, because it changes what the fix is.

Criterion 1 — the decision, with reasons

For whisper-1 the issue's mechanism is backwards. OpenAI documents temperature on this
endpoint as: "If set to 0, the model will use log probability to automatically increase the
temperature until certain thresholds are hit." On the Whisper decoder, 0 is the value that
selects the fallback ladder, not the one that removes it. #490's use of it there stands, and it
is kept.

It is dropped for the streaming gpt-4o-transcribe family, which is a different architecture
that wording does not describe. There, 0 is plain greedy decoding — the regime that produces
repetition loops — and it was added for the phantom-turn problem, which on that model is handled by
the energy gate and the noise filter. It was never shown to help there, so the model's own default
is now used.

Honest limit: if that default is itself 0, this change is inert and the loops have another
cause. The client cannot observe it. Which is exactly why the temperature line is not the
deliverable:

Criterion 2 — the deliverable, which holds whichever way that lands

isRepetitionLoop (audio.ts) judges the shape, not the words: a phrase of up to 5 words
repeated ≥4× consecutively and covering ≥50% of the transcript. isNoiseTranscript could never
catch these — it matches a fixed list of silence hallucinations, and a loop is built from your own
vocabulary, so every phrase in it is plausible. Wired into planSend as a third drop reason,
judged before the language gate so a decoder glitch cannot earn a "say that again in your
language" nudge, and the words stay on the failed bubble like the other two drops.

Both bars are set high on purpose: a false positive here drops a real turn, which is the harm
being prevented. "no, no, no, that is not what I meant", "very very good", and a real instruction
that merely ends in a stutter all still send. The recorded "chess academy" ×6 and
"chess-academy" ×14 do not.

Criterion 3 — divergence surfaced

Two things were wrong, both verified in code:

• the report was gated behind a pags:voice-debug localStorage flag, so in production it
  never fired at all;
• dictationDiverged asks about volume — for a long turn it needs the transcript to be >40%
  shorter, so a clause lost off the end of a paragraph does not move it.

lostTail(heard, final) asks the other question — how many words at the end of the live capture
the transcript does not account for — and ≥4 now reports unconditionally as voice-transcript.
It is a floor, not an exact count: the walk stops at the first word the transcript does have, so
your 23:04:28 case scores 6 for 10 lost words. Undercounting is the right direction for something
that only decides whether a row is worth writing.

Criterion 4 — tests

The exact transcript that reached the agent is asserted to be refused; the ×14 loop likewise;
punctuation/casing variants; six negative cases so a real turn is never eaten; ordering vs the
language gate; and lostTail against the recorded truncation.

Verified live

Deployed console bundle at https://proagentstore.online/console/:

let n=this.model!==`whisper-1`;n||t.append(`temperature`,`0`)

i.e. temperature is sent to whisper-1 only. The detector and dropNote strings are in the bundle too.

Not verified — and the query that closes it

The mechanism remains inferred; I did not reproduce it and I am not claiming it is fixed. The
detector prevents the symptom regardless of cause; the temperature change is the experiment you
named, now shipped as the default.

The measurement: in a week, count list_errors rows over a comparable window —
source:voice carrying the new repetition drop, and voice-transcript "dropped a clause heard
live". Two readings:

• repetition rows drop to zero → greedy decoding was the cause and the temperature change did it;
• repetition rows keep arriving but are now caught → the API default was already 0, the
  temperature line is inert, and the detector is the whole fix.

Either way nothing reaches an agent as your words again, which is the part that had consequences.
The voice-transcript rows are the more valuable half: they are the first evidence this platform
has ever collected about truncation, and if they show a pattern (one language, one mode, long turns)
that is a sharper issue than this one.