Skip to content

[bug] collapseRepeat keeps the first occurrence's context forever — 11 voice discards shared one sample, and new telemetry fields vanish when a new-build row folds into an old one #538

Description

@serge-ivo

A repeated error keeps the first occurrence's numbers forever — so 11 turns shared one reading

workers/api/src/lib/error-log.ts:59-64, collapseRepeat:

UPDATE error_log
   SET repeat_count = repeat_count + 1, last_seen_at = datetime('now')
 WHERE id = (SELECT id FROM error_log WHERE source = ?1 AND message = ?2 AND level = ?3 …)

context is never updated. Occurrences 2..N are counted and time-stamped, and every measurement they carried is discarded.

What it cost today, concretely

The voice investigation on 2026-08-12 read this row:

"voice turn discarded at end-of-turn — the live gate heard no words"
  repeat_count 11, created 08:50:18, last_seen 09:02:01
  context: { peakLevel:0.664, noiseFloor:0, onsetFloor:0.1, frames:244,
             gateAlive:true, gateHeardSpeech:false }

Those numbers describe one turn — the 08:50:18 one. The other ten are invisible. The diagnosis held because the first sample happened to be unambiguous (0.664 against a 0.1 floor), but the questions that actually mattered — was the peak similar every time? did frames vary? was the gate dead throughout or only sometimes? — were unanswerable from a log that had physically recorded them and then thrown them away.

This is the telemetry 97bf5a4 (#510) added hours earlier precisely so this class of failure could be diagnosed from a query instead of a code read. Collapsing discards most of it.

It will now silently drop the newest diagnostics

9fb7cd6 (#535) added gateHearing and gateSawWords to these rows. The collapse key is (source, message, level) and these messages are deliberately fixed strings — so a new-build occurrence folding into an old-build row inside the dedup window keeps the old context and the new fields never appear. Anyone reading the log would reasonably conclude the fields were not deployed.

This no longer bites the end-of-turn message (that path can't emit after #535), but it applies to the noise-rejection and clip-gate rows, and to every future field added to any collapsed source.

Design tension worth naming

Collapsing exists for a good reason — client:voice-control produced 8 refusals in one burst on 2026-08-11, and one row per refusal would bury the log. The bug is not the collapse, it is that the row keeps the wrong sample. Options:

  1. Keep the latest context (SET context = ?) — one-line change; the count still shows the burst, and the sample tracks the newest occurrence. Loses the first, which is sometimes the interesting one.
  2. Keep first and last — a last_context column. Cheap, and covers "did it change over the burst".
  3. Collapse on a narrower key — include a coarse bucket of the numeric context so materially different occurrences do not merge. More faithful, most complex, and risks the burst-flood the collapse prevents.
  4. Do not collapse rows that carry a numeric context at all, only ones that do not.

(2) looks like the best value; (1) is the smallest honest improvement. Decide on the code and the real row volume.

Acceptance criteria

  1. A burst of N occurrences with differing context does not present as N copies of the first one's numbers.
  2. Whatever is kept, the row states which occurrence the context belongs to — a reader must not mistake a stale sample for the latest.
  3. A row created by an older build that later collapses a newer occurrence does not hide fields the newer build added.
  4. The burst-flood protection the collapse exists for is preserved; state the row-volume assumption.
  5. A test with two occurrences carrying different context, asserting the surviving row is not silently the first.

Verified

error-log.ts:59-64 reads exactly as quoted — the UPDATE sets only repeat_count and last_seen_at. The 11-occurrence row with a single frames:244 reading was measured today via list_errors. The claim about gateHearing/gateSawWords being dropped on a fold follows from the collapse key being (source, message, level) with fixed message strings; not yet observed live, because nothing has touched the microphone since the deploy.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions