[fix] a dropped relay socket is a pause in the live transcript, not the end of it - #285
Merged
Merged
Conversation
…he end of it A phone call, a Wi-Fi→LTE handover or a relay restart drops the STT WebSocket, and until now that cost either the words spoken during the gap (meetings) or the whole session (dictation, where any `.closed`/`.error` ended it and an error surfaced raw wire text to the user). The relay cannot resume a session — one socket carries one Soniox session, and a reconnect is necessarily a fresh leg whose clock restarts at zero. `SttRelayClient.Options.idPrefix`/`timeOffsetMs` already anticipated that; what was missing was anything to hold the audio in between, and a clock honest about where that audio belongs. ParleyKit gains the two testable pieces: - `RelayAudioBridge` — the microphone's only counterparty. It forwards to the current leg and holds chunks while there is none, bounded (45 s for meetings, 20 s for dictation) with drop-oldest, and hands the next leg both the held audio and the offset of its *first held sample*. Offsetting a leg to "now" instead would file a gap's worth of speech after the words that followed it. - `ReconnectPolicy` — the 1, 2, 4, 8 … capped ladder and the attempt ceiling, in one place instead of a `pow()` inline per recorder. `MeetingRecorder` moves onto both, so the meeting reconnect that landed in #272 no longer throws the gap away. `DictationCoordinator` gets reconnect at all: a dropped socket redials on the shorter dictation ladder with the microphone still open, the tentative tail is folded into the settled text rather than lost with the socket, and only an exhausted ladder ends the session — keeping every word the keyboard already typed. Reconnecting now looks different from failed, in both languages: an amber spinner on the meeting screen, "Reconnecting…" plus "keep talking" on the dictation screen and in the keyboard, against full-weight ink for a live transcript that is genuinely over. The keyboard learns a `reconnecting` downlink state so it stops going quiet mid-session. One failure is deliberately not retried: a 402 from the relay means the account is out of quota, and the next handshake is refused the same way. `SttRelayEvent.error` now carries the code so both paths can say that instead of spending the ladder to arrive at "lost the connection". Not covered here: Android has no audio-focus or interruption handling at all, and `MeetingSession` drops gap audio the same way this fixes on iOS. Tracked separately. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Lanznx
added a commit
that referenced
this pull request
Aug 20, 2026
Carries the reconnect (#285) and the microphone window (#289) to TestFlight, which is the only place the device checklists on #283 and #286 can be run — neither PR could verify its own load-bearing assumption from a simulator. The App Store is still serving 1.3 (checked across three storefronts; an earlier reading of 1.2 was a stale cache). 1.4 and 1.4.1 were tagged and built but never released, so their What's New copy has never been read. Rather than leave a user updating from 1.3 to be told about only half of what changed, the 1.4.1 section becomes 1.4.2 and gains this release's two entries — the window and the reconnect — on top of the copy it already carried forward. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 18, 2026
YJack0000
added a commit
that referenced
this pull request
Sep 23, 2026
…ictation (#418) * [fix] a relay client cancelled during its handshake still opened a socket (failing test) `SttRelayClient.cancel()` runs `tearDown` on a detached task, and `start()` never checks whether that already happened. A client cancelled while its owner was still waiting on the microphone went on to connect, send the config frame and start its keepalive loop, and nothing ever closed it. The test asserts the refusal `Spent`, which this commit declares and does not yet throw: today the attempt reaches the network and fails with a transport error instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * [fix] a relay client refuses to start once it has been finished or cancelled `start()` now throws `Spent` when `finish()` or `cancel()` came first, and again if a cancel lands during the handshake. The flag is a lock rather than actor state because `cancel()` is synchronous: an owner that cancels a client whose `start()` is still in flight needs the refusal to hold from the moment `cancel()` returns, not from whenever the detached `tearDown` gets the actor. Before this, a dictation stopped while `launch` was still waiting on the microphone had its client cancelled by `finishUp`, and that client then finished connecting anyway: config frame sent, keepalive running, socket open until the relay idled it out. That is a second live socket on the next dictation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * [fix] a dictation that ended mid-launch could take over the next one Every relay event and every redial already checks `leg` before it touches the session (#285, #396). Nothing else in the coordinator did. `launch()` resumes after the permission prompt, the microphone start and the socket handshake; `stop()` resumes after the finalize; `begin()` resumes after the stop it ran for the previous session. Each of them went on to write `relay`, `capture`, `state` and the downlink as if it still owned the session, and a session that had ended or been replaced during the wait paid for it: a `.listening` published over a `done`, a `Connection failed` published under the next session's id, the next session's relay cancelled and its transcript settled as `done`, or a second `AudioCapture` feeding the bridge alongside the first. Now every continuation checks `owns(owner)`, the same `leg` test the relay path uses, before it touches anything. `dismiss()` loses its unconditional `active = false`, which could have flipped a newer session's flag. The three microphone openers collapse into `openMicrophone()`, which joins an open already in flight instead of starting a second one. That is the overlap #404 documented and left to `begin(session:)`: the keyboard's URL fallback fires 700 ms into a background microphone start, and the foreground `launch` it triggers found `capture == nil` and opened another. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
14 tasks
Lanznx
added a commit
that referenced
this pull request
Sep 28, 2026
…quota, and the live screen looks like iOS (#486) * [fix] Android: a reconnect says it is reconnecting, a 402 says it is quota, and the live screen looks like iOS A dropped relay socket set RELAY_ERROR/RELAY_CLOSED and the screen said, in error red, that transcription had stopped unexpectedly — while the session was already redialling and the words were being held for the next leg. The live transcript now has iOS's split (MeetingRecorder.TranscriptionHealth, #285): RECONNECTING is an amber spinner and "Transcription dropped — reconnecting…", cleared when a leg comes back up or words arrive; only a real end (out of quota, reconnect budget spent, signed out) says live transcription stopped, drawn in ink with a bolt, because the recording itself is fine. The collapsed panel's health mark shows the same glyph as the line it stands for. A finished meeting whose upload hit 402 said "It uploads as soon as the network is back". Finished now carries waitingForQuota (the same cause the importer already reads off DrainResult.quotaExhausted) and says iOS's quota sentence; the recording still stays queued. The uploader reports which recordings it copied into an org, so the status can say "Synced, and shared to “Org”"; the other endings take iOS's copy too ("Synced to the cloud", "Sync failed for now…", "That recording was too short to keep", "Wrapping up…"). The screen: - the 12-segment level meter is replaced by iOS's scrolling capsule waveform (#371): pale bars, newest brightest, gliding between samples, in the full column and the compact row; - Discard is an outlined capsule with a trash glyph, 44dp tall, with a TalkBack description that says it asks first (#381); - transcript lines lose the blue dot on every line; only the turn still being spoken has a blue label; the words are selectable, tail included, and a long press offers Copy of the whole turn; - after Stop the transcript, outcome and filing card stay until Done or Back, as on iOS, instead of closing after 1.2 s; a settled session left behind is cleared before the next recording's consent prompt; - before the microphone opens the empty transcript says iOS's "Hit record and put the phone on the table to catch the whole room." Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * [refactor] LiveWaveform splits its clock and its drawing into helpers Sonar kotlin:S3776 flagged the composable at cognitive complexity 21 (max 15). The sampling/frame effect moves into rememberWaveformClock and a WaveformClock holder, and the per-frame drawing into DrawScope.drawWaveform / drawBar. Behaviour is unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: YJack0000 <jack@pathors.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Oct 4, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Part 1 of #283 — the relay half, on iOS.
What was actually broken
The report is "a phone call ends the recording", and reading the code it was three things. Only one of them still is.
The relay never reconnected on the dictation path, and threw the gap away on the meeting path.
DictationCoordinatorturned any.closedintofinishUp()and any.errorintofail(message)— with the raw wire string (relay error 402: …) shown to the user.MeetingRecorderdid reconnect (that landed in #272) but dropped the sink the moment the socket died, so everything said during the backoff was gone from the live transcript, and the new leg's offset wassystemUptime - captureStartedAt— "now" — which files a gap's worth of speech after the words that followed it.Can the relay resume a session? No. One socket carries one Soniox session; there is no resume token in the protocol (
SonioxProtocol.swift— config frame, keepalive, finalize, and that is the whole upstream vocabulary), andSttRelayClientis explicitly single-use. So a reconnect is necessarily a fresh Soniox leg whose token clock restarts at zero. That is exactly whatOptions.idPrefixandOptions.timeOffsetMswere put there for; what was missing was something to hold the audio across the gap and a clock honest about where that audio belongs. This PR does not paper over the lack of resume — it builds on the per-leg offset the protocol already assumed.iOS audio interruptions: already fixed, left alone.
AudioCapture(#272) observesinterruptionNotification,routeChangeNotification,.AVAudioEngineConfigurationChangeandmediaServicesWereResetNotification, rebuilds the engine rather than leaving a tap on a dead node, deliberately ignores the absent.shouldResume(which is exactly the "other party hung up" case), and runs a two-second watchdog for what iOS announces to nobody. That covers the reported case. I did not touch it.What this PR adds
RelayAudioBridge(ParleyKit). The microphone's only counterparty. It forwards chunks to the current leg and holds them while there is none, bounded (45 s meetings, 20 s dictation) with drop-oldest, and on the nextattachit hands the new leg both the held audio, in order, and the offset of its first held sample. Overflow advances that offset with the front of the buffer, so a leg is never told it is being fed audio it is not.attachtakes a factory closure so the offset, flush and swap are one step the audio thread cannot land in the middle of.ReconnectPolicy(ParleyKit). The 1, 2, 4, 8 … capped ladder and the attempt ceiling, in one place instead of apow()inline per recorder..dictationredials faster and gives up sooner — a two-minute session with someone standing there is not a meeting.MeetingRecordermoves onto both, so the gap is kept instead of dropped and the offset comes from the bridge.DictationCoordinatorgets reconnect at all: a dropped socket redials with the microphone still open, each leg gets its own id prefix so leg 2's first sentence cannot overwrite leg 1's, the tentative tail is folded into the settled text rather than dying with the socket, and only an exhausted ladder ends the session — keeping every word the keyboard already typed into the user's document.Reconnecting no longer looks like failed, in both languages:
The keyboard learns a
reconnectingdownlink state, so it stops going quiet mid-session. Red stays reserved for a session that actually ended; the meeting's "stopped" line is ink rather than red, because the recording is still running and colouring it like a failure would say the opposite of what the sentence says.One failure is deliberately not retried. A 402 means the account is out of quota and the next handshake is refused identically, so
SttRelayEvent.errornow carries the relay's code and both paths say what happened instead of spending the whole ladder to arrive at "lost the connection".Verified
swift test --package-path ios/ParleyKit— 37 tests, 16 new: hold/flush ordering, the bound and its drop-oldest, timestamp continuity across a simulated reconnect leg (10 s of meeting + 3 s into the gap ⇒ the new leg is offset to 10 000 ms, not 13 000), discard-keeps-the-clock, a concurrent-sender test that no chunk is lost or reordered while a leg is swapped underneath, the backoff ladder and ceiling, and the quota code the parser and the event have to agree on.xcodebuild -project ios/App/Parley.xcodeproj -scheme Parley -destination 'generic/platform=iOS Simulator' CODE_SIGNING_ALLOWED=NO build— clean (afterxcodegen generate).bunx tsc --noEmitandbunx vitest run(231 tests) — clean. Untouched by this change; run because CLAUDE.md asks for it.enandzh-Hantentries in the app and keyboard catalogs.Not verified — needs a device
None of the below is reachable from a simulator or a unit test, and I have not run any of it. From the checklist in #283, for iOS:
Two behaviours worth a specific eye on a device: the relay-side timestamps of a reconnected leg (the unit test proves the offset the client is given, not what the relay does with it), and whether a 402 arrives as an in-band error frame (handled) or as a socket close with code 1011 (falls through to the ordinary reconnect ladder, ending in the honest "live transcription stopped" line 75 s later).
Not in this PR
Android — no audio focus handling at all, and
MeetingSessionthrows the reconnect gap away exactly as iOS did. It is a separate PR by someone who can build it: there is no Android SDK on the machine this was written on and CI skipsandroid/**on pull requests, so shipping unbuildable Kotlin here would have been worse than not shipping it. Filed with the full design and the device checklist as #284.Closes nothing on its own — #283 stays open until #284 lands.
🤖 Generated with Claude Code