[fix] the recording survives leaving the foreground, and one tap means one recording - #272
Merged
Merged
Conversation
Three reports, one cause: nothing owned the recording. The live screen kept
the capture, the relay and the uploader in `@State` and only set
`isRecording` once every one of them was up.
**The meeting died after switching apps.** `UIBackgroundModes: audio` keeps
the process alive, but iOS has four ways of taking the microphone back —
`interruptionNotification` (call, Siri, another app opening a recording
session), `routeChangeNotification`, `.AVAudioEngineConfigurationChange`, and
`mediaServicesWereResetNotification` — and `AudioCapture` listened for none of
them. All four look identical from the outside: the app is still up,
`isRecording` is still true, the level meter sits at zero, and a minute later
the relay idle-closes for want of audio and reports "Relay closed". It reads
as a dropped connection; the network was never the problem.
`AudioCapture` now treats capture as something to hold rather than start. It
observes all four notifications, rebuilds engine, tap and converter on each,
retries with backoff, and polls every two seconds for what iOS announces to
nobody (a route that never came back, an interruption with no `.ended`). Each
transition is reported through `onStatus`, so the screen says the microphone
is paused instead of going quietly silent, and a microphone that is genuinely
gone ends the meeting properly rather than recording nothing.
**Two sockets transcribing one room.** Between the tap and `isRecording`
going true sat a WebSocket handshake. A second tap in that window ran
`start()` again: two `AVAudioEngine`s on the microphone, two relay sockets
streaming the same room into one transcript, two uploaders of which only the
last stayed referenced. `MeetingRecorder` makes the state machine the type —
`phase` moves to `.starting` synchronously on the tap, every entry point is
guarded by it, and the button reads it rather than a success flag.
**Seconds before recording began.** The relay handshake blocked the
microphone, and `setActive(true)` ran on the main actor. Now the client is
built before it is connected and buffers what the microphone gives it, so the
microphone opens first and the handshake runs alongside it;
`AudioCapture.start()` is `async` and does its work off the main actor.
Two more the same code hid:
- Every chunk was its own `Task { await client.send(pcm:) }` — one
unstructured task per 85 ms of audio, with no ordering between them, so the
provider could be handed speech with two chunks swapped. A single writer
now drains an `AsyncStream`, which also bounds the queue and is what makes
the connect window safe to record through.
- A closed socket ended live transcription for good. It now reconnects with
backoff while the microphone keeps running. Each leg needs its own id
prefix and time offset, because the provider restarts its numbering and its
clock at zero and the second leg's `mix-0` would otherwise overwrite the
opening of the meeting.
Dictation shares `AudioCapture` and the relay client, so it gets the
interruption recovery, the off-main-actor start and the ordered writer too.
Verified: `swift test` (20), Debug and Release builds, `bunx tsc --noEmit`,
`bunx vitest run`. The interruption and route paths need hardware — see the
PR for the device checks.
The Android pipeline was already the healthier of the two — foreground service, explicit state machine, ordered chunks, double-start guarded — so this is the two gaps it shared with iOS, plus the latency win. **A silenced microphone was invisible.** `MicCapture` documents the case and leaves it: from Android 10, when another app takes the microphone the framework does not fail, it feeds silence, and `AudioRecord.read` keeps returning success. A meeting could record forty minutes of nothing and report no problem. `MeetingSession` now registers an `AudioRecordingCallback` and publishes `micSilenced`; the screen shows it in error colour. The recording deliberately keeps running, because the takeover usually ends by itself and what follows is still worth having. **A dropped relay never came back.** `Closed` and `Error` mid-meeting only raised a banner and the transcript stopped for good. They now reconnect with backoff, capped per meeting, with the same per-leg id prefix and time offset the Swift side needed — the provider restarts its numbering and its clock at zero, so without them the second leg's `mix-0` overwrites the first minute of the transcript. **The microphone waited on the handshake.** `connect()` splits into `open()` and `awaitOpen()`. The config frame is queued as soon as the socket is created; OkHttp buffers frames until the upgrade lands and writes them in order, so it is guaranteed to reach the relay ahead of any audio and the microphone can open at the same time as the socket. Verified: `:parleykit:test`, `:app:testDebugUnitTest`, `:app:compileDebugKotlin`.
|
YJack0000
added a commit
that referenced
this pull request
Aug 20, 2026
App Store Connect refuses a `CFBundleVersion` it has already seen, and 1.3 (9) is on TestFlight, so the tag `ios-release.yml` checks against has to move with it. Both targets carry the number and both move together; `Info.plist` is regenerated from `project.yml` rather than edited, since the next `xcodegen generate` would otherwise overwrite it. 1.4 rather than 1.3 (10) because what it carries is behavioural, not a rebuild: the microphone now survives the app leaving the foreground, one tap can no longer start two recordings, and recording begins on the tap instead of after the relay handshake (#272).
Lanznx
added a commit
that referenced
this pull request
Aug 20, 2026
`project.yml` reached 1.4.1 and both tags went out, but `AppStore/metadata` still stopped at 1.3 — so the version about to be submitted has no What's New copy in either locale, and nothing tells a user why the globe key moved. 1.3 is what the App Store is serving, so these notes cover everything since it: the voice pane becoming a control panel (#266), the recording holding on to the microphone across foreground changes and the double-recording fix (#272), and the permission prompt surviving the keyboard hand-off (#281). The globe paragraph is written to be read by someone who read 1.3's. 1.3 announced the key on every device; this took it back where iOS already draws one, and saying so plainly is cheaper than letting people think it vanished. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 20, 2026
Lanznx
added a commit
that referenced
this pull request
Aug 20, 2026
`project.yml` reached 1.4.1 and both tags went out, but `AppStore/metadata` still stopped at 1.3 — so the version about to be submitted has no What's New copy in either locale, and nothing tells a user why the globe key moved. 1.3 is what the App Store is serving, so these notes cover everything since it: the voice pane becoming a control panel (#266), the recording holding on to the microphone across foreground changes and the double-recording fix (#272), and the permission prompt surviving the keyboard hand-off (#281). The globe paragraph is written to be read by someone who read 1.3's. 1.3 announced the key on every device; this took it back where iOS already draws one, and saying so plainly is cheaper than letting people think it vanished. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Lanznx
added a commit
that referenced
this pull request
Aug 20, 2026
…he end of it (#285) A phone call, a Wi-Fi→LTE handover or a relay restart drops the STT WebSocket, and until now that cost either the words spoken during the gap (meetings) or the whole session (dictation, where any `.closed`/`.error` ended it and an error surfaced raw wire text to the user). The relay cannot resume a session — one socket carries one Soniox session, and a reconnect is necessarily a fresh leg whose clock restarts at zero. `SttRelayClient.Options.idPrefix`/`timeOffsetMs` already anticipated that; what was missing was anything to hold the audio in between, and a clock honest about where that audio belongs. ParleyKit gains the two testable pieces: - `RelayAudioBridge` — the microphone's only counterparty. It forwards to the current leg and holds chunks while there is none, bounded (45 s for meetings, 20 s for dictation) with drop-oldest, and hands the next leg both the held audio and the offset of its *first held sample*. Offsetting a leg to "now" instead would file a gap's worth of speech after the words that followed it. - `ReconnectPolicy` — the 1, 2, 4, 8 … capped ladder and the attempt ceiling, in one place instead of a `pow()` inline per recorder. `MeetingRecorder` moves onto both, so the meeting reconnect that landed in #272 no longer throws the gap away. `DictationCoordinator` gets reconnect at all: a dropped socket redials on the shorter dictation ladder with the microphone still open, the tentative tail is folded into the settled text rather than lost with the socket, and only an exhausted ladder ends the session — keeping every word the keyboard already typed. Reconnecting now looks different from failed, in both languages: an amber spinner on the meeting screen, "Reconnecting…" plus "keep talking" on the dictation screen and in the keyboard, against full-weight ink for a live transcript that is genuinely over. The keyboard learns a `reconnecting` downlink state so it stops going quiet mid-session. One failure is deliberately not retried: a 402 from the relay means the account is out of quota, and the next handshake is refused the same way. `SttRelayEvent.error` now carries the code so both paths can say that instead of spending the ladder to arrive at "lost the connection". Not covered here: Android has no audio-focus or interruption handling at all, and `MeetingSession` drops gap audio the same way this fixes on iOS. Tracked separately. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



What this changes
The phone stops losing the microphone when the meeting is not on screen, one tap stops being able to start two recordings, and recording begins when the button is pressed instead of one network round trip later. Android gets the two robustness gaps it shared with iOS, plus the same latency fix.
Why
Three reports from dogfooding, which turned out to be three faces of one thing: nothing owned the recording.
LiveViewkept the capture, the relay and the uploader in@Stateand only setisRecordingonce all of them were up.1. The meeting died after switching apps
UIBackgroundModes: audiokeeps the process alive, but it says nothing about iOS taking the microphone back — and there are four ways it does, every one of them routine while the user is in another app.AudioCapturelistened for none.interruptionNotificationrouteChangeNotification.AVAudioEngineConfigurationChangemediaserverdrestartsmediaServicesWereResetNotificationAll four look identical from outside: the app is still up,
isRecordingis still true, the level meter sits at zero for the rest of the meeting, and about a minute later the relay idle-closes for want of audio and the screen says "Relay closed". It reads as a dropped connection. The network was never the problem — the microphone died first.AudioCapturenow treats capture as something to hold: all four notifications observed, engine + tap + converter rebuilt on each, backoff retry, and a two-second watchdog for what iOS announces to nobody (a route that never came back, an interruption that never posted.ended). Transitions surface throughonStatus, so the screen says the microphone is paused rather than going quietly silent; a microphone that is genuinely gone ends the meeting properly instead of recording nothing.2. Two sockets transcribing one room
Not a subtle race — a window you can hit on purpose:
sequenceDiagram participant U as User participant V as LiveView participant R as Relay U->>V: tap → consent → start() Note over V: isRecording is still false V->>R: await client.start() — TLS + HTTP upgrade U->>V: button still says "Start recording" → tap again Note over V: second start() V->>R: a second socket, a second AVAudioEngine, a second uploader R-->>V: both legs write segments into one transcript Note over V: self.relay holds only the last one;<br/>the first keeps streaming, unreferencedMeetingRecordermakes the state machine the type.phasemoves to.startingsynchronously on the tap,start()/stop()are guarded by it, and the button reads it rather than a success flag. No amount of care insidestart()fixes this while "am I recording?" is a flag set at the end.3. Seconds before recording began
The relay handshake blocked the microphone, and
setActive(true)— which negotiates the route, and takes its time when a Bluetooth device is around — ran on the main actor.Two more the same code hid
Task { await client.send(pcm:) }— one unstructured task per 85 ms of audio, with no ordering between them. The provider could be handed speech with two chunks swapped and would faithfully transcribe the garble. A single writer now drains anAsyncStream, which also bounds the queue and is what makes the connect window safe to record through.mix-0would otherwise overwrite the opening of the meeting. Covered by a new test on both platforms.Nothing about the audio file depends on the socket, so a dropped relay costs live transcript, not the recording — and the cloud transcribes the uploaded audio anyway. That is why this reconnects quietly instead of failing the meeting.
Dictation shares
AudioCaptureand the relay client, so it picks up the interruption recovery, the off-main-actor start and the ordered writer for free.Android
Already the healthier pipeline — foreground service, explicit state machine, ordered chunks, double-start guarded — so this is the gaps it shared, plus the latency win:
MicCapturedocuments the case and leaves it: from Android 10, when another app takes the microphone the framework does not fail, it feeds silence, andAudioRecord.readkeeps returning success. A meeting could record forty minutes of nothing and report no problem.MeetingSessionnow registers anAudioRecordingCallbackand publishesmicSilenced; the screen shows it in error colour. The recording deliberately keeps running — the takeover usually ends by itself and what follows is still worth having.connect()splits intoopen()+awaitOpen(). The config frame is queued as soon as the socket is created; OkHttp buffers frames until the upgrade lands and writes them in order, so it is guaranteed to reach the relay ahead of any audio.Related: #109 is the same class of bug (never more than one active listener) on desktop voice typing. This PR does not touch that path, so it does not close it.
How it was verified
bunx tsc --noEmitpassesbunx vitest runpasses (231 tests) — no TypeScript changed, run to confirm the base is cleanSegmentBuilderTests.swiftandSegmentBuilderTest.ktzh-Hantanden— 10 inios/App/Parley/Localizable.xcstrings, 1 inandroid/.../res/values*/strings.xml(this PR adds nosrc/i18n/messages.tsstrings)swift test— 20 testsxcodebuildDebug and Release./gradlew :app:compileDebugKotlin :app:testDebugUnitTest :parleykit:testDevice checks before this goes in a build
Known limitation, deliberately not addressed
The record button is inert while a finished meeting uploads ("Wrapping up…"). A long meeting on cellular can hold that for minutes. Fixing it means splitting
MeetingUploader.finishAndUploadinto "persist to the pending queue" and "upload", and releasing the button after the first. Left out on purpose: closing the two-meetings-at-once hole first is the safer order.