[fix] the second dictation of a session stops inheriting the first one's socket - #396
Merged
Merged
Conversation
…e's socket Reported from a device: the first dictation works end to end, and from the second onwards the session either fails with `Connection failed. Please try again.`, or comes up listening to nothing. **A relay leg from the previous session could speak into the next one.** `handle(_:from:)` guards on `eventLeg == leg`, which protects a session from its own replaced legs — but `launch()` reset `leg` to 0 for every session, so the previous session's first leg and this one's carried the same number and the guard could not tell them apart. `finishUp()` is the one ending that drops the relay without cancelling it (`fail`, `cancel` and `endSessionWithMicTaken` all cancel), so a successful ⏹ left a live socket holding a closure that publishes `.closed` whenever the relay idles it out. A backgrounded process runs no main-actor work until something resumes it, and the thing that resumes it is the next `parley://dictate` — so the stale obituary was delivered *into* the session that had just replaced it. `handle` read it as this session's socket dying and called `scheduleReconnect()`, which cancels the current client: that is the connection failure when it lands during `start()`, and a pane listening to a socket that no longer exists when it lands after. `leg` is now bumped rather than reset, which is exactly what `captureGeneration` already does eleven lines above it and for exactly the same reason. That makes the stale event unrepresentable rather than merely unlikely; cancelling the socket sooner would not, because `readLoop`'s catch publishes `.closed` for a deliberate teardown as readily as for a dead peer. `finishUp()` now cancels too, for its own reason — every successful dictation was leaving a connection for the relay to time out. **The keyboard decided once whether it could hear the app.** All four Darwin observers were registered inside a `viewDidLoad` test of `hasFullAccess`, while `viewWillAppear` re-read the same value every appearance *because it changes*. The file contradicted itself, and the window is the main path rather than a corner: iOS kills this extension almost every time the user bounces to the app, so the second dictation is always served by a freshly loaded keyboard whose `viewDidLoad` ran during an app switch. One that read false there kept a working-looking voice pane that heard no downlink, no window, no readiness and no presence. Arming is now idempotent and retried on every appearance. Not addressed, because neither can be settled from this repository: whether the relay caps concurrent voice_typing streams per account (the leaked sockets above would have been hitting it), and what "I have to authorize again" refers to — the code cannot re-show iOS's microphone alert once it has been granted.
✅ SonarQube Quality Gate passed — pathorsAI_parley0 open issues on this PR. |
YJack0000
added a commit
that referenced
this pull request
Sep 23, 2026
…ictation (#418) * [fix] a relay client cancelled during its handshake still opened a socket (failing test) `SttRelayClient.cancel()` runs `tearDown` on a detached task, and `start()` never checks whether that already happened. A client cancelled while its owner was still waiting on the microphone went on to connect, send the config frame and start its keepalive loop, and nothing ever closed it. The test asserts the refusal `Spent`, which this commit declares and does not yet throw: today the attempt reaches the network and fails with a transport error instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * [fix] a relay client refuses to start once it has been finished or cancelled `start()` now throws `Spent` when `finish()` or `cancel()` came first, and again if a cancel lands during the handshake. The flag is a lock rather than actor state because `cancel()` is synchronous: an owner that cancels a client whose `start()` is still in flight needs the refusal to hold from the moment `cancel()` returns, not from whenever the detached `tearDown` gets the actor. Before this, a dictation stopped while `launch` was still waiting on the microphone had its client cancelled by `finishUp`, and that client then finished connecting anyway: config frame sent, keepalive running, socket open until the relay idled it out. That is a second live socket on the next dictation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * [fix] a dictation that ended mid-launch could take over the next one Every relay event and every redial already checks `leg` before it touches the session (#285, #396). Nothing else in the coordinator did. `launch()` resumes after the permission prompt, the microphone start and the socket handshake; `stop()` resumes after the finalize; `begin()` resumes after the stop it ran for the previous session. Each of them went on to write `relay`, `capture`, `state` and the downlink as if it still owned the session, and a session that had ended or been replaced during the wait paid for it: a `.listening` published over a `done`, a `Connection failed` published under the next session's id, the next session's relay cancelled and its transcript settled as `done`, or a second `AudioCapture` feeding the bridge alongside the first. Now every continuation checks `owns(owner)`, the same `leg` test the relay path uses, before it touches anything. `dismiss()` loses its unconditional `active = false`, which could have flipped a newer session's flag. The three microphone openers collapse into `openMicrophone()`, which joins an open already in flight instead of starting a second one. That is the overlap #404 documented and left to `begin(session:)`: the keyboard's URL fallback fires 700 ms into a background microphone start, and the foreground `launch` it triggers found `capture == nil` and opened another. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reported from a device on build 28: the first dictation works end to end; from the second onwards it fails — either
Connection failed. Please try again., or a pane that comes up listening to nothing.Diagnosed by reading, with two of the three links confirmed by throwaway tests against a local WebSocket server (written, run, deleted — not in this diff).
A relay leg from the previous session could speak into the next one
handle(_:from:)guards oneventLeg == leg, which protects a session from the legs it has already replaced. Butlaunch()resetlegto0for every session, so the previous session's first leg and this session's first leg carried the same number, and the guard could not tell them apart.That matters because of which ending abandons a live socket.
finishUp()— the successful ⏹ path, i.e. the one the user actually takes — didrelay = nilwith nocancel();fail(),cancel()andendSessionWithMicTaken()all cancel.SttRelayClient.finish()sends the finalize frame and deliberately leaves the socket open for the drain, but it also cancels the keepalive and the liveness watchdog, so afterwards nothing keeps that connection alive and nothing kills it. It sits there holding a closure that publishes.closedwhenever the relay eventually idles it out.A backgrounded process runs no main-actor work until something resumes it — and the thing that resumes it is the next
parley://dictate. So the stale obituary was routinely delivered into the session that had just replaced it:.closedarrives tagged leg 0; the new session is also on leg 0 → guard passes.handlereads it as this session's socket dying →scheduleReconnect().scheduleReconnect()cancels the current client.launch()is suspended ontry await client.start(), the cancelled client's in-flight send throws →Connection failed. Please try again..listening, the live socket is cancelled and the pane keeps drawing ⏹ over a connection that no longer exists.The fix is that
legis now bumped rather than reset — which is exactly whatcaptureGenerationalready does eleven lines above it, for exactly this reason. The file's own comment at the capture generation says a status "published while giving up in the background can be delivered after the user's next tap"; the relay side had the same hazard and the comment claiming legs already handled it was wrong across sessions.This makes the stale event unrepresentable rather than merely unlikely. Cancelling the socket sooner would not:
readLoop's catch publishes.closedfor a deliberate teardown exactly as readily as for a dead peer, so cancelling changes when the stale event is emitted, not whether.finishUp()now cancels the relay anyway, for a separate reason — every successful dictation was leaving a connection for the relay to time out at its leisure.Every leg is now id-prefixed, including the first.
leg == 0 ? nil : …was only ever an economy, and there is no leg 0 any more.The keyboard decided once whether it could hear the app
All four
DarwinObservers were registered inside aviewDidLoadtest ofhasFullAccess, whileviewWillAppearre-read the same value on every appearance — with a comment saying it does so because the value changes. The file contradicted itself.The window is not a corner case, it is the main path: iOS kills this extension almost every time the user bounces to the app (the code says so, in
drainDownlink's own comment), so the second dictation is always served by a freshly loaded keyboard whoseviewDidLoadran during an app switch. One that readfalsethere kept a working-looking voice pane — taps still mint sessions — while hearing no downlink, no window note, no readiness and no presence. The transcript would then only move on the next appearance, and ⏹ would look like it did nothing until the liveness watchdog fired ~25 s later.Arming is now idempotent and retried from every
viewWillAppear.Not a v1.15 regression
Worth stating plainly, since the previous two PRs were:
git diff ios-v1.14..origin/mainis empty forAudioCapture.swift,SttRelayClient.swift,RelayAudioBridge.swift,ReconnectPolicy.swiftandCaptureRecovery.swift. Theleg = 0line dates to #285 (2026-08-20), the commit that introduced the reconnect ladder. The Live Activity work did not cause this and therequestRefusedcache added in #395 was verified to hold across sessions —end()deliberately does not clear it, so the budget is oneActivity.requestper foregrounding, not per word.What this does not fix
Two of the three reported shapes are addressed; one is not.
launch()only reachesrequestPermission()on.undetermined, and the keyboard will not mint a session at all unless the readiness mailbox saysmicGranted. So either "authorize" means the trip through Parley / the Full Access notice (which the keyboard fix above would explain), or the grant is genuinely being lost. One question settles it: did iOS itself pop its grey "Parley would like to access the microphone" alert, or did the keyboard send you into Parley or into Settings?Also unresolved and unresolvable from this repository: whether the relay caps concurrent
feature=voice_typingstreams per account. If it does, the leaked sockets above were hitting it, and the socket hygiene in this PR fixes that too — but confirming it needs relay-side logs fromparley-internal.The structural finding behind all of this
ios/App/project.ymldeclares three targets and no test target.DictationCoordinator,AudioCaptureandKeyboardViewControllerall live in app/extension targets, so all 422 ParleyKit tests exercise value types only. There is no test covering any second-session path and structurally there cannot be one.That is why this class of bug — "works once, then stops" — keeps being found on a device rather than in CI. #328, #281, #360, #348 and now this one are the same shape. Worth its own ticket.
Verification
xcodegen generate+xcodebuild -scheme Parley -configuration **Release** -destination generic/platform=iOS→ BUILD SUCCEEDED.swift test --package-path ios/ParleyKit→ 422 tests, 1 skipped, 0 failures.parley://dictateis intercepted bysimctl's own confirmation dialog. Said plainly rather than implied: this fix is verified to compile and to be consistent with the code around it, and is not verified to resolve the founder's symptom on hardware.🤖 Generated with Claude Code