Skip to content

[fix] the second dictation of a session stops inheriting the first one's socket - #396

Merged
Lanznx merged 1 commit into
mainfrom
ios-second-dictation
Sep 18, 2026
Merged

Lanznx merged 1 commit into
mainfrom
ios-second-dictation

Conversation

@YJack0000

Copy link
Copy Markdown
Contributor

Reported from a device on build 28: the first dictation works end to end; from the second onwards it fails — either Connection failed. Please try again., or a pane that comes up listening to nothing.

Diagnosed by reading, with two of the three links confirmed by throwaway tests against a local WebSocket server (written, run, deleted — not in this diff).

A relay leg from the previous session could speak into the next one

handle(_:from:) guards on eventLeg == leg, which protects a session from the legs it has already replaced. But launch() reset leg to 0 for every session, so the previous session's first leg and this session's first leg carried the same number, and the guard could not tell them apart.

That matters because of which ending abandons a live socket. finishUp() — the successful ⏹ path, i.e. the one the user actually takes — did relay = nil with no cancel(); fail(), cancel() and endSessionWithMicTaken() all cancel. SttRelayClient.finish() sends the finalize frame and deliberately leaves the socket open for the drain, but it also cancels the keepalive and the liveness watchdog, so afterwards nothing keeps that connection alive and nothing kills it. It sits there holding a closure that publishes .closed whenever the relay eventually idles it out.

A backgrounded process runs no main-actor work until something resumes it — and the thing that resumes it is the next parley://dictate. So the stale obituary was routinely delivered into the session that had just replaced it:

  1. .closed arrives tagged leg 0; the new session is also on leg 0 → guard passes.
  2. handle reads it as this session's socket dying → scheduleReconnect().
  3. scheduleReconnect() cancels the current client.
  4. If that lands while launch() is suspended on try await client.start(), the cancelled client's in-flight send throws → Connection failed. Please try again.
  5. If it lands after .listening, the live socket is cancelled and the pane keeps drawing ⏹ over a connection that no longer exists.

The fix is that leg is now bumped rather than reset — which is exactly what captureGeneration already does eleven lines above it, for exactly this reason. The file's own comment at the capture generation says a status "published while giving up in the background can be delivered after the user's next tap"; the relay side had the same hazard and the comment claiming legs already handled it was wrong across sessions.

This makes the stale event unrepresentable rather than merely unlikely. Cancelling the socket sooner would not: readLoop's catch publishes .closed for a deliberate teardown exactly as readily as for a dead peer, so cancelling changes when the stale event is emitted, not whether.

finishUp() now cancels the relay anyway, for a separate reason — every successful dictation was leaving a connection for the relay to time out at its leisure.

Every leg is now id-prefixed, including the first. leg == 0 ? nil : … was only ever an economy, and there is no leg 0 any more.

The keyboard decided once whether it could hear the app

All four DarwinObservers were registered inside a viewDidLoad test of hasFullAccess, while viewWillAppear re-read the same value on every appearance — with a comment saying it does so because the value changes. The file contradicted itself.

The window is not a corner case, it is the main path: iOS kills this extension almost every time the user bounces to the app (the code says so, in drainDownlink's own comment), so the second dictation is always served by a freshly loaded keyboard whose viewDidLoad ran during an app switch. One that read false there kept a working-looking voice pane — taps still mint sessions — while hearing no downlink, no window note, no readiness and no presence. The transcript would then only move on the next appearance, and ⏹ would look like it did nothing until the liveness watchdog fired ~25 s later.

Arming is now idempotent and retried from every viewWillAppear.

Not a v1.15 regression

Worth stating plainly, since the previous two PRs were: git diff ios-v1.14..origin/main is empty for AudioCapture.swift, SttRelayClient.swift, RelayAudioBridge.swift, ReconnectPolicy.swift and CaptureRecovery.swift. The leg = 0 line dates to #285 (2026-08-20), the commit that introduced the reconnect ladder. The Live Activity work did not cause this and the requestRefused cache added in #395 was verified to hold across sessions — end() deliberately does not clear it, so the budget is one Activity.request per foregrounding, not per word.

What this does not fix

Two of the three reported shapes are addressed; one is not.

  • Connection failure and the session hangs — both explained above.
  • "I have to authorize again" — not resolved, because the code as written cannot produce it. launch() only reaches requestPermission() on .undetermined, and the keyboard will not mint a session at all unless the readiness mailbox says micGranted. So either "authorize" means the trip through Parley / the Full Access notice (which the keyboard fix above would explain), or the grant is genuinely being lost. One question settles it: did iOS itself pop its grey "Parley would like to access the microphone" alert, or did the keyboard send you into Parley or into Settings?

Also unresolved and unresolvable from this repository: whether the relay caps concurrent feature=voice_typing streams per account. If it does, the leaked sockets above were hitting it, and the socket hygiene in this PR fixes that too — but confirming it needs relay-side logs from parley-internal.

The structural finding behind all of this

ios/App/project.yml declares three targets and no test target. DictationCoordinator, AudioCapture and KeyboardViewController all live in app/extension targets, so all 422 ParleyKit tests exercise value types only. There is no test covering any second-session path and structurally there cannot be one.

That is why this class of bug — "works once, then stops" — keeps being found on a device rather than in CI. #328, #281, #360, #348 and now this one are the same shape. Worth its own ticket.

Verification

  • xcodegen generate + xcodebuild -scheme Parley -configuration **Release** -destination generic/platform=iOS → BUILD SUCCEEDED.
  • swift test --package-path ios/ParleyKit → 422 tests, 1 skipped, 0 failures.
  • Simulator: the app builds, installs and launches on an iPhone 17 Pro sim, but the bug cannot be reproduced there — transcription needs a signed-in account and the hosted relay, and parley://dictate is intercepted by simctl's own confirmation dialog. Said plainly rather than implied: this fix is verified to compile and to be consistent with the code around it, and is not verified to resolve the founder's symptom on hardware.

🤖 Generated with Claude Code

…e's socket

Reported from a device: the first dictation works end to end, and from the
second onwards the session either fails with `Connection failed. Please try
again.`, or comes up listening to nothing.

**A relay leg from the previous session could speak into the next one.**
`handle(_:from:)` guards on `eventLeg == leg`, which protects a session from
its own replaced legs — but `launch()` reset `leg` to 0 for every session, so
the previous session's first leg and this one's carried the same number and the
guard could not tell them apart. `finishUp()` is the one ending that drops the
relay without cancelling it (`fail`, `cancel` and `endSessionWithMicTaken` all
cancel), so a successful ⏹ left a live socket holding a closure that publishes
`.closed` whenever the relay idles it out. A backgrounded process runs no
main-actor work until something resumes it, and the thing that resumes it is
the next `parley://dictate` — so the stale obituary was delivered *into* the
session that had just replaced it. `handle` read it as this session's socket
dying and called `scheduleReconnect()`, which cancels the current client: that
is the connection failure when it lands during `start()`, and a pane listening
to a socket that no longer exists when it lands after.

`leg` is now bumped rather than reset, which is exactly what `captureGeneration`
already does eleven lines above it and for exactly the same reason. That makes
the stale event unrepresentable rather than merely unlikely; cancelling the
socket sooner would not, because `readLoop`'s catch publishes `.closed` for a
deliberate teardown as readily as for a dead peer. `finishUp()` now cancels too,
for its own reason — every successful dictation was leaving a connection for the
relay to time out.

**The keyboard decided once whether it could hear the app.** All four Darwin
observers were registered inside a `viewDidLoad` test of `hasFullAccess`, while
`viewWillAppear` re-read the same value every appearance *because it changes*.
The file contradicted itself, and the window is the main path rather than a
corner: iOS kills this extension almost every time the user bounces to the app,
so the second dictation is always served by a freshly loaded keyboard whose
`viewDidLoad` ran during an app switch. One that read false there kept a
working-looking voice pane that heard no downlink, no window, no readiness and
no presence. Arming is now idempotent and retried on every appearance.

Not addressed, because neither can be settled from this repository: whether the
relay caps concurrent voice_typing streams per account (the leaked sockets above
would have been hitting it), and what "I have to authorize again" refers to —
the code cannot re-show iOS's microphone alert once it has been granted.
@github-actions

Copy link
Copy Markdown

✅ SonarQube Quality Gate passed — pathorsAI_parley

0 open issues on this PR.

@Lanznx
Lanznx merged commit 2bb378e into main Sep 18, 2026
3 checks passed
YJack0000 added a commit that referenced this pull request Sep 23, 2026
…ictation (#418)

* [fix] a relay client cancelled during its handshake still opened a socket (failing test)

`SttRelayClient.cancel()` runs `tearDown` on a detached task, and `start()`
never checks whether that already happened. A client cancelled while its
owner was still waiting on the microphone went on to connect, send the
config frame and start its keepalive loop, and nothing ever closed it.

The test asserts the refusal `Spent`, which this commit declares and does not
yet throw: today the attempt reaches the network and fails with a transport
error instead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* [fix] a relay client refuses to start once it has been finished or cancelled

`start()` now throws `Spent` when `finish()` or `cancel()` came first, and
again if a cancel lands during the handshake. The flag is a lock rather than
actor state because `cancel()` is synchronous: an owner that cancels a client
whose `start()` is still in flight needs the refusal to hold from the moment
`cancel()` returns, not from whenever the detached `tearDown` gets the actor.

Before this, a dictation stopped while `launch` was still waiting on the
microphone had its client cancelled by `finishUp`, and that client then
finished connecting anyway: config frame sent, keepalive running, socket open
until the relay idled it out. That is a second live socket on the next
dictation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* [fix] a dictation that ended mid-launch could take over the next one

Every relay event and every redial already checks `leg` before it touches
the session (#285, #396). Nothing else in the coordinator did. `launch()`
resumes after the permission prompt, the microphone start and the socket
handshake; `stop()` resumes after the finalize; `begin()` resumes after the
stop it ran for the previous session. Each of them went on to write `relay`,
`capture`, `state` and the downlink as if it still owned the session, and a
session that had ended or been replaced during the wait paid for it: a
`.listening` published over a `done`, a `Connection failed` published under
the next session's id, the next session's relay cancelled and its transcript
settled as `done`, or a second `AudioCapture` feeding the bridge alongside
the first.

Now every continuation checks `owns(owner)`, the same `leg` test the relay
path uses, before it touches anything. `dismiss()` loses its unconditional
`active = false`, which could have flipped a newer session's flag.

The three microphone openers collapse into `openMicrophone()`, which joins
an open already in flight instead of starting a second one. That is the
overlap #404 documented and left to `begin(session:)`: the keyboard's URL
fallback fires 700 ms into a background microphone start, and the foreground
`launch` it triggers found `capture == nil` and opened another.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants