[fix] the keyboard knows whether Parley is there: honest mic glyph, and a session nobody serves ends - #348
Merged
Conversation
…nd a session nobody serves ends Two reports from the voice keyboard, one missing fact behind both. **The record button said "opens Parley" while Parley was right there.** The glyph was a microphone only while a microphone window was open; otherwise it promised the jump, even with Parley in the foreground hosting the keyboard, where the no-jump Darwin start has always been served in place. It was also wrong the other way, silently: a Parley lingering in the background after a dictation *answered* the start note, then failed to open a microphone — iOS refuses to let a backgrounded process start recording — and published "Couldn't open the microphone. Open Parley…" for as long as the linger lasted. **Came back to a keyboard that says it is listening, transcribes nothing, and cannot be stopped.** The downlink says `listening` and keeps saying it whatever happens to the app. A backgrounded Parley is suspended without a hook when the host app takes the audio session, killed by jetsam, or swiped out of the app switcher, and in every case the keyboard kept drawing a stop button that nothing answered. A sixth App Group mailbox closes both: `AppPresence`, a heartbeat the app writes every 10 s for the process's whole life — `awake`, and `servesInPlace` (foreground, or a running microphone to borrow) — plus a goodbye written when the linger expires or the process terminates. - The keyboard draws the microphone and "Tap to speak" when a window is open *or* a fresh presence says a tap would stay put (`KeyboardBridge.staysPut`); the jump glyph otherwise. - The app declines a Darwin start it cannot honour — backgrounded with no running capture — so the keyboard's 700 ms fallback opens the app and the microphone is opened in the foreground, where it can be. The glyph is causal, not merely descriptive. - `Downlink.looksDead(presence:)`: a live state whose own stamp and the presence heartbeat are both older than 25 s is presumed dead. Terminal states never look dead, so a `done` still lands after a relaunch. - `checkLiveness` in the keyboard watches the earliest of three deadlines — presumed death, 10 s from minting with no answer, 3 s from ⏹ with no `finishing` — and gives up at it: the pane is cleared as ✕ would clear it, the app is sent the same cancel (delivered when it is resumed, if ever), and the slot says "Parley stopped listening. Tap the mic to try again." ⏹ itself no longer takes the pane out of `listening` on the press; that only meant the next drain put the button back. - `DictationCoordinator.init` reaps a live-state downlink left by a dead process, rewriting it as an error, so a relaunched app's fresh heartbeat cannot vouch for a session it never had. Verified: swift test (276, 14 new), xcodebuild Debug for the iOS Simulator (app + keyboard). New strings carry zh-Hant entries. Design doc updated (mailbox list, state table, "Knowing whether the app is there").
✅ SonarQube Quality Gate passed — pathorsAI_parley0 open issues on this PR. |
YJack0000
added a commit
that referenced
this pull request
Sep 21, 2026
… of sending every dictation through the app Reported by the owner: a few versions ago, once the microphone permission was granted and Parley had been opened once, the keyboard stopped jumping to Parley. Now every dictation jumps to Parley once. The jump is the keyboard's 700 ms fallback firing because the app declined the Darwin start. #348 (a220edc, shipped in 1.10) added a second guard to `DictationCoordinator.armRequestObserver`: a backgrounded process with no running capture refuses the note outright, on the premise that iOS never lets a backgrounded process start recording. With the microphone window off (the default) that is the state every lingering Parley is in after a dictation, so every tap since 1.10 has taken the round trip. The premise came from one phone's log. On the owner's phone the same activation succeeds, which is why #222 and #281 could ship the linger as a working no-jump path, and why the report reads as a regression. The observer now asks the audio session instead of assuming either answer. A lingering process with nothing to borrow opens a microphone before it acknowledges the start (`openMicrophoneInBackground`) and answers only if iOS let it. A refused activation stays silent, so the keyboard opens Parley exactly as it does today, and the error loop #348 was fixing ("Couldn't open the microphone" on every tap for the length of the linger) cannot come back. The condition the presence heartbeat publishes and the observer reads is one predicate, `canServeInPlace`; the heartbeat keeps reporting the pessimistic answer, so the record button promises a jump that may then not happen rather than the reverse. The design doc says the same where it used to state the premise as a rule. One new race, named in the doc: the `starting` ack now lands after `AudioCapture.start()` instead of milliseconds after the note, and the keyboard waits 700 ms. A slower background activation makes the keyboard open `parley://dictate` over a microphone already running, and `begin(session:)` stops and relaunches the session in the foreground: today's jump plus a restart. The activation time is unmeasured and now logged (subsystem com.pathors.parley, category Dictation) so a phone can report it. Verified: `swift test` for ParleyKit through xcodebuild on macOS (421 tests, 0 failures) and a Debug build of the Parley target with the iOS SDK (app + keyboard). Not verified on a phone: whether a given iOS build lets a backgrounded, recently foregrounded app with `UIBackgroundModes: audio` start recording is the fact this fix stops guessing at, and the simulator does not enforce it. This Mac has no simulator runtime installed and no signed-in account to dictate with. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
YJack0000
added a commit
that referenced
this pull request
Sep 22, 2026
… of sending every dictation through the app (#404) Reported by the owner: a few versions ago, once the microphone permission was granted and Parley had been opened once, the keyboard stopped jumping to Parley. Now every dictation jumps to Parley once. The jump is the keyboard's 700 ms fallback firing because the app declined the Darwin start. #348 (a220edc, shipped in 1.10) added a second guard to `DictationCoordinator.armRequestObserver`: a backgrounded process with no running capture refuses the note outright, on the premise that iOS never lets a backgrounded process start recording. With the microphone window off (the default) that is the state every lingering Parley is in after a dictation, so every tap since 1.10 has taken the round trip. The premise came from one phone's log. On the owner's phone the same activation succeeds, which is why #222 and #281 could ship the linger as a working no-jump path, and why the report reads as a regression. The observer now asks the audio session instead of assuming either answer. A lingering process with nothing to borrow opens a microphone before it acknowledges the start (`openMicrophoneInBackground`) and answers only if iOS let it. A refused activation stays silent, so the keyboard opens Parley exactly as it does today, and the error loop #348 was fixing ("Couldn't open the microphone" on every tap for the length of the linger) cannot come back. The condition the presence heartbeat publishes and the observer reads is one predicate, `canServeInPlace`; the heartbeat keeps reporting the pessimistic answer, so the record button promises a jump that may then not happen rather than the reverse. The design doc says the same where it used to state the premise as a rule. One new race, named in the doc: the `starting` ack now lands after `AudioCapture.start()` instead of milliseconds after the note, and the keyboard waits 700 ms. A slower background activation makes the keyboard open `parley://dictate` over a microphone already running, and `begin(session:)` stops and relaunches the session in the foreground: today's jump plus a restart. The activation time is unmeasured and now logged (subsystem com.pathors.parley, category Dictation) so a phone can report it. Verified: `swift test` for ParleyKit through xcodebuild on macOS (421 tests, 0 failures) and a Debug build of the Parley target with the iOS SDK (app + keyboard). Not verified on a phone: whether a given iOS build lets a backgrounded, recently foregrounded app with `UIBackgroundModes: audio` start recording is the fact this fix stops guessing at, and the simulator does not enforce it. This Mac has no simulator runtime installed and no signed-in account to dictate with. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two voice-keyboard reports, one missing fact behind both: the keyboard had no way to know whether the Parley process was there.
listeningand keeps saying it whatever happens to the app — suspended by an audio interruption from the host app, jetsammed, swiped away. The keyboard kept drawing a stop button that nothing answered.How
A sixth App Group mailbox,
AppPresence: a heartbeat the app writes every 10 s for the process's whole life (awake,servesInPlace), plus a goodbye at linger expiry andwillTerminate.KeyboardBridge.staysPut); jump glyph otherwise.Downlink.looksDead(presence:): a live state whose own stamp and the presence heartbeat are both older than 25 s is presumed dead. Terminal states never look dead, so adonestill lands after a keyboard relaunch.checkLivenesswatches the earliest of: presumed death, 10 s from minting with no answer, 3 s from ⏹ with nofinishing. At the deadline it clears the pane as ✕ would, sends the app the same cancel (delivered when/if it resumes), and shows Parley stopped listening. Tap the mic to try again.DictationCoordinator.initrewrites a live-state downlink left by a dead process as an error, so a relaunched app's fresh heartbeat cannot vouch for a session it never had.Design doc: mailbox list, the idle-state table, and a new section Knowing whether the app is there in
docs/design/ios-voice-keyboard.md.Assumption worth stating
The decline-in-background rule rests on the project's earlier on-device finding (#289) that iOS returns
AVAudioSessionErrorCodeCannotStartRecordingfor a backgrounded start even withUIBackgroundModes: audio. If that turns out to be wrong on some OS version, the cost is one extra jump on the linger path — never a broken session.Verified
swift testinios/ParleyKit: 276 tests, 0 failures (14 new inAppPresenceTests)xcodebuildDebug for the iOS Simulator: app + keyboard build cleanzh-Hantentries in both catalogsNot yet verified on a device: the two repro scenarios themselves (keyboard inside Parley's own text field; host app interrupting Parley's audio session mid-dictation). Device checklist in the PR description of the next iOS build.