Skip to content

[fix] a lingering Parley answers the keyboard's mic tap again instead of sending every dictation through the app - #404

Merged
YJack0000 merged 1 commit into
mainfrom
ios-keyboard-no-jump-regression
Sep 22, 2026
Merged

YJack0000 merged 1 commit into
mainfrom
ios-keyboard-no-jump-regression

Conversation

@YJack0000

@YJack0000 YJack0000 commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

Why

The owner reports that a few versions ago, once the microphone permission was granted and Parley had been opened once, the keyboard stopped jumping to Parley, and that now every dictation jumps to Parley once.

The jump is the keyboard's 700 ms fallback: startDictation publishes the request over the App Group and opens parley://dictate only when the app does not acknowledge. #348 (a220edc, shipped in iOS 1.10) added a second guard to DictationCoordinator.armRequestObserver that makes a backgrounded process with no running capture refuse the note outright, on the premise that iOS never lets a backgrounded process start recording. With the microphone window off, which is the default, that is the state every lingering Parley is in after a dictation, so every tap since 1.10 has taken the round trip. The premise came from one phone's log. On the owner's phone the same activation succeeds, which is why #222 and #281 shipped the linger as a working no-jump path and why the report reads as a regression.

The observer now asks the audio session instead of assuming either answer. A lingering process with nothing to borrow opens a microphone before it acknowledges the start and answers only if iOS let it. A refused activation stays silent, so the keyboard opens Parley exactly as it does today, and the error loop #348 fixed ("Couldn't open the microphone" on every tap for the length of the linger) cannot come back.

Scope

  • DictationCoordinator.armRequestObserver tries openMicrophoneInBackground() instead of declining when canServeInPlace is false.
  • DictationCoordinator.canServeInPlace is the one predicate behind both the guard and the servesInPlace heartbeat. The heartbeat still reports the pessimistic answer, so the record button promises a jump that may then not happen rather than the reverse.
  • docs/design/ios-voice-keyboard.md, the AppPresence doc and the KeyboardBridge.opensApp doc stop stating the premise as a rule.
  • Out of scope: the microphone window path, which reads intact; HostReturnPolicy, which is the jump back after a dictation, not the jump to start one.

Tradeoffs

Reverting #348's guard alone would bring back its error loop on phones where the background activation is refused. Publishing the starting ack only after the microphone opens would have cost every no-jump start the activation time inside the 700 ms window. Trying before acknowledging keeps the ack fast where the microphone was already open and pays the activation only on the path that used to be refused outright.

Blast Radius

Only the Darwin start path from a backgrounded app with no running capture changes. Foreground starts, starts that borrow an open window, and the URL path are untouched. Where iOS refuses the activation the user sees today's behaviour, a jump after 700 ms plus one refused setActive. Where it succeeds the dictation stays in the host app, which is what the linger was built for. One new race: the starting ack now lands after AudioCapture.start() instead of milliseconds after the note, and the keyboard waits 700 ms, so a background activation slower than that makes the keyboard open parley://dictate over a microphone already running and begin(session:) stops and relaunches the session in the foreground. That degrades to today's jump plus a restart, not to a lost dictation.

Verification

  • xcodebuild -scheme ParleyKit -destination 'platform=macOS' test: 421 tests, 1 skipped, 0 failures. swift test itself cannot run on this Mac (the Command Line Tools' SwiftPM fails to link the manifest, and xcrun under DEVELOPER_DIR=/Applications/Xcode.app/... aborts on an arm64-only libxcrun.dylib), so xcodebuild drives the same package.
  • xcodebuild -target Parley -sdk iphoneos build CODE_SIGNING_ALLOWED=NO with the asset catalog excluded: app, keyboard and activities compile. The simulator SDK build and the catalog step both stop at actool, which needs a simulator runtime, and none is installed.
  • Not verified on a phone or simulator. Whether a given iOS build lets a backgrounded, recently foregrounded app with UIBackgroundModes: audio start recording is the fact this fix stops guessing at; the simulator does not enforce it, this Mac has no simulator runtime, and there is no signed-in account to dictate with. The background start() latency is unmeasured here; the app now logs it (subsystem com.pathors.parley, category Dictation, "background mic start took N ms" or "refused after N ms") so the owner's phone can report it through Console. No ParleyKit test was added: the ack decision lives in the app target's coordinator and in AVAudioSession, not in package logic.

🤖 Generated with Claude Code

@github-actions

Copy link
Copy Markdown

✅ SonarQube Quality Gate passed — pathorsAI_parley

0 open issues on this PR.

… of sending every dictation through the app

Reported by the owner: a few versions ago, once the microphone permission was
granted and Parley had been opened once, the keyboard stopped jumping to Parley.
Now every dictation jumps to Parley once.

The jump is the keyboard's 700 ms fallback firing because the app declined the
Darwin start. #348 (a220edc, shipped in 1.10) added a second guard to
`DictationCoordinator.armRequestObserver`: a backgrounded process with no
running capture refuses the note outright, on the premise that iOS never lets a
backgrounded process start recording. With the microphone window off (the
default) that is the state every lingering Parley is in after a dictation, so
every tap since 1.10 has taken the round trip. The premise came from one phone's
log. On the owner's phone the same activation succeeds, which is why #222 and
#281 could ship the linger as a working no-jump path, and why the report reads
as a regression.

The observer now asks the audio session instead of assuming either answer. A
lingering process with nothing to borrow opens a microphone before it
acknowledges the start (`openMicrophoneInBackground`) and answers only if iOS
let it. A refused activation stays silent, so the keyboard opens Parley exactly
as it does today, and the error loop #348 was fixing ("Couldn't open the
microphone" on every tap for the length of the linger) cannot come back. The
condition the presence heartbeat publishes and the observer reads is one
predicate, `canServeInPlace`; the heartbeat keeps reporting the pessimistic
answer, so the record button promises a jump that may then not happen rather
than the reverse. The design doc says the same where it used to state the
premise as a rule.

One new race, named in the doc: the `starting` ack now lands after
`AudioCapture.start()` instead of milliseconds after the note, and the keyboard
waits 700 ms. A slower background activation makes the keyboard open
`parley://dictate` over a microphone already running, and `begin(session:)`
stops and relaunches the session in the foreground: today's jump plus a
restart. The activation time is unmeasured and now logged (subsystem
com.pathors.parley, category Dictation) so a phone can report it.

Verified: `swift test` for ParleyKit through xcodebuild on macOS (421 tests, 0
failures) and a Debug build of the Parley target with the iOS SDK
(app + keyboard). Not verified on a phone: whether a given iOS build lets a
backgrounded, recently foregrounded app with `UIBackgroundModes: audio` start
recording is the fact this fix stops guessing at, and the simulator does not
enforce it. This Mac has no simulator runtime installed and no signed-in
account to dictate with.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@YJack0000
YJack0000 force-pushed the ios-keyboard-no-jump-regression branch from 0949e61 to 8e93565 Compare September 21, 2026 06:58
@YJack0000
YJack0000 merged commit 487f3b7 into main Sep 22, 2026
3 checks passed
@YJack0000
YJack0000 deleted the ios-keyboard-no-jump-regression branch September 22, 2026 18:07
YJack0000 added a commit that referenced this pull request Sep 22, 2026
…he panes are tabs (#406)

* [feature] the 注音 pane types like the system keyboard: toneless continuous input, delete under ㄦ

Closes #400. The composer holds a buffer of up to six syllables and
segments a run of untoned symbols on its own, so ㄋㄧㄏㄠ reads as
ㄋㄧ ㄏㄠ and commits as 你好 with no tone key between them. The
dictionary gains `~`-prefixed toneless rows (the union of a reading's
five tones, ordered by corpus frequency) because the first tone is
written with no mark and could not double as "tone not typed yet".
The candidate bar follows the oldest pending syllable; tapping one
commits it and moves on; return commits everything; punctuation from
the symbol planes commits first.

Delete moves from the function row to the eleventh column of the
fourth symbol row, under ㄦ, where the system 注音 keyboard keeps it.
Rows two and three are staggered by thirds rather than centred, and
123 / return are two and a half keys wide, as on the system keyboard.

* [chore] iOS 1.16 (build 31): release notes, and the 注音 pane design as it now is

Three things ship in 1.16, and all three are keyboard-side.

The 注音 pane takes toneless continuous typing (#400). The composer holds an
ordered buffer of up to six syllables instead of one, so a symbol that cannot
extend the last syllable starts a new one rather than overwriting a slot — the
rule that used to turn ㄋㄧ followed by ㄏ into ㄏㄧ, and the reason a sentence
could not be typed without a tone key between every syllable. Tones now sharpen
the guess instead of gating the next syllable, the candidate bar serves the
oldest pending syllable, and the dictionary carries `~`-prefixed toneless rows
because first tone is written with no mark and so cannot double as the toneless
key. Delete moves to the eleventh column of the fourth symbol row, where the
system keyboard keeps it, and rows 2 and 3 are staggered rather than centred.

#405 makes the pane track read the touch alongside the keys, so a swipe between
the voice, English and 注音 panes works from anywhere on the keyboard and does
not type the key it started on.

#404 stops the keyboard's mic tap taking a detour through Parley on every tap:
a backgrounded Parley now tries the microphone before declining, and only the
starts iOS actually refuses fall back to opening the app.

Version bump: CFBundleShortVersionString 1.15 → 1.16 and CFBundleVersion 30 → 31
across all three targets in project.yml, with the Info.plists regenerated by
xcodegen. What's New — 1.16 is written in both locales; asc_submit.py parses
both and they are far under Apple's 4,000-character limit.

* [feature] the 注音 pane predicts by phrase, and the panes are tabs

Two lone 聲母 now offer the word: ㄋㄏ is 你好 before a vowel or tone is
typed, the way the system keyboard has it. A phrase table generated from
McBopomofo's BPMFMappings.txt (libtabe BSD readings, attributed in
THIRD-PARTY.md) is matched by prefix within each syllable and indexed by
the first symbol of the first two; phrases lead the bar, single
characters follow, and return commits a greedy longest-phrase walk. The
generator scores by corpus count plus character frequency and gives a
short list of conversational words a floor, because the corpus is news
and this keyboard types messages. Both tables warm on a background queue
when the 注音 pane becomes current.

The mode strip's dots become named tabs (語音 / 英文 / 注音): in use nobody
took the dots for a control. The swipe is unchanged. The mic chip loses
its minutes to make the row fit at 320pt. The composition chip hugs its
text instead of taking the whole cap.

Design doc, What's New (both locales) and the dictionary's doc comment
updated to match.

* [fix] candidates in the 注音 bar are separated, so two words stop reading as one

The bar now mixes one- and two-character candidates; at 2pt spacing
會出好處會場 read as a single string. Each candidate gets an 11pt gutter
and a hairline between neighbours, as the system keyboard spaces them.

* [refactor] the two 注音 generators share one data module

Sonar flagged the phrase generator for copying the dict generator's
download, parsing and validation helpers (14.6% duplicated lines), a
21-complexity parse function and two nested ternaries. The shared parts
move to scripts/zhuyin-data.mjs; both generators still write
byte-identical resources.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant