Add privacy and security mitigations to the Web Speech API - #165
Open
evanbliu wants to merge 4 commits into
Open
Add privacy and security mitigations to the Web Speech API#165evanbliu wants to merge 4 commits into
evanbliu wants to merge 4 commits into
Conversation
…I to the Web Speech API
…I to the Web Speech API
4 tasks
Collaborator
Author
|
@padenot - I just realized this PR was never merged. Can you please take a look when you get a chance? |
…I to the Web Speech API
hoch
reviewed
Aug 25, 2026
- Fix formatting for `true` code span - Fix typo: 'an per-API' -> 'a per-API' - Remove redundant Alternate Options subsection
hoch
reviewed
Sep 11, 2026
hoch
left a comment
Member
There was a problem hiding this comment.
Mostly nits, but I do have some questions.
|
|
||
| The mitigation described in <a href="#on-device-model-privacy-availability-masking">Download Masking</a> works against attempts to silently fingerprint using {{SpeechRecognition/available()}}. The specification also contains requirements to prevent {{SpeechRecognition/install()}} from being easily used for fingerprinting, by introducing friction: | ||
|
|
||
| * The {{SpeechRecognition/install()}} method both requires and consumes [=user activation=], when it would initiate a download. |
Member
There was a problem hiding this comment.
I noticed we note here that install() requires and consumes user activation, but looking at the algorithm for SpeechRecognition.install() below, it doesn't seem to have steps checking or consuming transient activation yet.
What do you think about aligning the two? Would you prefer adding those steps to install() in this PR, or should we track that in a quick follow-up issue?
|
|
||
| To prevent cross-origin fingerprinting and site collusion, user agents <strong>must</strong> partition language pack availability by [=storage key=]. User agents <strong>must</strong> mask the current download status by returning {{"downloadable"}} from {{SpeechRecognition/available()}} by default, even if the actual underlying language pack is {{"available"}} or {{"downloading"}} on the user's device. | ||
|
|
||
| This masking state is established for each (API, options, [=storage key=]) tuple. The user agent <strong>must only</strong> set the state to "unmasked" for a specific [=storage key=] after a web page within that key explicitly calls {{SpeechRecognition/install()}} with a given set of options, and successfully starts a download or the promise resolves to `true` (indicating the language pack is ready). |
Member
There was a problem hiding this comment.
Since SpeechRecognition is the only interface managing model downloads here, do you think we still need API in this tuple? Wondering if (options, [=storage key=]) might be slightly more direct for readers.
|
|
||
| * The {{SpeechRecognition/install()}} method both requires and consumes [=user activation=], when it would initiate a download. | ||
| * The {{SpeechRecognition/install()}} method allows the user agent to prompt the user for permission, or to implicitly reject download attempts based on previous signals (such as an observed pattern of abuse). | ||
| * Access to {{SpeechRecognition/install()}} and {{SpeechRecognition/available()}} is gated on a per-API [=policy-controlled feature=], which means that only top-level origins and their delegates can use the API. |
Member
There was a problem hiding this comment.
Since both methods are specifically gated behind "on-device-speech-recognition", would it be clearer to name the feature directly here (e.g. "on-device-speech-recognition") rather than referring to a "per-API" feature?
|
|
||
| <strong>In the event of storage pressure, user agents should balance the utility of these APIs with the disk space they take up</strong>, possibly by having {{SpeechRecognition/install()}} resolve to `false` for new downloads or by freeing up disk space in other ways. However, user agents need to be mindful of the privacy impacts discussed in <a href="#on-device-model-privacy-availability-eviction">Download Eviction</a> when considering freeing up disk space by evicting language packs. <strong>User agents may involve the user in these decisions</strong>, e.g., via download-time prompts or a browser UI for managing downloaded language packs. | ||
|
|
||
| <strong>If a previously installed language pack is evicted (e.g., by the user or due to extreme storage pressure) while it might be in use or expected to be available, subsequent attempts to use it (e.g., via {{SpeechRecognition/start()}} with {{SpeechRecognition/lang}} set to that language and {{SpeechRecognition/processLocally}} as true) should fail gracefully. This might involve {{SpeechRecognition/available()}} returning {{"downloadable"}} or {{"unavailable"}}, and {{SpeechRecognition/start()}} potentially firing an {{SpeechRecognitionErrorEvent}} with an appropriate error code like {{SpeechRecognitionErrorCode/language-not-supported}} or {{SpeechRecognitionErrorCode/service-not-allowed}}.</strong> |
Member
There was a problem hiding this comment.
A couple of small phrasing nits here:
- Would
set to truebe clearer thanas truefor the boolean attribute? - Should this be
a {{SpeechRecognitionErrorEvent}}rather thanan?
|
|
||
| <strong>User agents should ensure that one web page's use of on-device speech recognition does not overly interfere with another web page's use of the API, or another web page's general operation, or the overall system stability.</strong> For example, it should not be possible for a background tab to monopolize speech processing resources, preventing a foreground tab from using them. | ||
|
|
||
| This specification does not mandate any particular mitigation strategy, but possible approaches include queuing requests, rate limiting, prioritizing foreground tabs, or detecting abusive behavior. <strong>If necessary to prevent resource exhaustion or instability, the user agent may cause speech recognition operations to fail (e.g., by firing an {{SpeechRecognitionErrorEvent}} with {{SpeechRecognitionErrorCode/service-not-allowed}}).</strong> |
Member
There was a problem hiding this comment.
Should this be a {{SpeechRecognitionErrorEvent}}?
|
|
||
| One implementation strategy for on-device speech recognition is to delegate to models or capabilities provided by the underlying operating system. This can offer benefits like a consistent user experience and efficient resource usage. | ||
|
|
||
| However, this approach comes with the usual considerations of exposing OS capabilities to the web. User agents must still ensure that all privacy and security requirements of this specification are met when using OS-provided models. This includes the requirements in <a href="#on-device-model-privacy-user-input">User Input and Speech Data</a> (preventing training on user data) and <a href="#on-device-model-security-runtime-resources">Runtime Shared Resources</a> (ensuring fair and stable resource sharing). |
Member
There was a problem hiding this comment.
Is this section non-normative? Otherwise I think we should use standard RFC 2119 uppercase MUST here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the privacy and security mitigations used by the Writing Assistance APIs to the on-device speech recognition part of the Web Speech API.
Closes #158
Preview | Diff