Skip to content

Update SpeechRecognitionResult with audio timing attributes #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alan33d wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update SpeechRecognitionResult with audio timing attributes #192

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
04a8f24
Update SpeechRecognitionResult with audio timing attributes
alan33d Jul 14, 2026
ad49253
Update audio segment time references in documentation
alan33d Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions index.bs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,8 @@ This does not preclude adding support for this as a future API enhancement, and
<li>The user agent may also give the user a longer explanation the first time speech input is used, to let the user know what it is and how they can tune their privacy settings to disable speech recording if required.</li>

<li>To mitigate the risk of fingerprinting, user agents MUST NOT personalize speech recognition when performing speech recognition on a {{MediaStreamTrack}}.</li>

<li>To mitigate fingerprinting vectors associated with high-precision timing, user agents MUST apply timestamp fuzzing and precision reduction to {{SpeechRecognitionResult/audioStartTime}} and {{SpeechRecognitionResult/audioEndTime}} before exposing these attributes to scripts (e.g. by rounding to 2ms precision).</li>

@padenot padenot Aug 13, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't a useful mitigation is it? Think about how you'd implement the same ASR system in WASM, will it be limited? It's absolutely trivial to implement audioStartTime etc. in WASM, and it will be sample-accurate w.r.t the audio stream.

</ol>

<h3 id="implementation-considerations">Implementation considerations</h3>
Expand Down Expand Up @@ -258,6 +260,8 @@ interface SpeechRecognitionResult {
readonly attribute unsigned long length;
getter SpeechRecognitionAlternative item(unsigned long index);
readonly attribute boolean isFinal;
readonly attribute DOMHighResTimeStamp? audioStartTime;
Comment thread
alan33d marked this conversation as resolved.
readonly attribute DOMHighResTimeStamp? audioEndTime;
};

// A collection of responses (used in continuous mode)
Expand Down Expand Up @@ -356,6 +360,16 @@ interface SpeechRecognitionPhrase {
</dd>
</dl>

<h4 id="speechrecoresult-attributes">SpeechRecognitionResult Attributes</h4>

<dl>
<dt><dfn attribute for=SpeechRecognitionResult>audioStartTime</dfn> attribute</dt>
<dd>A nullable {{DOMHighResTimeStamp}} representing the start of the audio segment corresponding to this recognition result, in milliseconds relative to the start of the audio stream. Returns null if the underlying recognition engine does not support audio segment start timestamps.</dd>

@padenot padenot Aug 13, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use seconds throughout those APIs, or microseconds in codec APIs. 64-bits floating point numbers have enough resolution here at even high sample rates, and it means we're not going to convert to/from seconds.

I don't think those should be nullalble. If your backend doesn't support timestamping, it can be implemented, or you can ship something decent (in the sense that it's going to be accurate enough for a lot of use case) in front of it.

We tend (not 100%) to not specificy APIs because of certain technologies limitations. Instead, we improve those technologies and write a good spec.

As discussed in the other PR, "relative to the start of the audio stream" is very fuzzy. How does it work if you get a MediaStream with a certain currentTime (internally), then fed into the Web Audio API (which has a different currentTime), then fed into an HTMLMediaElement (different currentTime), then captureStream(), and piped into SpeechRecognition. Which it is? The last one? Probably, and in that clock domain as well, which is important to mention because crossing clock domain makes alignment hard (which is what we want to do here).

Why is this not speechStartTime, mirroring speechStart (that also has a timestamp). What are the constraints between the two? It should be strictly equal probably?


<dt><dfn attribute for=SpeechRecognitionResult>audioEndTime</dfn> attribute</dt>
<dd>A nullable {{DOMHighResTimeStamp}} representing the end of the audio segment corresponding to this recognition result, in milliseconds relative to the start of the audio stream. Returns null if the underlying recognition engine does not support audio segment end timestamps.</dd>
</dl>

<p class=issue>The group has discussed whether WebRTC might be used to specify selection of audio sources and remote recognizers.
See <a href="https://lists.w3.org/Archives/Public/public-speech-api/2012Sep/0072.html">Interacting with WebRTC, the Web Audio API and other external sources</a> thread on public-speech-api@w3.org.</p>

Expand Down
Loading