feat: respect AudioContextOptions.latencyHint when opening the Android output stream by DanielOrmeno · Pull Request #1232 · software-mansion/react-native-audio-api

DanielOrmeno · 2026-08-16T18:24:26Z

Summary

Implements AudioContextOptions.latencyHint (category form) and maps it to each backend's stream configuration, so playback-heavy apps can opt out of the underrun-prone low-latency stream on Android, and latency-sensitive apps can opt into a short buffer on iOS.

Closes #1231 — measurements there: on an entry-level Samsung the hard-coded LowLatency stream is granted 192 frames (4 ms) and misses ~210–230 render deadlines per second while an 11-source graph plays. The same graph on PerformanceMode::None plays clean.

Mapping

`latencyHint`	Android (Oboe `PerformanceMode`)	iOS (`AVAudioSession.preferredIOBufferDuration`)	Web
omitted	`LowLatency` — unchanged	not requested — unchanged	browser default
`'interactive'`	`LowLatency`	128 frames	forwarded
`'balanced'`	`None`	512 frames	forwarded
`'playback'`	`PowerSaving`	not requested — the default is already deep	forwarded

iOS is the mirror image of the Android problem: the backend never requested a buffer duration, so it ran deep and could not go low. That is why the two columns aren't symmetric, and why an omitted hint is not treated as interactive — doing so would drop every existing iOS playback app to a ~2.7 ms buffer. An unrecognised string is likewise treated as no hint, where a browser throws a TypeError.

On iOS the duration belongs to the process-wide session rather than to one stream, so live requests reconcile by taking the shortest; balanced and playback therefore lose to a concurrent interactive. Releasing the last request asks for what the session ran at beforehand, snapped to the nearest power-of-two frame count, because the session grants those and rounds a raw request down.

What changed

AudioContextLatencyHint in common/cpp/audioapi/core/types/ plus the hint→frames policy; latencyHintFromString sits in js_enum_parser with the other string→enum conversions.
Android's AudioPlayer maps the hint to a PerformanceMode when opening the stream.
iOS: AudioSessionManager reconciles buffer-duration requests per client; NativeAudioPlayer holds one while rendering and releases it on stop, suspend, cleanup and every failure path.
JSI createAudioContext gains an optional second string argument, in both the module installer and the WPT binding.
TS: AudioContextOptions.latencyHint?: AudioContextLatencyCategory, forwarded to the browser on web.
Docs: audio-context.mdx constructor table plus a per-platform table.

Non-breaking: every new constructor parameter is defaulted, and an omitted hint is today's behaviour on both platforms.

Out of scope, per #1231: numeric hints in seconds, which would map to setBufferSizeInFrames on Android.

Test plan

typecheck, lint:js, format:check (clang-format + ktlint), cpplint — clean.
test:js — 120/120, including a suite pinning that an omitted hint reaches native as omitted, on both the native and web paths.
common/cpp/test/run-tests.sh smoke — 428/428, including AudioContextLatencyHintTest over the parse table and the frames policy. Mutation-checked: remapping interactive, making playback or an absent hint request a buffer, and changing the balanced multiple each fail the suite.
build:ios (xcodebuild, iOS Simulator) — clean.
WPT audiocontextoptions.html run on this branch and on pristine main — 37/41 both, identical.
iOS behaviour on an iPhone 17 simulator via context.outputLatency: omitted 10.10 ms with no request issued, interactive 2.77 ms (128 frames granted, at 48 kHz and 44.1 kHz contexts), balanced 10.77 ms (512 frames), deep buffer restored after close.
Android validated earlier on the device from [Android] Hard-coded PerformanceMode::LowLatency causes continuous underruns (crackle) for multi-source playback on low-end devices #1231 via a backport to 0.13.2: interactive reproduces the underruns, balanced plays the 11-source session clean (AudioStream::getXRunCount() logging).
Not run on physical iPhone hardware; the wired and Bluetooth routes are unmeasured.

🤖 Generated with Claude Code

…d output stream AudioContext now accepts the Web Audio latencyHint category and the Android backend maps it to Oboe performance modes: interactive -> LowLatency (unchanged default), balanced -> None, playback -> PowerSaving. On web the hint is forwarded to the browser AudioContext; on iOS it is accepted but does not change the stream yet. Numeric hints are not supported yet. Motivation: the hard-coded LowLatency stream is granted a very small buffer (192 frames / 4 ms on an entry-level Samsung) and misses ~220 render deadlines per second when a multi-source graph renders, heard as continuous crackle. The same graph on a PerformanceMode::None stream plays clean. Measured via AudioStream::getXRunCount on device.

DanielOrmeno · 2026-09-16T02:22:37Z

Updated the branch against current main — the conflict was a single import line in globals.d.ts (AudioRecorderOptions landed on the line the hint's type was added to). No behaviour change; the rest of the diff is as it was. I merged rather than rebased so the PR history stays intact, but happy to rebase or squash if you'd prefer that.

Verified locally on the updated branch:

format:check, lint:js, typecheck — clean
yarn build && yarn test:js — 113/113
check-audio-enum-sync — in sync
C++ still compiled only by CI (no NDK in my environment)

CI hasn't run on this PR at all — the workflow runs are sitting at action_required, so I think it needs a maintainer to approve them. Could someone kick that off when convenient?

On next steps, I'd welcome any direction: whether the interactive/balanced/playback → Oboe mapping is the shape you want, and whether the two follow-ups noted in #1231 (numeric hints → setBufferSizeInFrames, iOS preferredIOBufferDuration) should become separate PRs.

One thing we ran into downstream that may deserve its own issue: on a Samsung SM-A176U1 with a USB audio interface, the hard-coded SharingMode::Exclusive output stream crashed the vendor HAL on start (requestStart → -899, then the AAudio service died); requesting Shared avoids it, and low-latency still goes through AAudio's MMAP path. Glad to file that separately with logs if it's useful.

Thanks, and no rush — just wanted to get the branch mergeable again.

mdydek · 2026-09-16T14:40:45Z

to be honest ideally we wanted to have also ios side implemented in this pr, we played around a little bit, but couldn't achieved any satisfying solution, that's why it was sitting idle for a while, also thank you for sharing that sometimes exclusive is also too much

DanielOrmeno · 2026-09-18T02:49:54Z

Makes total sense, I'm having a go at ios too, will update the PR accordingly.

The Android half of this PR had nothing to pair with on iOS because the backend never requested a buffer duration at all: playback ran at whatever AVAudioSession defaulted to for the category. iOS is the mirror image of the Android bug — there was no way to ask for LOW latency, so 'interactive' was unimplementable. An absent hint now stays absent rather than becoming INTERACTIVE, so each backend keeps the stream it opened before the hint existed. Android reads absence as LowLatency exactly as before; iOS leaves the session untouched, because dropping every existing playback app to a 2.7 ms buffer is how issue software-mansion#1231 would arrive on iOS. An unrecognised string is treated the same way, where a browser throws a TypeError. 'interactive' asks for one render quantum and 'balanced' for four, in frames of the SESSION's rate rather than the context's — asking in seconds made 'interactive' request 256 frames on a 44.1 kHz context driving a 48 kHz output, and at low context rates asked for a buffer deeper than the default it was meant to shorten. 'playback' asks for nothing, the session default being deep already. The duration belongs to the process-wide session rather than to one stream, so live requests are reconciled by taking the shortest, and releasing the last one asks for what the session ran at beforehand. That baseline is snapped to the nearest power-of-two frame count because the session grants those and rounds a raw request down: asking for a 23 ms baseline as-is was granted 11 ms, and rounding up alone asked for 43 ms. Verified on an iPhone 17 simulator by reading back context.outputLatency: omitted 10.10 ms with no request made, 'interactive' 2.77 ms (128 frames granted) at both 48 kHz and 44.1 kHz, 'balanced' 10.77 ms (512 frames), and the deep buffer restored once the interactive context closed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

DanielOrmeno · 2026-09-18T03:56:22Z

Pushed the iOS side.

Why it's shaped differently from the Android half: the iOS backend never called setPreferredIOBufferDuration at all, so output ran at whatever AVAudioSession defaulted to for the category (~21–23 ms). Android hard-codes low latency and can't get headroom; iOS ran deep and couldn't get low latency. So on iOS balanced and playback are close to no-ops, and interactive is the hint that was previously unimplementable.

That changed one thing from the original PR: an omitted hint now stays omitted (std::optional) rather than defaulting to INTERACTIVE. On Android that's identical to today. On iOS, treating omitted as interactive would drop every shipped playback app to a ~2.7 ms buffer — #1231 arriving on iOS. Unrecognised strings are treated the same way rather than throwing, which is a deliberate deviation from the spec's TypeError.

Measured on an iPhone 17 simulator via context.outputLatency: omitted 10.10 ms with no request issued, interactive 2.77 ms (128 frames granted), balanced 10.77 ms (512 frames), playback 10.77 ms, and the deep buffer restored after the interactive context closed.

Two platform details worth knowing:

iOS rounds a requested duration down to a power-of-two frame count — asking for a 23 ms baseline back was granted 11 ms — so the restore snaps to the nearest power of two.
The request has to be in frames, not seconds. Deriving it from the context's rate made interactive ask for 256 frames on a 44.1 kHz context driving 48 kHz output. The 2.77 ms above holds at both rates.

The hint→frames policy and the string parse moved into common/cpp so common/cpp/test can reach them; the platform TUs aren't in that target's glob, so neither mapping table had test coverage available before.

On WPT, I built the node addon and ran audiocontextoptions.html on this branch and on pristine main — 37/41 both, identical. interactive baseLatency == defaultLatency only passes because iOS getBaseLatency() returns one render quantum regardless of the granted buffer; fixing that looks right but flips the subtest, so it's not in here.

A few things I'd rather you decided:

createAudioRecorder takes one JSI object via AudioRecorderOptions::CreateFromJSIValue, while createAudioContext hand-decodes args[1] in two installers. An AudioContextOptions struct is probably the shape now there's a second field — happy to convert.
nextPowerOfTwo now exists twice under ios/audioapi (mine and NativeAudioRecorder.m's). Sharing it needs a new C-safe header since the recorder is plain ObjC, so I left it rather than touch that file.
AudioEngine already mints client handles in sourceRegistrations, and my reconciliation registry in AudioSessionManager is a second one. Folding the frame count onto the source registration may be the better seam.

Nothing has run on physical iPhone hardware — the simulator doesn't grant buffer durations the way a device does, and the wired and Bluetooth routes are unmeasured. Happy to do that before you merge.

CI hasn't run on this branch yet (still action_required).

github-actions · 2026-09-18T09:14:53Z

WPT non-regression comparison

PASS — no regressions · 1 improved section(s) · overall 2716 → 2717 (+1)

Spec section	Base pass	Head pass	Delta
AudioBufferSourceNode	217	218	+1

Unchanged sections (27)

Spec section	Base pass	Head pass
Processing model	0	0
Other	52	52
AnalyserNode	138	138
AudioBuffer	140	140
AudioContext	58	58
AudioNode	261	261
AudioParam	629	629
BiquadFilterNode	275	275
ChannelMergerNode	30	30
ChannelSplitterNode	7	7
ConstantSourceNode	64	64
ConvolverNode	203	203
DelayNode	104	104
DestinationNode	0	0
DynamicsCompressorNode	4	4
GainNode	15	15
IIRFilterNode	87	87
MediaElementAudioSourceNode	0	0
MediaStreamAudioDestinationNode	1	1
MediaStreamAudioSourceNode	0	0
OfflineAudioContext	35	35
OscillatorNode	104	104
PannerNode	75	75
PeriodicWave	33	33
ScriptProcessorNode	0	0
StereoPannerNode	102	102
WaveShaperNode	82	82

_{Baseline: 1966b4ac91efb990f1bc626fdb8899b034b770f4 · Candidate: 9790b7e9f59db67e2a3f2a3c4d55a24aed0c8737}

_{Workflow run · this comment is updated on every push.}

mdydek · 2026-09-18T15:38:42Z

ok few observations when playing around on your branch with various option for the latencies:

both android emulator and ios simulator do not properly mimic all hardware behavior, thus their reported data and values can differ from the physical devices
tested on the app, where the audio thread stalls were introduced once per 250 ms for x ms, to get to know real hardware buffer which serves as the protecting layer for the rendering spikes where the audio is rendered longer than usually (upper limits provided):
- android: interactive: 5ms, balanced: 70ms, playback: 70ms
- ios: interactive: 5ms, balanced (iobuffer as 8 x RQ): 20ms, playback (iobuffer as 32 x RQ): 80ms
on ios requesting greater amount of iobuffer than 4096 results in 1028

I think the approach is correct, need probably some adjustments f.e. ios buffer size constants, but overally respecting this implementation should provide a way for better ux of the audio

mdydek

ios side introduces much more lines than android and I think it could be simplified a little bit

mdydek · 2026-09-18T15:42:59Z

  std::atomic<int32_t> lastCallbackFrameCount_{0};
  std::mutex *driverMutex_;
  std::weak_ptr<AudioContext> context_;
+  std::optional<AudioContextLatencyHint> latencyHint_;


I don't like the idea that in one of the player the latency hint is its state, whereas in the other it is not

mdydek · 2026-09-18T15:44:17Z

+inline int preferredIOBufferFramesFor(std::optional<AudioContextLatencyHint> latencyHint) {
+  if (!latencyHint.has_value()) {
+    return 0;
+  }
+
+  switch (*latencyHint) {
+    case AudioContextLatencyHint::INTERACTIVE:
+      return RENDER_QUANTUM_SIZE;
+    case AudioContextLatencyHint::BALANCED:
+      return 4 * RENDER_QUANTUM_SIZE;
+    case AudioContextLatencyHint::PLAYBACK:
+      return 0;
+  }
+
+  return 0;
+}


android completely do not use this function, move it to the utils in the ios side

mdydek · 2026-09-18T15:46:27Z

+    case AudioContextLatencyHint::BALANCED:
+      return 4 * RENDER_QUANTUM_SIZE;
+    case AudioContextLatencyHint::PLAYBACK:
+      return 0;


Suggested change

case AudioContextLatencyHint::BALANCED:

return 4 * RENDER_QUANTUM_SIZE;

case AudioContextLatencyHint::PLAYBACK:

return 0;

case AudioContextLatencyHint::BALANCED:

return 8 * RENDER_QUANTUM_SIZE;

case AudioContextLatencyHint::PLAYBACK:

return 32 * RENDER_QUANTUM_SIZE;

my proposition for the values, 0 is treated as default (8 * RQS), we can enlarge it to 32 * RQS

mdydek · 2026-09-18T15:47:21Z

trivial test, remove it

mdydek · 2026-09-18T15:59:37Z

+    int frames = 0;
+    for (NSNumber *clientFrames in self.ioBufferFramesRequests.objectEnumerator) {
+      int clientRequest = [clientFrames intValue];
+      if (frames == 0 || clientRequest < frames) {
+        frames = clientRequest;
+      }
+    }
+
+    if (frames == 0 && self.baselineIOBufferDuration <= 0.0) {
+      return;
+    }
+
+    double sampleRate = self.audioSession.sampleRate;
+    if (sampleRate <= 0.0) {
+      return;
+    }


looks like kind off too much work for taking min from the dictionary and few returns

closetcaiman added feature New user-facing features or major capabilities web-api-spec Implementations, updates, or parity alignment with the W3C Web Audio API specification labels Aug 17, 2026

github-actions Bot added the conflicts This PR has conflicts with the base branch. label Sep 11, 2026

merge: resolve conflict with main (globals.d.ts import)

c5d0605

github-actions Bot removed the conflicts This PR has conflicts with the base branch. label Sep 16, 2026

Merge remote-tracking branch 'upstream/main' into ios-latency-hint

04deb7f

mdydek requested changes Sep 18, 2026

View reviewed changes

github-actions Bot added the conflicts This PR has conflicts with the base branch. label Sep 22, 2026

Navigation Menu

Uh oh!

feat: respect AudioContextOptions.latencyHint when opening the Android output stream - #1232

feat: respect AudioContextOptions.latencyHint when opening the Android output stream#1232
DanielOrmeno wants to merge 4 commits into
software-mansion:mainfrom
DanielOrmeno:feat/audio-context-latency-hint

DanielOrmeno commented Aug 16, 2026 •

edited

Loading

Uh oh!

DanielOrmeno commented Sep 16, 2026

Uh oh!

mdydek commented Sep 16, 2026

Uh oh!

DanielOrmeno commented Sep 18, 2026

Uh oh!

DanielOrmeno commented Sep 18, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

mdydek commented Sep 18, 2026 •

edited

Loading

Uh oh!

mdydek left a comment

Uh oh!

mdydek Sep 18, 2026

Uh oh!

mdydek Sep 18, 2026

Uh oh!

mdydek Sep 18, 2026

Uh oh!

mdydek Sep 18, 2026

Uh oh!

mdydek Sep 18, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

DanielOrmeno commented Aug 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Mapping

What changed

Test plan

Uh oh!

DanielOrmeno commented Sep 16, 2026

Uh oh!

mdydek commented Sep 16, 2026

Uh oh!

DanielOrmeno commented Sep 18, 2026

Uh oh!

DanielOrmeno commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 18, 2026

WPT non-regression comparison

Uh oh!

mdydek commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mdydek left a comment

Choose a reason for hiding this comment

Uh oh!

mdydek Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

mdydek Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

mdydek Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

mdydek Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

mdydek Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DanielOrmeno commented Aug 16, 2026 •

edited

Loading

DanielOrmeno commented Sep 18, 2026 •

edited

Loading

mdydek commented Sep 18, 2026 •

edited

Loading