Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .claude/skills/host-objects/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,8 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, getChannelData) {
}
```

The view aliases native memory for as long as JS keeps it, and the `shared_ptr` inside the `jsi::ArrayBuffer` keeps that memory alive on its own. When the native side later needs the view to stop aliasing (Web Audio's "acquire the content" on `AudioBufferSourceNode.start()`), it must retain the returned object and neutralise it afterwards — see `detachReturnedChannelData` in the real `AudioBufferHostObject`.

### External memory pressure

Call `setExternalMemoryPressure` whenever returning a HostObject or typed array that wraps a large native buffer. This lets the JS GC schedule collection correctly:
Expand Down
3 changes: 3 additions & 0 deletions packages/audiodocs/docs/fundamentals/best-practices.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ user experience, and maintainability. Here are some key best practices to consid

- **Scheduled source nodes are single-use**: [`AudioBufferSourceNode`](../sources/audio-buffer-source-node.mdx), [`OscillatorNode`](../sources/oscillator-node.mdx), and other [`AudioScheduledSourceNode`](../sources/audio-scheduled-source-node.mdx) subclasses can be [`start()`](../sources/audio-scheduled-source-node.mdx#start)ed only once. Create a new node to replay a sound, but reuse the same [`AudioBuffer`](../sources/audio-buffer.mdx) — nodes are inexpensive to create.

- **Seeking**: since a started `AudioBufferSourceNode` can't be repositioned, seeking means stopping/disconnecting it and creating a fresh node with the same `AudioBuffer` at a new `start()` offset. Reassigning the same buffer this way is cheap. The underlying PCM data is copied once per `AudioBuffer`, not once per node, so recreating the source node on every seek doesn't re-copy it.

- **Use [`AudioBufferQueueSourceNode`](../sources/audio-buffer-queue-source-node.mdx) for chunked playback**: When audio arrives in segments (streaming TTS, progressive download), enqueue buffers into a queue source node rather than recreating the entire graph per chunk.

## [**AudioParam**](../core/audio-param.mdx) changes
Expand Down Expand Up @@ -87,4 +89,5 @@ Prefer logging plain values instead:
```tsx
console.log({ channelCount: node.channelCount, numberOfInputs: node.numberOfInputs });
```

:::
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include <cstddef>
#include <memory>
#include <utility>
#include <vector>

namespace audioapi {

Expand All @@ -24,7 +25,43 @@ AudioBufferHostObject::AudioBufferHostObject(const std::shared_ptr<AudioBuffer>
}

AudioBufferHostObject::AudioBufferHostObject(AudioBufferHostObject &&other) noexcept
: HostObject(std::move(other)), audioBuffer_(std::move(other.audioBuffer_)) {}
: HostObject(std::move(other)),
audioBuffer_(std::move(other.audioBuffer_)),
immutableCopyCache_(std::move(other.immutableCopyCache_)),
returnedChannelDataArrays_(std::move(other.returnedChannelDataArrays_)) {}

void AudioBufferHostObject::detachReturnedChannelData(jsi::Runtime &runtime) {
if (returnedChannelDataArrays_.empty()) {
return;
}

auto defineProperty = runtime.global()
.getPropertyAsObject(runtime, "Object")
.getPropertyAsFunction(runtime, "defineProperty");
auto zeroDescriptor = jsi::Object(runtime);
zeroDescriptor.setProperty(runtime, "value", 0);

std::vector<bool> channelDetached(audioBuffer_->getNumberOfChannels(), false);

for (const auto &returned : returnedChannelDataArrays_) {
auto array = returned.array.lock(runtime);
if (array.isObject()) {
for (const auto *sizeProperty : {"length", "byteLength", "byteOffset"}) {
defineProperty.call(runtime, array, sizeProperty, zeroDescriptor);
}
}

if (!channelDetached[returned.channel]) {
audioBuffer_->detachSharedChannel(returned.channel);
channelDetached[returned.channel] = true;
}
}

returnedChannelDataArrays_.clear();
// A view could have been written through right up until now, i.e. after the cached
// copy was taken.
immutableCopyCache_.invalidate();
}

JSI_PROPERTY_GETTER_IMPL(AudioBufferHostObject, sampleRate) {
return {audioBuffer_->getSampleRate()};
Expand All @@ -43,14 +80,21 @@ JSI_PROPERTY_GETTER_IMPL(AudioBufferHostObject, numberOfChannels) {
}

JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, getChannelData) {
auto channel = static_cast<int>(args[0].getNumber());
// The returned Float32Array is a live, JS-writable view straight into
// audioBuffer_'s storage, so a copy cached before now can no longer be trusted.
// Caching resumes once the view is neutralised by detachReturnedChannelData().
immutableCopyCache_.invalidate();

auto channel = static_cast<size_t>(args[0].getNumber());
auto audioArrayBuffer = audioBuffer_->getSharedChannel(channel);
auto arrayBuffer = jsi::ArrayBuffer(runtime, audioArrayBuffer);

auto float32ArrayCtor = runtime.global().getPropertyAsFunction(runtime, "Float32Array");
auto float32Array = float32ArrayCtor.callAsConstructor(runtime, arrayBuffer).getObject(runtime);

float32Array.setExternalMemoryPressure(runtime, audioArrayBuffer->size());
returnedChannelDataArrays_.push_back(
{.channel = channel, .array = jsi::WeakObject(runtime, float32Array)});

return float32Array;
}
Expand All @@ -76,6 +120,9 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, copyFromChannel) {
}

JSI_HOST_FUNCTION_IMPL(AudioBufferHostObject, copyToChannel) {
// Mutates audioBuffer_ in place, so any previously cached copy is now stale.
immutableCopyCache_.invalidate();

auto arrayBuffer =
args[0].getObject(runtime).getPropertyAsObject(runtime, "buffer").getArrayBuffer(runtime);
auto *source = reinterpret_cast<float *>(arrayBuffer.data(runtime));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,13 @@

#include <audioapi/jsi/HostObject.h>
#include <audioapi/utils/AudioBuffer.hpp>
#include <audioapi/utils/ImmutableBufferCache.h>

#include <jsi/jsi.h>
#include <cstddef>
#include <memory>
#include <utility>
#include <vector>

namespace audioapi {
using namespace facebook;
Expand All @@ -23,6 +25,8 @@ class AudioBufferHostObject : public HostObject {
if (this != &other) {
HostObject::operator=(std::move(other));
audioBuffer_ = std::move(other.audioBuffer_);
immutableCopyCache_ = std::move(other.immutableCopyCache_);
returnedChannelDataArrays_ = std::move(other.returnedChannelDataArrays_);
}
return *this;
}
Expand All @@ -34,6 +38,33 @@ class AudioBufferHostObject : public HostObject {
return audioBuffer_->getSize() * audioBuffer_->getNumberOfChannels() * sizeof(float) * 2;
}

/// @brief Returns a defensive copy of `audioBuffer_` suitable for handing to an
/// `AudioBufferSourceNode`, reusing a cached copy across repeated `.buffer = x`
/// reassignments of this same JS-visible buffer (e.g. seeking, which recreates the
/// source node but keeps reusing the already-decoded buffer). Without this, every
/// reassignment allocated a brand-new full-size copy, which is where
/// https://github.com/software-mansion/react-native-audio-api/issues/1263 came from.
/// @note The cache is dropped whenever `audioBuffer_` may have diverged from it:
/// `copyToChannel` mutates in place, `getChannelData` hands out a live JS-writable
/// view, and `detachReturnedChannelData` is the last moment such a view could have
/// been written through.
[[nodiscard]] std::shared_ptr<AudioBuffer> getOrCreateImmutableCopy() {
return immutableCopyCache_.getOrCreate(audioBuffer_);
}

/// @brief Web Audio's "acquire the content" step for the views handed out by
/// `getChannelData`. Call once playback of this buffer has been scheduled. Every
/// previously returned Float32Array stops aliasing `audioBuffer_` and, if JS still
/// holds it, reads as zero-length; the next `getChannelData` call hands out a fresh
/// view, mirroring what a browser does when it detaches those ArrayBuffers.
void detachReturnedChannelData(jsi::Runtime &runtime);

/// @brief Whether any `getChannelData` view is live, i.e. handed out since the last
/// `detachReturnedChannelData`.
[[nodiscard]] bool hasReturnedChannelData() const {
return !returnedChannelDataArrays_.empty();
}

JSI_PROPERTY_GETTER_DECL(sampleRate);
JSI_PROPERTY_GETTER_DECL(length);
JSI_PROPERTY_GETTER_DECL(duration);
Expand All @@ -42,5 +73,16 @@ class AudioBufferHostObject : public HostObject {
JSI_HOST_FUNCTION_DECL(getChannelData);
JSI_HOST_FUNCTION_DECL(copyFromChannel);
JSI_HOST_FUNCTION_DECL(copyToChannel);

private:
struct ReturnedChannelDataArray {
size_t channel;
jsi::WeakObject array;
};

utils::ImmutableBufferCache immutableCopyCache_;
/// Float32Array views handed out by `getChannelData` since the last
/// `detachReturnedChannelData`, kept so they can be neutralised then.
std::vector<ReturnedChannelDataArray> returnedChannelDataArrays_;
};
} // namespace audioapi
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,9 @@ JSI_PROPERTY_SETTER_IMPL(AudioBufferSourceNodeHostObject, onLoopEnded) {
}

JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, start) {
hasBeenStarted_ = true;
acquireBufferContent(runtime);

auto handle = node_->handle;
auto event = [handle,
node = audioBufferSourceNode_,
Expand All @@ -139,6 +142,22 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, start) {
return jsi::Value::undefined();
}

void AudioBufferSourceNodeHostObject::acquireBufferContent(jsi::Runtime &runtime) {
if (bufferHostObject_ == nullptr || !bufferHostObject_->hasReturnedChannelData()) {
return;
}

bufferHostObject_->detachReturnedChannelData(runtime);
// The copy handed to the node in setBuffer() predates any writes made through those
// views since, so hand it the post-write content that has now been fenced off.
auto buffers = prepareNodeBuffers(bufferHostObject_->audioBuffer_, bufferHostObject_);
auto event =
[handle = node_->handle, node = audioBufferSourceNode_, buffers](BaseAudioContext &) {
node->replaceBufferContent(buffers.copiedBuffer, buffers.audioBuffer);
};
audioBufferSourceNode_->scheduleAudioEvent(std::move(event));
}

JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, setBuffer) {
if (args[0].isNull()) {
setBuffer(nullptr);
Expand All @@ -147,56 +166,76 @@ JSI_HOST_FUNCTION_IMPL(AudioBufferSourceNodeHostObject, setBuffer) {
thisValue.asObject(runtime).setExternalMemoryPressure(
runtime, getMemoryPressure() + bufferHostObject->getSizeInBytes());

setBuffer(bufferHostObject->audioBuffer_);
setBuffer(bufferHostObject->audioBuffer_, bufferHostObject);
}

// Per Web Audio, assigning a buffer to an already-started source acquires its
// content right away, because start() had nothing to acquire back then.
if (hasBeenStarted_) {
acquireBufferContent(runtime);
}

return jsi::Value::undefined();
}

void AudioBufferSourceNodeHostObject::setBuffer(const std::shared_ptr<AudioBuffer> &buffer) {
AudioBufferSourceNodeHostObject::NodeBuffers AudioBufferSourceNodeHostObject::prepareNodeBuffers(
const std::shared_ptr<AudioBuffer> &buffer,
const std::shared_ptr<AudioBufferHostObject> &bufferHostObject) {
// TODO: add optimized memory management for buffer changes, e.g.
// when the same buffer is reused across threads and
// buffer modification is not allowed on JS thread
auto handle = node_->handle;

std::shared_ptr<AudioBuffer> copiedBuffer;
std::shared_ptr<DSPAudioBuffer> audioBuffer;
const size_t newChannelCount = buffer == nullptr ? AudioBufferSourceOptions::kDefaultChannelCount
: buffer->getNumberOfChannels();
NodeBuffers buffers;

if (buffer == nullptr) {
copiedBuffer = nullptr;
audioBuffer = std::make_shared<DSPAudioBuffer>(
buffers.copiedBuffer = nullptr;
buffers.audioBuffer = std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE,
AudioBufferSourceOptions::kDefaultChannelCount,
audioBufferSourceNode_->getContextSampleRate());
return buffers;
}

if (pitchCorrection_) {
initStretch(static_cast<int>(buffer->getNumberOfChannels()), buffer->getSampleRate());
auto extraTailFrames =
static_cast<size_t>((inputLatency_ + outputLatency_) * buffer->getSampleRate());
size_t totalSize = buffer->getSize() + extraTailFrames;
buffers.copiedBuffer = std::make_shared<AudioBuffer>(
totalSize, buffer->getNumberOfChannels(), buffer->getSampleRate());
buffers.copiedBuffer->copy(*buffer, 0, 0, buffer->getSize());
buffers.copiedBuffer->zero(buffer->getSize(), extraTailFrames);
} else if (bufferHostObject != nullptr) {
// Reuse a cached copy across repeated `.buffer = x` reassignments of the same
// JS-visible buffer (e.g. seeking, which recreates the source node but keeps
// reusing the already-decoded buffer) instead of deep-copying every time.
// See https://github.com/software-mansion/react-native-audio-api/issues/1263.
buffers.copiedBuffer = bufferHostObject->getOrCreateImmutableCopy();
} else {
if (pitchCorrection_) {
initStretch(static_cast<int>(buffer->getNumberOfChannels()), buffer->getSampleRate());
auto extraTailFrames =
static_cast<size_t>((inputLatency_ + outputLatency_) * buffer->getSampleRate());
size_t totalSize = buffer->getSize() + extraTailFrames;
copiedBuffer = std::make_shared<AudioBuffer>(
totalSize, buffer->getNumberOfChannels(), buffer->getSampleRate());
copiedBuffer->copy(*buffer, 0, 0, buffer->getSize());
copiedBuffer->zero(buffer->getSize(), extraTailFrames);
} else {
copiedBuffer = std::make_shared<AudioBuffer>(*buffer);
}

audioBuffer = std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE,
copiedBuffer->getNumberOfChannels(),
audioBufferSourceNode_->getContextSampleRate());
buffers.copiedBuffer = std::make_shared<AudioBuffer>(*buffer);
}

buffers.audioBuffer = std::make_shared<DSPAudioBuffer>(
RENDER_QUANTUM_SIZE,
buffers.copiedBuffer->getNumberOfChannels(),
audioBufferSourceNode_->getContextSampleRate());
return buffers;
}

void AudioBufferSourceNodeHostObject::setBuffer(
const std::shared_ptr<AudioBuffer> &buffer,
const std::shared_ptr<AudioBufferHostObject> &bufferHostObject) {
bufferHostObject_ = bufferHostObject;
auto buffers = prepareNodeBuffers(buffer, bufferHostObject);

// Update channelCount on the host thread before renegotiation so MAX /
// CLAMPED_MAX downstream nodes see the new width immediately.
const size_t newChannelCount = buffer == nullptr ? AudioBufferSourceOptions::kDefaultChannelCount
: buffer->getNumberOfChannels();
updateChannelCount(newChannelCount);

auto event =
[handle, node = audioBufferSourceNode_, copiedBuffer, audioBuffer](BaseAudioContext &) {
node->setBuffer(copiedBuffer, audioBuffer);
[handle = node_->handle, node = audioBufferSourceNode_, buffers](BaseAudioContext &) {
node->setBuffer(buffers.copiedBuffer, buffers.audioBuffer);
};
audioBufferSourceNode_->scheduleAudioEvent(std::move(event));
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,29 @@ class AudioBufferSourceNodeHostObject : public AudioBufferBaseSourceNodeHostObje
double loopStart_;
double loopEnd_;

void setBuffer(const std::shared_ptr<AudioBuffer> &buffer);
/// The JS-visible buffer behind the last `setBuffer`, kept so the "acquire the
/// content" step can run on it. Null when the buffer came from options or was cleared.
std::shared_ptr<AudioBufferHostObject> bufferHostObject_;
bool hasBeenStarted_ = false;

struct NodeBuffers {
std::shared_ptr<AudioBuffer> copiedBuffer;
std::shared_ptr<DSPAudioBuffer> audioBuffer;
};

NodeBuffers prepareNodeBuffers(
const std::shared_ptr<AudioBuffer> &buffer,
const std::shared_ptr<AudioBufferHostObject> &bufferHostObject);

void setBuffer(
const std::shared_ptr<AudioBuffer> &buffer,
const std::shared_ptr<AudioBufferHostObject> &bufferHostObject = nullptr);

/// Web Audio's "acquire the content" step: runs on start() when a buffer is set, and on
/// setBuffer() once already started. Cuts off every live getChannelData() view and
/// re-hands the node the fenced-off content.
/// https://webaudio.github.io/web-audio-api/#acquire-the-content
void acquireBufferContent(jsi::Runtime &runtime);
};

} // namespace audioapi
Loading