Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .agents/skills/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,6 @@ This directory contains specialized skills (recipes) to guide contributors and A
- [Add Native Extension](./add-native-extension/SKILL.md) — C++ operations and JSI bindings.
- [Add Task Pipeline](./add-task-pipeline/SKILL.md) — TypeScript task pipelines and React hooks.
- [Model Schema Validation](./model-schema-validation/SKILL.md) — Model specs, dynamic shapes, and schema validation.
- [Add API Tests](./add-api-tests/SKILL.md) — TypeScript API test suites and the fake native runtime.
- [Verify and Build](./verify-and-build/SKILL.md) — TypeScript typechecking, native rebuilding, and troubleshooting.
- [Skills Maintenance](./skills-maintenance/SKILL.md) — Keeping skills synchronized with core primitives.
136 changes: 136 additions & 0 deletions .agents/skills/add-api-tests/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
---
name: add-api-tests
description: Use when adding or changing anything under src/ — a task pipeline, a hook, a native op wrapper, a registry entry — and you need to cover it with the TypeScript API test suites.
metadata:
id: add_api_tests
scope: packages/react-native-executorch/__tests__/*
---

# Skill: Add TypeScript API Tests

Every change under `src/` belongs in the Jest suites at
[`packages/react-native-executorch/__tests__/`](../../../packages/react-native-executorch/__tests__/README.md).
They run on a laptop or a CI runner — no simulator, no device, no `.pte` — and
finish in a few seconds.

```bash
yarn workspace react-native-executorch test
yarn workspace react-native-executorch test __tests__/tasks # one directory
yarn workspace react-native-executorch test -u # update snapshots
```

Types come along for free: `yarn typecheck` already covers `__tests__/`.

---

## 🧩 The Fake Native Runtime

There is no stubbing of individual JSI calls. `__tests__/support/fakeJsi.ts`
implements the whole `__rnexecutorch_jsi__` contract in JavaScript — tensors
hold real data, `math`/`cv`/`speech` operators compute real values — so a task
pipeline runs end to end and its own logic is what the assertions measure.

A test describes the model it wants, then drives the real pipeline:

```typescript
import { f32, method } from '../../src/core/schema';
import { fakeJsi } from '../support/fakeJsi';
import { tracked } from '../support/lifetime';
import { STRETCH_PREPROCESSING, exported, imageBuffer, writesOutputs } from '../support/fixtures';

fakeJsi.registerModel('/models/task.pte', {
schema: exported(method('forward', [f32(1, 3, 4, 4)], [f32(1, 3)])),
execute: writesOutputs([1, 0, 2]),
});

const runner = tracked(await createMyTask({ modelPath: '/models/task.pte', modelOpts }));
expect(await runner.runTask(imageBuffer(8, 8))).toEqual(/* ... */);
```

Key helpers:

| Helper | Use |
| :--- | :--- |
| `fakeJsi.registerModel(path, program)` | Make `loadModel(path)` succeed with a given schema and `execute` |
| `fakeJsi.registerTokenizer(path, vocabulary)` | Same for `loadTokenizer` |
| `exported(spec)` | Reinterpret a spec built with `method`/`f32`/`i64` as an *exported* one (it verifies no symbolic dims are left) |
| `writesOutputs(...)`, `copiesInputToOutput()` | Ready-made `execute` implementations |
| `tracked(pipeline)` | Auto-dispose at the end of the test |
| `imageBuffer(w, h, format)` | A deterministic input image |
| `cachePathFor(url)` | Where the fetcher will download a URL, so a hook test can register its model up front |
| `fakeNet.serve(url, route)` | Script the server: status, body, `Range` support, and a `gate` to hold a download open |

---

## 🧠 What to Cover for a New Task Pipeline

1. **Schema acceptance** — one test per variant the pipeline declares
(`batched`, `unbatched`, ...), asserting the factory resolves.
2. **Schema rejection** — a model that matches no variant, asserting the error
names the mismatch (`Rank mismatch`, `inconsistent bindings`, ...). A caller
should learn what is wrong from the message.
3. **Configuration mismatch** — e.g. a `labels` array that disagrees with the
model's output dimension.
4. **Postprocessing** — the part that is yours: sorting, thresholding,
suppression, colormaps, coordinate scaling. Choose fixture values that make
the expected output obvious in the test.
5. **Options** — every default in `modelOpts`, and every per-call override.
6. **Disposal** — `dispose()` leaves `fakeJsi.liveTensors()` at 0 and
`fakeJsi.liveModels()` empty, and repeated calls do not accumulate scratch
tensors.
7. **Sync/async parity** — `runTaskWorklet(x)` equals `await runTask(x)`.

For a new hook, add a case to `__tests__/hooks/`: not-ready before the download
lands, methods exposed after, errors surfaced through the shared `error` field,
and every native handle released on unmount.

---

## 🔒 Leak Checking

Native memory is not garbage collected, so the setup file asserts after **every
test** that nothing allocated through the fake was left undisposed. That gives
each pipeline suite disposal coverage for free.

- Wrap construction in `tracked()` — it disposes at the end of the test and
stops a failing assertion from cascading into a second, misleading error.
- A test that deliberately leaks calls `allowNativeLeaks()` with a comment
saying why.

---

## 📐 Source-Level Conventions

`__tests__/api/workletDirective.test.ts` parses `src/` with the TypeScript
compiler and enforces the conventions no type can express:

- every exported function that calls into `rnexecutorchJsi` starts with
`'worklet';`
- no `async` function is marked as a worklet
- only `src/native/bridge.ts` names the `__rnexecutorch_jsi__` global
- `core/` never imports from `extensions/`, and `hooks/` never imports from
`native/`

If you add a new native wrapper without the directive, that suite fails — add
the directive rather than the exception.

---

## 📋 Verification Checklist

When adding or changing code under `src/`, verify that:

- [ ] `yarn workspace react-native-executorch test` passes.
- [ ] A new task pipeline has a suite covering acceptance, rejection,
postprocessing, options and disposal.
- [ ] A new hook has a lifecycle case in `__tests__/hooks/`.
- [ ] A new registry entry passes `__tests__/api/modelRegistry.test.ts` without
the rules being loosened (https URL, pinned revision,
`modelname_backend_precision.pte`, backend-matching folder, default
aliasing one of its own variants).
- [ ] A new export is reflected in the `api/apiSurface` snapshot, and the change
is intentional (a removal or rename is a breaking change).
- [ ] Any new fake behaviour in `__tests__/support/` is faithful where fidelity
changes an assertion, and its simplifications are commented.
- [ ] No test was made to pass by calling `allowNativeLeaks()` without an
explanation.
21 changes: 8 additions & 13 deletions .agents/skills/add-task-pipeline/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,28 +188,22 @@ Wrap the task pipeline in a custom React Hook using the core hooks `useResourceD

```typescript
import { useModel } from './useModel';
import { useResourceDownload } from './useResourceDownload';
import { useResourceDownload, type ResourceOptions } from './useResourceDownload';
import { createMyTask, type MyTaskModel } from '../extensions/<domain>/tasks/<task>';

export function useMyTask(config: MyTaskModel, options?: { preventLoad?: boolean }) {
// 1. Resolve remote or local asset model path and download progress
const { localPath, downloadProgress, downloadError } = useResourceDownload(
config.modelPath,
options?.preventLoad
);
export function useMyTask(config: MyTaskModel, options?: ResourceOptions) {
// 1. Resolve every remote URL in the config to a local path, with progress
// weighted across all of them.
const { resource, downloadProgress, downloadError } = useResourceDownload(config, options);

// 2. Instantiate and compile the task pipeline (with automatic lifecycle cleanup)
const { model, error } = useModel(
createMyTask,
localPath ? { ...config, modelPath: localPath } : null,
[localPath]
);
const { model, error } = useModel(createMyTask, resource ?? null);

return {
isReady: !!model,
error: downloadError || error,
downloadProgress,
localPath,
resource,
runTask: model?.runTask,
runTaskWorklet: model?.runTaskWorklet,
};
Expand All @@ -233,3 +227,4 @@ When adding a task pipeline or React hook, verify that:
- [ ] Every parameter is bucketed per Principle 6: varies across variants → option; fixed by the export → `const` in the task file; per-call choice → executor argument.
- [ ] No exposed option has exactly one valid value, and no two `models.ts` variants pass an identical options object.
- [ ] The React Hook utilizes `useModel` and properly returns progress, ready state, errors, and task execution bindings.
- [ ] The pipeline has a suite under `__tests__/tasks/` covering schema acceptance and rejection, postprocessing, every option, and full disposal — see the [Add API Tests skill](../add-api-tests/SKILL.md).
1 change: 1 addition & 0 deletions .agents/skills/core-guidelines/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ Use the following index to locate the specific procedural guides for your task:
| **Add a new native operator or C++ binding** | [SKILL.md](../add-native-extension/SKILL.md) | Procedural guide to implementing C++ functions, exposing them via JSI, and writing TypeScript bridge wrappers. |
| **Create a task pipeline or hook** | [SKILL.md](../add-task-pipeline/SKILL.md) | Guide to building end-to-end TS pipelines (e.g. object detection) and exposing them via React hooks. |
| **Verify, rebuild, or troubleshoot changes** | [SKILL.md](../verify-and-build/SKILL.md) | Workflows for rebuilding TS/C++ and resolving common JSI runtime errors. |
| **Test TypeScript changes** | [SKILL.md](../add-api-tests/SKILL.md) | Covering `src/` with the Jest API suites and their fake native runtime. |
| **Validate model constraints & schemas** | [SKILL.md](../model-schema-validation/SKILL.md) | Guide on specifying model specs, dynamic shapes, and runtime constraints for model validation. |
| **Maintain or refactor codebase patterns** | [SKILL.md](../skills-maintenance/SKILL.md) | Guide to keeping workspace skills in sync with codebase state to prevent documentation decay. |

Expand Down
1 change: 1 addition & 0 deletions .agents/skills/skills-maintenance/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Use this guide when you introduce, modify, or deprecate core codebase patterns,
- [add-task-pipeline](../add-task-pipeline/SKILL.md) for TypeScript pipeline orchestration, pre-allocation, and lifecycle hooks.
- [model-schema-validation](../model-schema-validation/SKILL.md) for schema verification constraints.
- [verify-and-build](../verify-and-build/SKILL.md) for compilation and troubleshooting steps.
- [add-api-tests](../add-api-tests/SKILL.md) for the TypeScript test suites and the fake native runtime.

3. **Verify Example Correctness**:
- Ensure all code blocks and examples in updated skills compile/work and match actual usage in the repository.
Expand Down
11 changes: 10 additions & 1 deletion .agents/skills/verify-and-build/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,14 @@ To check types and compile the TypeScript source code:
```bash
yarn typecheck
```
- **Run the TypeScript API Tests**:
```bash
yarn workspace react-native-executorch test
```
_Jest suites over the public `src/` surface — hooks, task pipelines, core
primitives, the fetcher and the model registry — running against a fake
native runtime, so they need no simulator, device or `.pte` file. See the
[Add API Tests skill](../add-api-tests/SKILL.md)._
- **Build Bundles**:
```bash
yarn prepare
Expand Down Expand Up @@ -172,7 +180,7 @@ This project does **not** bundle local `.pte` model files inside the React Nativ

## 🚫 Avoid / Anti-Patterns

- **Do NOT run code without verification:** Do not test TypeScript changes in the app without first running `yarn typecheck` (verify types) and `yarn prepare` (build target bundles).
- **Do NOT run code without verification:** Do not test TypeScript changes in the app without first running `yarn typecheck` (verify types), `yarn workspace react-native-executorch test` (API suites) and `yarn prepare` (build target bundles).
- **Do NOT skip native rebuilds after C++ edits:** If any C++ files or config bindings are added/modified, do not attempt to run the app without executing `pod install` (for iOS) or letting Gradle sync (for Android).
- **Do NOT run `lint:cpp` with the system `clang-tidy`**: Use the Homebrew LLVM binary: `CLANG_TIDY=$(brew --prefix llvm)/bin/clang-tidy yarn workspace react-native-executorch lint:cpp`.
- **Do NOT log complex objects inside worklets:** Avoid passing complex circular objects directly to `console.log()` inside functions annotated with `'worklet';` as it can hang or crash the worklet runtime thread.
Expand All @@ -185,6 +193,7 @@ This project does **not** bundle local `.pte` model files inside the React Nativ
When verifying or compiling your modifications, check that:

- [ ] TypeScript typechecking passes without errors (`yarn typecheck`).
- [ ] The TypeScript API tests pass (`yarn workspace react-native-executorch test`), and any new `src/` behavior is covered by them.
- [ ] Bundles compile successfully (`yarn prepare`).
- [ ] `pod install` has been run inside `apps/<domain-app>/ios/` after any native C++ edits.
- [ ] `lint:cpp` passes cleanly: `CLANG_TIDY=$(brew --prefix llvm)/bin/clang-tidy yarn workspace react-native-executorch lint:cpp`.
Expand Down
7 changes: 7 additions & 0 deletions .cspell-wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -314,3 +314,10 @@ Partitioner
denoised
ttfa
TTFA
letterboxed
macrotask
microtask
microtasks
unbatched
sdcard
dontMock
13 changes: 13 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,19 @@ module.exports = {
'no-console': 'warn',
},
},
{
// The JSDoc rules exist to keep the generated API reference complete.
// Test helpers are not part of that surface, and requiring a tag per
// parameter on a three-line fixture crowds out the prose that explains
// why the fixture exists.
files: ['packages/react-native-executorch/__tests__/**/*.{ts,tsx}'],
rules: {
'jsdoc/require-param': 'off',
'jsdoc/require-param-description': 'off',
'jsdoc/require-returns': 'off',
'jsdoc/require-returns-description': 'off',
},
},
{
files: ['**/*.md'],
processor: 'markdown/markdown',
Expand Down
18 changes: 18 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,21 @@ jobs:

- name: Build all packages
run: yarn workspaces foreach --all --exclude react-native-executorch-bare-resource-fetcher --exclude react-native-executorch-expo-resource-fetcher --exclude react-native-executorch-webrtc --topological-dev run prepare

api-tests:
name: TypeScript API tests
runs-on: ubuntu-latest
# A full run takes a few seconds; this only has to catch a hang.
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Setup
uses: ./.github/actions/setup

# No native libraries, no simulator, no `.pte` download — the suites run
# against the fake JSI runtime in `__tests__/support/`, so the setup
# action's TypeScript-only install is all this job needs.
- name: Run TypeScript API tests
run: yarn workspace react-native-executorch test --ci
97 changes: 97 additions & 0 deletions packages/react-native-executorch/__tests__/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# TypeScript API tests

Jest suites covering the public TypeScript surface under `src/` — the hooks,
the task pipelines, the core primitives, the resource fetcher and the model
registry. They run on a developer machine or a CI runner: no simulator, no
emulator, no device, no `.pte` file, and the whole run finishes in a few
seconds.

```bash
yarn workspace react-native-executorch test
yarn workspace react-native-executorch test --watch
yarn workspace react-native-executorch test __tests__/tasks # one directory
```

Types are checked by the existing `yarn typecheck`, which already covers this
directory.

## Why a fake native runtime, not stubs

Every path through `src/` bottoms out in `__rnexecutorch_jsi__`: a pipeline
allocates tensors, hands them to `model.execute`, and pushes them through
`softmax`, `resize` and `nms` on the way in and out. Stubbing those calls per
test would mean each assertion checks the stub rather than the pipeline — the
sorting in `classify`, the suppression in `detectObjects` and the colormap in
`segment` would all go untested.

So `support/fakeJsi.ts` implements the native contract in JavaScript instead:

| Piece | What it does |
| --- | --- |
| `support/fakeTensor.ts` | Typed-array-backed tensors with the real `setData`/`getData` byte semantics, `copyTo` windows, and use-after-dispose errors |
| `support/fakeOps.ts` | JS implementations of the `math`, `cv` and `speech` operators |
| `support/fakeJsi.ts` | `createTensor`, `loadModel`, `loadTokenizer`, and the resource trackers |
| `support/blobUtilMock.ts` | In-memory filesystem plus a programmable server (status, body, `Range` support, and a gate to hold a download open) |
| `support/workletsMock.ts` | Runs worklets inline — a worklet is an ordinary function marked for a second runtime |

A test describes the model it wants and drives the real pipeline over it:

```ts
fakeJsi.registerModel('/models/classifier.pte', {
schema: exported(method('forward', [f32(1, 3, 4, 4)], [f32(1, 3)])),
execute: writesOutputs([1, 0, 2]),
});

const classifier = tracked(await createClassifier(config));
expect((await classifier.classify(imageBuffer(8, 8))).map((r) => r.label))
.toEqual(['bird', 'cat', 'dog']);
```

The fake is faithful where fidelity changes an assertion — writing a float into
`uint8` storage rounds and clamps the way OpenCV's `saturate_cast` does, and the
tokenizer's methods are closures rather than prototype methods because a real
JSI host object's are — and deliberately simple elsewhere: `resize` is
nearest-neighbor whatever interpolation is asked for, and models the geometry
only. The numerical behavior of the real operators belongs to the C++ suites in
[`cpp/tests/`](../cpp/tests/README.md); what these suites own is the TypeScript
above them.

## Leak checking

Native memory is not garbage collected, so anything a test allocates through
the fake and does not dispose is a leak in the code under test. The setup file
asserts that after every test, so each pipeline suite gets disposal coverage
for free.

Wrap construction in `tracked()` and the harness disposes it at the end of the
test — which also keeps a failing assertion from cascading into a second,
misleading leak error. A test that means to leak calls `allowNativeLeaks()`.

## Layout

| Path | Contents |
| --- | --- |
| `core/` | `tensor`, `model`, `runtime`, and the `schema` spec matcher |
| `fetcher/` | `download` (caching, resume, cancellation, shared requests), telemetry, the Android backend |
| `tasks/` | One suite per task pipeline, plus the shared construction-failure behavior |
| `hooks/` | `useModel`, `useResourceDownload`, and the task hooks end to end |
| `extensions/` | The pure-TypeScript helpers: box/point scaling, seeded generators |
| `api/` | Export snapshot, model registry rules, label constants, source-level conventions |
| `support/` | The fake runtime, the mocks, and the fixtures |

## What is deliberately not covered

**Numerical behavior of the native operators.** `resize` interpolation,
`cvtColor` conversions and the exact `nms` arithmetic are the C++ suites' job;
duplicating them here would only test the fake.

**The long stateful worklets.** Whisper's decode loop, the VAD rolling window
and the SDXS diffusion step depend on real model weights, so faking them would
mostly assert against the fixture. What they do get is schema acceptance,
rejection of a mismatched model, and full disposal — including Whisper's nested
tokenizer and VAD pipeline.

**The thread hop.** Worklets run inline here, so serialization onto a real
worklet runtime is not exercised. The `'worklet'` directive convention that
makes that hop possible *is* checked, by parsing `src/` in
`api/workletDirective.test.ts`.
Loading