Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 11 additions & 220 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,220 +1,11 @@
# AGENTS.md — audiodsp

CircuitPython's audio system (`audiocore`, `synthio`, `audiomixer`, effects,
`audiomp3`), ported to MicroPython as `USER_C_MODULES` usermods. Import
names match CircuitPython exactly (`import audiocore`, `import synthio`, …)
for source compatibility; only this repo's own name differs.

## Layout

- Root: `micropython.mk` (unix/windows, Make-based ports), `micropython.cmake`
(esp32/rp2, CMake-based ports) — build glue for `USER_C_MODULES` discovery
- `src/` — one directory per module (`audiocore/`, `synthio/`, `audiomixer/`,
`audiospeed/`, `audiofreeverb/`, `audiofilters/`, `audiodelays/`,
`audiomp3/`, `audiodynamics/`, `audioroute/`, `audiomath/`, `audioecho/`,
`audioconvolve/`, `audiobiquad/`, `audioshaper/`, `audioladder/`,
`audiooscillator/`,
`audioverb/`, `audiomodal/`, `audiosampler/`, `audiovorbis/`), plus
`src/cp_compat/`
(CircuitPython-only core primitives ported as standalone compat shims, each
individually verified against mainline MicroPython before use — not assumed
missing) and `src/shared/` (runtime-neutral DSP the MicroPython usermod and
the CPython extension both compile)
- `src/cpython/` — the whole CPython target: `_audiodsp.c`, the extension built
in place, and the thirteen modules that wrap it (`audiocore.py`,
`synthio.py`,
…). They install as top-level modules, so it is `import audiocore` no matter
which of the three runtimes is underneath. Nothing puts this directory on
`sys.path`: audiodsp is a dependency, imported from wherever it is installed.
- `lib/` — the pure-Python tier: `lib/audiorender/` (whole-composition
offline rendering — numpy, desktop-only, never frozen) and
`lib/audiodsp_util/` (`float32`, the round trip that makes a setting derived
in Python the same number on a board as on a desktop — see
[docs/how-it-works/correctness-standard.md](docs/how-it-works/correctness-standard.md)). Both ship
inside the `omnipython-audiodsp` wheel. The instrument and effect libraries that
used to sit beside it — `audioinstruments` (55 `synthio` instruments) and
`audioeffects` (45 effect classes, racks included) — live in
[audiocomponents](https://github.com/PyDevices/audiocomponents) now, as
their own distributions depending on `omnipython-audiodsp`; nothing in this
repository builds, tests, publishes or freezes them.
- `circuitpython.mk` + `src/circuitpython_spike/` — build `audiodynamics`,
`audioroute`, `audiomath`, `audioecho`, `audioshaper`, `audioladder`,
`audioconvolve`, `audiobiquad`, `audioverb`, `audiomodal`, `audiosampler`,
`audiooscillator`, `audiovorbis`, `audiopump` and `audiograph` into
CircuitPython 11 as a user C module (`micropython.mk` hands a CircuitPython
tree to `circuitpython.mk`). None of the fifteen is a CircuitPython port: the
first two come from micropython-vst3's `vstaudio` engine and the last
thirteen are audiodsp's own,
so CircuitPython gains them
here rather than the other way round.
- `docs/porting-plan.md` — the full phased porting history, architecture,
and target layout
- `docs/how-it-works/upstream-sync.md` — when and how to resync the CircuitPython-derived
modules with upstream, what the last resync took, and the list of every
deliberate difference (the pump, `deinit`, the recorded deviations)
- `docs/how-it-works/upstream-diff.md` — the reasons and measurements behind each
deviation from upstream CircuitPython (verbatim-kept quirks vs. genuine port
bugs found and fixed)
- `tests/parity/` — oracle-diff scripts, run unchanged against both this
port and `bin/circuitpython`
- `tests/vendor/` — vendored third-party test fixtures (e.g. `synthtools`)

- `third_party/tremor/` — Xiph's Tremor (lowmem), the integer Vorbis decoder
`audiovorbis` wraps, BSD-3-Clause, vendored (see its `ORIGIN.txt`). Never
listed in a build on its own: `src/shared/audiodsp_vorbis.c` includes it
into one translation unit with every symbol renamed. See
[docs/reference/vorbis.md](docs/reference/vorbis.md).

## Sibling dependencies (cloned, not vendored)

- `mp3` (upstream `adafruit/Adafruit_MP3`, the Helix MP3 decoder core
`audiomp3` wraps) — RPSL 1.0/RCSL 1.0 licensed, *not* MIT; carried
unmodified per upstream's own terms, same as CircuitPython itself. Kept
as a separate sibling clone rather than folded into this (MIT) repo's own
tree.

Both are expected under `.deps/` (`scripts/fetch_deps.sh`), or as siblings
of this repository in the parent workspace, same pattern as
`pygraphics`/`displayif`.

## Testing

- Oracle diffing is the core discipline: every parity script under
`tests/parity/` runs unchanged against this port and `bin/circuitpython`,
rendering PCM and diffing byte-for-byte (or documenting the exact,
bounded exception in `docs/how-it-works/upstream-diff.md`).
- The nine modules that are **ours** have no oracle, by decision
([docs/how-it-works/correctness-standard.md](docs/how-it-works/correctness-standard.md), 2026-09-09).
Upstream CircuitPython has no counterpart to them, so they are held to two
things: every target renders them identically, and their numeric traits hold.
`.venv/bin/python tests/parity/verify_dsp.py --micropython ../micropython/ports/unix/build-pydevices/micropython`
is the first. There is **no stored digest** — the gate is the comparison, and
it **refuses a run with fewer than two interpreters** rather than passing one
that cannot fail. Add `--circuitpython ../bin/circuitpython-oracle-11.0.0-alpha.2`
for the three-way. The arithmetic is all in `src/shared/`, so two
interpreters disagreeing is never a difference of intent: it is a width, an
undefined shift, a compiler's choice or an architecture.
- The traits are the other half, one file per module —
`tests/test_cpython_<module>.py`, each opening with its trait table and the
bar for every row. Agreement cannot see a change that moves all three targets
together; the traits can, because they are measured against arithmetic rather
than against our own last answer. `audiobiquad`'s probe also prints two
invariants as integers beside its PCM (the block at which a tail reaches
exact zero, and the null depth at each feedback value), because a comparison
over PCM alone would not say whether either still held.
- **The vstaudio oracle is retired.** `audiodynamics` and `audioroute` came
from micropython-vst3's engine and used to be held to `vstaudio_dsp.c`
compiled unmodified. That file was deleted from micropython-vst3 in `6ea60d3`
and the plug-in links audiodsp now: the relationship reversed, so the engine is
a consumer of this package rather than a grader of it. The build script, the
usermod and `golden/dsp_nodes.json` are gone.
- The instruments parity gate — `run_instruments_parity.py`, its two probes,
`instrument_sequences.py` and the `instruments_*.json` digests — lives in
[audiocomponents](https://github.com/PyDevices/audiocomponents) now, under
its `tests/parity/`, beside the packages it renders. Its `REBUILT` rule (a
name there records a sound changed **on purpose**, and adding one is
the maintainer's call, never an agent's) is documented in that repository's
AGENTS.md. It is not run from here.
- `python3 -m flake8` is the lint gate (`.flake8`, defect checks only —
layout is deliberately not gated). It runs in CI on every push.
- Full regression after any change: rebuild interpreters
(`build_interpreters.sh` in the parent workspace), run the tier 0-5
parity suite plus `tests/parity/synthtools_acceptance.py`, and the LVGL
smoke test.
- See `docs/porting-plan.md`'s "Testing strategy" section for the complete
methodology.
- **The parity/oracle gates are workspace-local by design.** They need
built interpreters (`bin/circuitpython`, the workspace MicroPython) and
golden captures that live outside this repository, so external
contributors cannot run them and CI does not try. **Since 2026-09-03 the
scripts' zero-argument defaults no longer point into the workspace** — a
standalone user should not be steered into a directory only this
workspace has — so in *this* workspace every parity command needs the
explicit `--micropython`/`--circuitpython` (or `CP_DIR`, `MP_UNIX`,
`ULAB_DIR`) override shown above. `verify_dsp.py` used to print `skipping
micropython (not built at ...)` and carry on with whatever was left, which
was not a pass; it now **refuses** a run with fewer than two interpreters,
because one interpreter cannot disagree with itself and a run that cannot
fail is worse than no run. What CI
covers instead is what needs only the wheel: the CPython fixture tests in
`tests/test_cpython_*.py` and the five in-repo parity gates
(`verify_acceptance`, `verify_effects`, `verify_streaming`,
`verify_biquad`, `verify_mixdown_knee`), whose goldens are committed
here. The fifth (2026-09-06, audiodsp#27) is the only one whose material
crosses the mix-down limiter's +/-28000 knee, so it is the only one that
can see a synthio voice-ceiling change at all — the other four are
byte-identical at every ceiling value. It sees exactly one of the
ceiling's five sites, `src/cpython/synthio.py`'s `max_polyphony`, because
that is the only one the CPython target reads; `tests/
test_voice_ceiling_consistency.py` is still the guard for a half-applied
change. Its above-knee half is enforced against **this port**, not
against the oracle: the port is built at 64 and the pinned oracle at 14,
and no above-knee material can agree across those two. The component
contract tests (`test_audio_component_api`, `test_metadata_contract`,
`tools/validate_api.py`) went to audiocomponents with the packages they
check.
- **Two kinds of golden, two rules (2026-09-03).** The instruments
digests — `instruments_*.json`, now under audiocomponents'
`tests/parity/golden/` — record that *the port matches the pre-rewrite
original script, within one interpreter*: `run_instruments_parity.py`
there renders the originals from micropython-vst3 at `DEFAULT_OLD_REV` and
never consults the CircuitPython oracle. So a change to `src/cpython/`
*here* that is *right* still stales those cpython digests (audiodsp#25:
`b420dac` did exactly this). Rule: a CPython-target fix may re-capture the
affected cpython digests **only if it carries independent evidence against
the built oracle** — a test in `tests/test_cpython_*.py` run against
`bin/circuitpython` and cited in the message. The digests living in the
other repository changes only the mechanics: the audiocomponents
re-capture names the audiodsp commit that carries that evidence. A fix that
merely *asserts* oracle intent does not qualify; that would let it rewrite
its own reference. The accuracy program's *listening* goldens
(audiocomponents) are a different authority — a listener's ear — and move only
at his phrase. The stored digest is also the only thing in that gate that
notices the engine moving under both original and port — which is why it
stays: a live original-vs-port comparison was measured, approved and then
reversed the same day (audiodsp#26), because it would have stayed green
through `b420dac`. Each alarm costs one adjudication; that is the price of
the signal.

## The CircuitPython oracle — extend, never modify

CircuitPython at the tag in `CIRCUITPYTHON_ORACLE` is the **oracle** every
parity golden is measured against. The rule, for any agent working here:

- **The oracle binary is `bin/circuitpython-oracle-<version>` beside the sibling checkouts**,
built only by the interpreter build script's `--only cp-oracle` (CircuitPython's unix
coverage variant at `CIRCUITPY_SYNTHIO_MAX_CHANNELS=64`, with this repository as
its only audio user C module, through micropython-pydevices' `build_mp.py
--interpreter circuitpython`) and re-pinned in
`tests/test_voice_ceiling_consistency.py` in the same change that builds it.
**`bin/circuitpython` is not the oracle** — it is what that script's
`cp-unix` target installs, at the coverage variant's own 14-voice ceiling,
and it changes under you whenever anyone refreshes the interpreters
(audiodsp#89, twice in eight days).
- **Never edit CircuitPython's own sources for the oracle.** A modified oracle
silently redefines what "parity" means and invalidates every golden without
failing anything. Its *pin* is a different matter: it moves when this port
moves to a new CircuitPython release, deliberately, in a change that re-reads
every CP-shared gate and re-pins the binary's hash with the reason written
down. **Every pin move is a resync**: take every audio fix and every
addition (a new property, argument or class) upstream made since the old
pin, following `docs/how-it-works/upstream-sync.md`, and update its record of what was
taken and what differs on purpose. The oracle is built at the same version and the same ceiling this port
ships — an oracle at a different configuration cannot answer the only question
worth asking. See `docs/how-it-works/correctness-standard.md` and
`tests/test_voice_ceiling_consistency.py`.
- **Extending CircuitPython is fine and is the designed path**: new modules
live in this repo under `src/circuitpython_spike/` and build, with
`src/shared/` DSP, as a user C module (`circuitpython.mk`). That is
**additive-only by design**: it adds modules CircuitPython lacks and touches
none of its own. Never make it shadow or rewrite a CircuitPython module,
because the oracle is built with it, so a behavioral change would leak
*into* the oracle.
- Fixes to bugs that also exist upstream go in **this repo's targets only**
(MicroPython/CPython/`src/shared/`), recorded in `docs/how-it-works/upstream-diff.md`
— never into the CP tree. Approved deviations from the oracle are
enumerated there; **ask before adding one**.
- Quick self-check after any CP-adjacent work: `circuitpython.mk` compiles
only the ten modules above and `src/shared/`, never a file under
CircuitPython's own `shared-module/` or `shared-bindings/` for `synthio`,
`audiofilters`, `audiocore`, `audiomixer` or `audiodelays`.
# AGENTS.md

- CPython tests need the native extension built (`python -m build`, then install the wheel, or an editable install). Run the module you changed: `python -m unittest tests/test_cpython_<module>.py`. Don't run the whole suite for a small change; it is long.
- The five gates CI runs: `python tests/parity/verify_acceptance.py`, `verify_effects.py`, `verify_streaming.py`, `verify_biquad.py` and `verify_mixdown_knee.py`. Lint is `python -m flake8` (defect checks only).
- The deeper parity gates need built interpreters and the pinned CircuitPython oracle. Pass `--micropython` and `--circuitpython` explicitly; `verify_dsp.py` refuses a run with fewer than two interpreters. A stock `circuitpython` build is not the oracle (it has 14 voices; the oracle has 64). See [Running the parity gates](docs/how-it-works/parity-gates.md).
- Never edit CircuitPython's own sources for the oracle, and never let a new module shadow one of CircuitPython's. Fixes for bugs that also exist upstream go in this repo's targets only, recorded in [upstream-diff](docs/how-it-works/upstream-diff.md); ask before adding a deviation.
- Don't re-capture a golden, or move `CIRCUITPYTHON_ORACLE` or `DEPENDENCIES.lock`, without independent evidence and the reason written down ([CONTRIBUTING.md](CONTRIBUTING.md#rules-that-keep-the-port-honest)).
- The `audiomp3` decoder is fetched, not vendored: `scripts/fetch_deps.sh` puts it under `.deps/`. Tremor (`audiovorbis`) is vendored in `third_party/tremor`.
- Import names match CircuitPython (`import audiocore`, `import synthio`), whichever runtime is underneath. The instrument and effect libraries live in [audiocomponents](https://github.com/PyDevices/audiocomponents), not here.
- A frozen module wins over `/lib` on MicroPython and CircuitPython, so a copied or mip-installed `audiodsp` never runs until you rebuild or put `/lib` first. On a source checkout set `PYTHONPATH` (CPython) or `MICROPYPATH` (MicroPython).
- What each module is held to: [the correctness standard](docs/how-it-works/correctness-standard.md). Where everything is: [How audiodsp works](docs/how-it-works/index.md).
6 changes: 3 additions & 3 deletions CIRCUITPYTHON_ORACLE
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# 10.3.0 on 2026-09-09).
# Moving it is a re-port, not a bump: every move is a resync with upstream
# (docs/how-it-works/upstream-sync.md; the one before this was audiodsp#220). See also
# AGENTS.md "The CircuitPython oracle" and docs/how-it-works/upstream-diff.md. The 64-voice oracle binary built from
# it lives at bin/circuitpython-oracle-11.0.0-alpha.2 (build_interpreters.sh
# cp-oracle); tests/test_voice_ceiling_consistency.py pins its sha256.
# docs/how-it-works/parity-gates.md (how the 64-voice oracle binary is built)
# and docs/how-it-works/upstream-diff.md; tests/test_voice_ceiling_consistency.py
# pins the binary's sha256.
11.0.0-alpha.2 d53b2277cfad71589028e5da77c984c045db990d
1 change: 1 addition & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
@AGENTS.md
1 change: 1 addition & 0 deletions docs/how-it-works/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,7 @@ push-PCM transports.
- [Additions beyond CircuitPython](additions.md): why each added module exists.
- [Sound stability](sound-stability.md): what is stable across releases, and what isn't.
- [The correctness standard](correctness-standard.md): how each module is held to a reference.
- [Running the parity gates](parity-gates.md): the oracle binary, the deeper gates and the golden rules.
- [Keeping in step with upstream](upstream-sync.md): the resync procedure, and every deliberate difference.
- [Differences from upstream, with measurements](upstream-diff.md): the reasons behind each one.
- [Upstream reports](../upstream-reports/README.md): the fixes sent to CircuitPython.
Expand Down
Loading
Loading