From 5d96b769f763c7a8a36037b829d2cc76c105a19e Mon Sep 17 00:00:00 2001 From: Brad Barnett <127794626+bdbarnett@users.noreply.github.com> Date: Thu, 29 Oct 2026 14:53:28 -0500 Subject: [PATCH 1/2] audiosampler: a resonant low-pass, a modulation envelope and two LFOs per voice A voice can now play the rest of what a SoundFont region says about a note: a two-pole state-variable low-pass with resonance, a DAHDSR modulation envelope on pitch and cutoff, a vibrato LFO on pitch, and a modulation LFO on pitch, cutoff and volume. cutoff and vibrato_depth are live properties, so a mod wheel or a velocity curve can move them on a sounding note. They are worked out every 32 frames from polynomial approximations, so every target renders the same bits. A voice that uses none of them takes the old path and renders exactly as before, at the same cost. Measured in tests/test_cpython_audiosampler_modulation.py (response at and below the cutoff, the envelope's sweep, the LFOs' rate, depth and delay, the live properties, 64 filtered voices inside a block), with planted faults that fail them; timed by tests/parity/sampler_cost.py on any target. --- docs/guides/play-recorded-notes.md | 43 ++ docs/how-it-works/upstream-diff.md | 33 +- src/audiosampler/SamplerVoice.c | 102 +++- src/audiosampler/SamplerVoice.h | 2 + .../audiosampler/SamplerVoice.c | 132 ++++- .../shared-module/audiosampler/Sampler.h | 2 + src/cpython/_audiodsp.c | 73 ++- src/cpython/audiosampler.py | 74 ++- src/shared/audiodsp_sampler.c | 511 ++++++++++++++++++ src/shared/audiodsp_sampler.h | 111 ++++ tests/parity/sampler_cost.py | 107 ++++ tests/parity/sampler_probe.py | 28 + tests/test_cpython_audiosampler_modulation.py | 317 +++++++++++ 13 files changed, 1525 insertions(+), 10 deletions(-) create mode 100644 tests/parity/sampler_cost.py create mode 100644 tests/test_cpython_audiosampler_modulation.py diff --git a/docs/guides/play-recorded-notes.md b/docs/guides/play-recorded-notes.md index 986e4b02..ba665ceb 100644 --- a/docs/guides/play-recorded-notes.md +++ b/docs/guides/play-recorded-notes.md @@ -42,6 +42,49 @@ bend is played. `release()` enters the release stage from wherever the envelope is; `stop()` is silence from the next block; `playing` says whether the voice still sounds. +## Filter, modulation envelope and LFOs + +A voice can also do what a SoundFont region asks for besides its amplitude +envelope. Give it a resonant low-pass, a second envelope that sweeps the +cutoff and the pitch, a vibrato, and a slower LFO on pitch, cutoff and +volume: + +```python +sampler.voice[0].play(brass_c4, rate=44100, loop_start=2000, loop_end=6000, + loop_mode=audiosampler.LOOP_UNTIL_RELEASE, + attack=0.02, release=0.2, + cutoff=800, resonance=6, # Hz, dB + mod_attack=0.05, mod_decay=0.4, mod_sustain=0.3, + mod_to_cutoff=2400, # cents at full scale + vibrato_delay=0.3, vibrato_rate=5.5, vibrato_depth=15) +... +sampler.voice[0].vibrato_depth = 50 # a mod wheel +sampler.voice[0].cutoff = 1200 +``` + +`cutoff` is in Hz and `resonance` in dB: the filter's response at the cutoff +sits that far above its response at DC, and DC drops by half of it, as the +SoundFont specification puts it. Zero resonance is a Butterworth. A cutoff of +19.9 kHz or more, with no resonance and nothing moving it, is no filter at +all, and so is leaving it out. + +The modulation envelope has the amplitude envelope's six stages, prefixed +`mod_`, and moves pitch by `mod_to_pitch` and the cutoff by `mod_to_cutoff` +cents at full scale. The vibrato LFO (`vibrato_delay`, `vibrato_rate` in Hz, +`vibrato_depth` in cents) moves pitch. The modulation LFO (`lfo_delay`, +`lfo_rate`) moves pitch by `lfo_to_pitch` cents, the cutoff by +`lfo_to_cutoff` cents and the level by `lfo_to_volume` dB at its peak. Both +LFOs are triangles that start at zero, rising, once their delay is over. +`cutoff` and `vibrato_depth` are live properties. + +These are worked out every 32 frames, which at 48 kHz is about fourteen +times a cycle of the fastest LFO a SoundFont can ask for (110 Hz). A voice that uses none of them +renders exactly as before and costs the same. One that does costs about +1.7 times as much on a desktop; the numbers for a board are in +[what the filter and the LFOs cost](../how-it-works/upstream-diff.md#what-the-filter-and-the-lfos-cost). + +## How it renders + Every voice is mixed in one C call per block, summed in float and quantised once, through a 4-point Hermite interpolator, so a loop at pitch 1.0 is a wire. For why the position is fixed point and how the pitch is verified by diff --git a/docs/how-it-works/upstream-diff.md b/docs/how-it-works/upstream-diff.md index 6af3cd61..a6a91de6 100644 --- a/docs/how-it-works/upstream-diff.md +++ b/docs/how-it-works/upstream-diff.md @@ -70,8 +70,8 @@ run). On an ESP32-P4 at 400 MHz, 64 voices of a dense General MIDI file took about 5 ms of the block before the frames between a voice's events got a loop of their own (state in registers, two frames' interpolation worked out together for the in-order core, the same arithmetic in the same order, -so the same bits): about 2.4 ms after. Memory is the voice table (about 100 bytes a voice, two -slots each), a float accumulator of one block, and the output block. The +so the same bits): about 2.4 ms after. Memory is the voice table (256 bytes a voice on a +64-bit desktop since the filter and LFOs arrived, 112 before, two slots each), a float accumulator of one block, and the output block. The buffers are the caller's, held by reference while a voice sounds. A `Sampler` carries up to 128 voices (`audiosampler.MAX_VOICES`). Cost @@ -84,6 +84,35 @@ renders voices at its own pace, so scale the P4's 64 voices (2.4 ms of a 5.3 ms block) to what you ask of it, and give `voice_count` the number it can keep up with. +### What the filter and the LFOs cost + +A voice that sets none of `cutoff`, `resonance`, the modulation envelope's +depths or the LFOs' takes the path every voice took before they existed, +and renders the same bits (`sampler_probe.py`'s blocks from before them +match). One that does renders in 32-frame chunks: once a chunk it works out +its pitch, its filter's coefficients and its volume, then runs the chunk +through a two-pole state-variable low-pass. The coefficients take a +polynomial tangent and `audiodsp_exp2_poly`, never `tanf` or `powf`, so +every target renders the same bits. + +[`tests/parity/sampler_cost.py`](../../tests/parity/sampler_cost.py) times +it on any target, in microseconds per 256-frame stereo block at 48 kHz +(5333 µs), every voice looping: *plain* is no filter, *filter* a 2 kHz +low-pass with 6 dB of resonance, *full* the filter with the modulation +envelope on pitch and cutoff, the vibrato LFO, and the modulation LFO on +pitch, cutoff and volume. + +| Voices | CPython plain | filter | full | MicroPython unix plain | filter | full | +|---|---|---|---|---|---|---| +| 32 | 32 | 56 | 65 | 79 | 104 | 112 | +| 48 | 47 | 83 | 95 | 111 | 154 | 163 | +| 64 | 62 | 102 | 122 | 152 | 207 | 225 | +| 128 | 115 | 208 | 247 | 308 | 407 | 452 | + +(x86-64, one core, gcc -O2.) + +P4_TABLE + ### How it is verified [`tests/parity/sampler_probe.py`](../../tests/parity/sampler_probe.py) in `verify_dsp`'s set, for the three-target diff --git a/src/audiosampler/SamplerVoice.c b/src/audiosampler/SamplerVoice.c index 2911c8ca..2e62e4aa 100644 --- a/src/audiosampler/SamplerVoice.c +++ b/src/audiosampler/SamplerVoice.c @@ -19,6 +19,8 @@ mp_obj_t audiosampler_samplervoice_new(audiosampler_sampler_obj_t *parent, self->level = mp_obj_new_float(1.0); self->pan = mp_obj_new_float(0.0); self->pitch = mp_obj_new_float(1.0); + self->cutoff = mp_const_none; + self->vibrato_depth = mp_obj_new_float(0.0); return MP_OBJ_FROM_PTR(self); } @@ -33,7 +35,8 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) { enum { ARG_buffer, ARG_rate, ARG_pitch, ARG_loop_start, ARG_loop_end, ARG_loop_mode, ARG_level, ARG_pan, ARG_delay, ARG_attack, ARG_hold, - ARG_decay, ARG_sustain, ARG_release }; + ARG_decay, ARG_sustain, ARG_release, + ARG_cutoff, ARG_resonance, ARG_mod_delay, ARG_mod_attack, ARG_mod_hold, ARG_mod_decay, ARG_mod_sustain, ARG_mod_release, ARG_mod_to_pitch, ARG_mod_to_cutoff, ARG_vibrato_delay, ARG_vibrato_rate, ARG_vibrato_depth, ARG_lfo_delay, ARG_lfo_rate, ARG_lfo_to_pitch, ARG_lfo_to_cutoff, ARG_lfo_to_volume }; static const mp_arg_t allowed_args[] = { { MP_QSTR_buffer, MP_ARG_REQUIRED | MP_ARG_OBJ, {.u_obj = MP_OBJ_NULL} }, { MP_QSTR_rate, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, @@ -49,6 +52,24 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, { MP_QSTR_decay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, { MP_QSTR_sustain, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, { MP_QSTR_release, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_cutoff, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_resonance, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_delay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_attack, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_hold, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_decay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_sustain, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_release, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_to_pitch, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_to_cutoff, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_vibrato_delay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_vibrato_rate, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_vibrato_depth, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_delay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_rate, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_to_pitch, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_to_cutoff, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_to_volume, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, }; audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(pos_args[0]); audiosample_check_for_deinit(&self->parent->base); @@ -88,6 +109,24 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, note.decay = float_arg(args[ARG_decay].u_obj, 0.0f); note.sustain = float_arg(args[ARG_sustain].u_obj, 1.0f); note.release = float_arg(args[ARG_release].u_obj, 0.0f); + note.cutoff = float_arg(args[ARG_cutoff].u_obj, 0.0f); + note.resonance = float_arg(args[ARG_resonance].u_obj, 0.0f); + note.mod_delay = float_arg(args[ARG_mod_delay].u_obj, 0.0f); + note.mod_attack = float_arg(args[ARG_mod_attack].u_obj, 0.0f); + note.mod_hold = float_arg(args[ARG_mod_hold].u_obj, 0.0f); + note.mod_decay = float_arg(args[ARG_mod_decay].u_obj, 0.0f); + note.mod_sustain = float_arg(args[ARG_mod_sustain].u_obj, 1.0f); + note.mod_release = float_arg(args[ARG_mod_release].u_obj, 0.0f); + note.mod_to_pitch = float_arg(args[ARG_mod_to_pitch].u_obj, 0.0f); + note.mod_to_cutoff = float_arg(args[ARG_mod_to_cutoff].u_obj, 0.0f); + note.vibrato_delay = float_arg(args[ARG_vibrato_delay].u_obj, 0.0f); + note.vibrato_rate = float_arg(args[ARG_vibrato_rate].u_obj, 0.0f); + note.vibrato_depth = float_arg(args[ARG_vibrato_depth].u_obj, 0.0f); + note.lfo_delay = float_arg(args[ARG_lfo_delay].u_obj, 0.0f); + note.lfo_rate = float_arg(args[ARG_lfo_rate].u_obj, 0.0f); + note.lfo_to_pitch = float_arg(args[ARG_lfo_to_pitch].u_obj, 0.0f); + note.lfo_to_cutoff = float_arg(args[ARG_lfo_to_cutoff].u_obj, 0.0f); + note.lfo_to_volume = float_arg(args[ARG_lfo_to_volume].u_obj, 0.0f); // Read and checked first, stored under the lock: a pull landing between // the kernel's play() and the reference store would read a buffer the @@ -115,6 +154,9 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, ? mp_obj_new_float(0.0) : args[ARG_pan].u_obj; self->pitch = args[ARG_pitch].u_obj == mp_const_none ? mp_obj_new_float(1.0) : args[ARG_pitch].u_obj; + self->cutoff = args[ARG_cutoff].u_obj; + self->vibrato_depth = args[ARG_vibrato_depth].u_obj == mp_const_none + ? mp_obj_new_float(0.0) : args[ARG_vibrato_depth].u_obj; return mp_const_none; } static MP_DEFINE_CONST_FUN_OBJ_KW(audiosampler_samplervoice_play_obj, 1, @@ -236,6 +278,61 @@ MP_PROPERTY_GETSET(audiosampler_samplervoice_pitch_obj, (mp_obj_t)&audiosampler_samplervoice_get_pitch_obj, (mp_obj_t)&audiosampler_samplervoice_set_pitch_obj); +// The low-pass cutoff in Hz, None for no filter. Live: a change lands +// within 32 frames, and a filter switched on starts from rest. +static mp_obj_t audiosampler_samplervoice_obj_get_cutoff(mp_obj_t self_in) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + return self->cutoff; +} +static MP_DEFINE_CONST_FUN_OBJ_1(audiosampler_samplervoice_get_cutoff_obj, + audiosampler_samplervoice_obj_get_cutoff); +static mp_obj_t audiosampler_samplervoice_obj_set_cutoff(mp_obj_t self_in, + mp_obj_t value) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + float cutoff = value == mp_const_none ? 0.0f + : (float)mp_obj_get_float(value); + audiodsp_pump_lock_acquire(); + audiodsp_sampler_set_cutoff(&self->parent->sampler, self->index, cutoff); + self->cutoff = value; + audiodsp_pump_lock_release(); + return mp_const_none; +} +static MP_DEFINE_CONST_FUN_OBJ_2(audiosampler_samplervoice_set_cutoff_obj, + audiosampler_samplervoice_obj_set_cutoff); +MP_PROPERTY_GETSET(audiosampler_samplervoice_cutoff_obj, + (mp_obj_t)&audiosampler_samplervoice_get_cutoff_obj, + (mp_obj_t)&audiosampler_samplervoice_set_cutoff_obj); + +// The vibrato LFO's depth in cents. Live, so a mod wheel is a series of +// writes here; each lands within 32 frames. +static mp_obj_t audiosampler_samplervoice_obj_get_vibrato_depth( + mp_obj_t self_in) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + return self->vibrato_depth; +} +static MP_DEFINE_CONST_FUN_OBJ_1(audiosampler_samplervoice_get_vibrato_depth_obj, + audiosampler_samplervoice_obj_get_vibrato_depth); +static mp_obj_t audiosampler_samplervoice_obj_set_vibrato_depth( + mp_obj_t self_in, mp_obj_t value) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + float cents = (float)mp_obj_get_float(value); + audiodsp_pump_lock_acquire(); + audiodsp_sampler_set_vibrato_depth(&self->parent->sampler, self->index, + cents); + self->vibrato_depth = value; + audiodsp_pump_lock_release(); + return mp_const_none; +} +static MP_DEFINE_CONST_FUN_OBJ_2(audiosampler_samplervoice_set_vibrato_depth_obj, + audiosampler_samplervoice_obj_set_vibrato_depth); +MP_PROPERTY_GETSET(audiosampler_samplervoice_vibrato_depth_obj, + (mp_obj_t)&audiosampler_samplervoice_get_vibrato_depth_obj, + (mp_obj_t)&audiosampler_samplervoice_set_vibrato_depth_obj); + // `deinit()` on a voice is `stop()`, with no fade, plus letting go of the // buffer: a voice // has nothing of its own to release, it belongs to its Sampler, but every @@ -267,6 +364,9 @@ static const mp_rom_map_elem_t audiosampler_samplervoice_locals_table[] = { { MP_ROM_QSTR(MP_QSTR_level), MP_ROM_PTR(&audiosampler_samplervoice_level_obj) }, { MP_ROM_QSTR(MP_QSTR_pan), MP_ROM_PTR(&audiosampler_samplervoice_pan_obj) }, { MP_ROM_QSTR(MP_QSTR_pitch), MP_ROM_PTR(&audiosampler_samplervoice_pitch_obj) }, + { MP_ROM_QSTR(MP_QSTR_cutoff), MP_ROM_PTR(&audiosampler_samplervoice_cutoff_obj) }, + { MP_ROM_QSTR(MP_QSTR_vibrato_depth), + MP_ROM_PTR(&audiosampler_samplervoice_vibrato_depth_obj) }, }; static MP_DEFINE_CONST_DICT(audiosampler_samplervoice_locals, audiosampler_samplervoice_locals_table); diff --git a/src/audiosampler/SamplerVoice.h b/src/audiosampler/SamplerVoice.h index 46ea477c..6bf64858 100644 --- a/src/audiosampler/SamplerVoice.h +++ b/src/audiosampler/SamplerVoice.h @@ -27,6 +27,8 @@ typedef struct { mp_obj_t level; mp_obj_t pan; mp_obj_t pitch; + mp_obj_t cutoff; + mp_obj_t vibrato_depth; } audiosampler_samplervoice_obj_t; extern const mp_obj_type_t audiosampler_samplervoice_type; diff --git a/src/circuitpython_spike/shared-bindings/audiosampler/SamplerVoice.c b/src/circuitpython_spike/shared-bindings/audiosampler/SamplerVoice.c index 8e552519..f968dc4a 100644 --- a/src/circuitpython_spike/shared-bindings/audiosampler/SamplerVoice.c +++ b/src/circuitpython_spike/shared-bindings/audiosampler/SamplerVoice.c @@ -26,6 +26,8 @@ mp_obj_t audiosampler_samplervoice_new(audiosampler_sampler_obj_t *parent, self->level = mp_obj_new_float(1.0); self->pan = mp_obj_new_float(0.0); self->pitch = mp_obj_new_float(1.0); + self->cutoff = mp_const_none; + self->vibrato_depth = mp_obj_new_float(0.0); return MP_OBJ_FROM_PTR(self); } @@ -50,6 +52,24 @@ static float float_arg(mp_obj_t value, float fallback) { //| decay: float = 0.0, //| sustain: float = 1.0, //| release: float = 0.0, +//| cutoff: Optional[float] = None, +//| resonance: float = 0.0, +//| mod_delay: float = 0.0, +//| mod_attack: float = 0.0, +//| mod_hold: float = 0.0, +//| mod_decay: float = 0.0, +//| mod_sustain: float = 1.0, +//| mod_release: float = 0.0, +//| mod_to_pitch: float = 0.0, +//| mod_to_cutoff: float = 0.0, +//| vibrato_delay: float = 0.0, +//| vibrato_rate: float = 0.0, +//| vibrato_depth: float = 0.0, +//| lfo_delay: float = 0.0, +//| lfo_rate: float = 0.0, +//| lfo_to_pitch: float = 0.0, +//| lfo_to_cutoff: float = 0.0, +//| lfo_to_volume: float = 0.0, //| ) -> None: //| """Start ``buffer`` (mono signed 16-bit) on this voice, replacing //| whatever it was playing, which fades out over 5 ms under the new @@ -59,13 +79,24 @@ static float float_arg(mp_obj_t value, float fallback) { //| of the buffer. The envelope stages are in seconds and ``sustain`` //| is a linear level; ``delay`` postpones the start of the sound. //| Raises ValueError for a loop that does not fit the buffer or a -//| ``loop_mode`` that is not 0, 1 or 3.""" +//| ``loop_mode`` that is not 0, 1 or 3. +//| +//| ``cutoff`` (Hz) and ``resonance`` (dB at the cutoff above DC) set a +//| two-pole low-pass; 19.9 kHz or more with no resonance and nothing +//| moving it is no filter. The ``mod_`` envelope moves pitch and +//| cutoff by ``mod_to_pitch`` and ``mod_to_cutoff`` cents at full +//| scale. The vibrato LFO moves pitch by ``vibrato_depth`` cents; the +//| modulation LFO moves pitch and cutoff by ``lfo_to_pitch`` and +//| ``lfo_to_cutoff`` cents and the level by ``lfo_to_volume`` dB. +//| Rates are in Hz and delays in seconds; both LFOs are triangles +//| from zero, rising.""" //| ... static mp_obj_t audiosampler_samplervoice_play(size_t n_args, const mp_obj_t *pos_args, mp_map_t *kw_args) { enum { ARG_buffer, ARG_rate, ARG_pitch, ARG_loop_start, ARG_loop_end, ARG_loop_mode, ARG_level, ARG_pan, ARG_delay, ARG_attack, ARG_hold, - ARG_decay, ARG_sustain, ARG_release }; + ARG_decay, ARG_sustain, ARG_release, + ARG_cutoff, ARG_resonance, ARG_mod_delay, ARG_mod_attack, ARG_mod_hold, ARG_mod_decay, ARG_mod_sustain, ARG_mod_release, ARG_mod_to_pitch, ARG_mod_to_cutoff, ARG_vibrato_delay, ARG_vibrato_rate, ARG_vibrato_depth, ARG_lfo_delay, ARG_lfo_rate, ARG_lfo_to_pitch, ARG_lfo_to_cutoff, ARG_lfo_to_volume }; static const mp_arg_t allowed_args[] = { { MP_QSTR_buffer, MP_ARG_REQUIRED | MP_ARG_OBJ, {.u_obj = MP_OBJ_NULL} }, { MP_QSTR_rate, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, @@ -81,6 +112,24 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, { MP_QSTR_decay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, { MP_QSTR_sustain, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, { MP_QSTR_release, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_cutoff, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_resonance, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_delay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_attack, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_hold, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_decay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_sustain, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_release, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_to_pitch, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_mod_to_cutoff, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_vibrato_delay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_vibrato_rate, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_vibrato_depth, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_delay, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_rate, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_to_pitch, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_to_cutoff, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, + { MP_QSTR_lfo_to_volume, MP_ARG_OBJ | MP_ARG_KW_ONLY, {.u_rom_obj = MP_ROM_NONE} }, }; audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(pos_args[0]); audiosample_check_for_deinit(&self->parent->base); @@ -120,6 +169,24 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, note.decay = float_arg(args[ARG_decay].u_obj, 0.0f); note.sustain = float_arg(args[ARG_sustain].u_obj, 1.0f); note.release = float_arg(args[ARG_release].u_obj, 0.0f); + note.cutoff = float_arg(args[ARG_cutoff].u_obj, 0.0f); + note.resonance = float_arg(args[ARG_resonance].u_obj, 0.0f); + note.mod_delay = float_arg(args[ARG_mod_delay].u_obj, 0.0f); + note.mod_attack = float_arg(args[ARG_mod_attack].u_obj, 0.0f); + note.mod_hold = float_arg(args[ARG_mod_hold].u_obj, 0.0f); + note.mod_decay = float_arg(args[ARG_mod_decay].u_obj, 0.0f); + note.mod_sustain = float_arg(args[ARG_mod_sustain].u_obj, 1.0f); + note.mod_release = float_arg(args[ARG_mod_release].u_obj, 0.0f); + note.mod_to_pitch = float_arg(args[ARG_mod_to_pitch].u_obj, 0.0f); + note.mod_to_cutoff = float_arg(args[ARG_mod_to_cutoff].u_obj, 0.0f); + note.vibrato_delay = float_arg(args[ARG_vibrato_delay].u_obj, 0.0f); + note.vibrato_rate = float_arg(args[ARG_vibrato_rate].u_obj, 0.0f); + note.vibrato_depth = float_arg(args[ARG_vibrato_depth].u_obj, 0.0f); + note.lfo_delay = float_arg(args[ARG_lfo_delay].u_obj, 0.0f); + note.lfo_rate = float_arg(args[ARG_lfo_rate].u_obj, 0.0f); + note.lfo_to_pitch = float_arg(args[ARG_lfo_to_pitch].u_obj, 0.0f); + note.lfo_to_cutoff = float_arg(args[ARG_lfo_to_cutoff].u_obj, 0.0f); + note.lfo_to_volume = float_arg(args[ARG_lfo_to_volume].u_obj, 0.0f); bool handed = audiodsp_sampler_audible(&self->parent->sampler, self->index); if (!audiodsp_sampler_play(&self->parent->sampler, self->index, @@ -139,6 +206,9 @@ static mp_obj_t audiosampler_samplervoice_play(size_t n_args, ? mp_obj_new_float(0.0) : args[ARG_pan].u_obj; self->pitch = args[ARG_pitch].u_obj == mp_const_none ? mp_obj_new_float(1.0) : args[ARG_pitch].u_obj; + self->cutoff = args[ARG_cutoff].u_obj; + self->vibrato_depth = args[ARG_vibrato_depth].u_obj == mp_const_none + ? mp_obj_new_float(0.0) : args[ARG_vibrato_depth].u_obj; return mp_const_none; } MP_DEFINE_CONST_FUN_OBJ_KW(audiosampler_samplervoice_play_obj, 1, @@ -266,6 +336,61 @@ MP_PROPERTY_GETSET(audiosampler_samplervoice_pitch_obj, (mp_obj_t)&audiosampler_samplervoice_get_pitch_obj, (mp_obj_t)&audiosampler_samplervoice_set_pitch_obj); +//| cutoff: Optional[float] +//| """The low-pass cutoff in Hz, None for no filter. Live: a change lands +//| within 32 frames, and a filter switched on starts from rest.""" +//| +static mp_obj_t audiosampler_samplervoice_obj_get_cutoff(mp_obj_t self_in) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + return self->cutoff; +} +MP_DEFINE_CONST_FUN_OBJ_1(audiosampler_samplervoice_get_cutoff_obj, + audiosampler_samplervoice_obj_get_cutoff); +static mp_obj_t audiosampler_samplervoice_obj_set_cutoff(mp_obj_t self_in, + mp_obj_t value) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + float cutoff = value == mp_const_none ? 0.0f + : (float)mp_obj_get_float(value); + audiodsp_sampler_set_cutoff(&self->parent->sampler, self->index, cutoff); + self->cutoff = value; + return mp_const_none; +} +MP_DEFINE_CONST_FUN_OBJ_2(audiosampler_samplervoice_set_cutoff_obj, + audiosampler_samplervoice_obj_set_cutoff); +MP_PROPERTY_GETSET(audiosampler_samplervoice_cutoff_obj, + (mp_obj_t)&audiosampler_samplervoice_get_cutoff_obj, + (mp_obj_t)&audiosampler_samplervoice_set_cutoff_obj); + +//| vibrato_depth: float +//| """The vibrato LFO's depth in cents. Live, so a mod wheel is a series +//| of writes here; each lands within 32 frames.""" +//| +static mp_obj_t audiosampler_samplervoice_obj_get_vibrato_depth( + mp_obj_t self_in) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + return self->vibrato_depth; +} +MP_DEFINE_CONST_FUN_OBJ_1(audiosampler_samplervoice_get_vibrato_depth_obj, + audiosampler_samplervoice_obj_get_vibrato_depth); +static mp_obj_t audiosampler_samplervoice_obj_set_vibrato_depth( + mp_obj_t self_in, mp_obj_t value) { + audiosampler_samplervoice_obj_t *self = MP_OBJ_TO_PTR(self_in); + audiosample_check_for_deinit(&self->parent->base); + float cents = (float)mp_obj_get_float(value); + audiodsp_sampler_set_vibrato_depth(&self->parent->sampler, self->index, + cents); + self->vibrato_depth = value; + return mp_const_none; +} +MP_DEFINE_CONST_FUN_OBJ_2(audiosampler_samplervoice_set_vibrato_depth_obj, + audiosampler_samplervoice_obj_set_vibrato_depth); +MP_PROPERTY_GETSET(audiosampler_samplervoice_vibrato_depth_obj, + (mp_obj_t)&audiosampler_samplervoice_get_vibrato_depth_obj, + (mp_obj_t)&audiosampler_samplervoice_set_vibrato_depth_obj); + //| def deinit(self) -> None: //| """Stop the voice and let go of its buffer. A voice has nothing of //| its own to release; this is `stop()`, with no fade, under the name @@ -296,6 +421,9 @@ static const mp_rom_map_elem_t audiosampler_samplervoice_locals_dict_table[] = { { MP_ROM_QSTR(MP_QSTR_level), MP_ROM_PTR(&audiosampler_samplervoice_level_obj) }, { MP_ROM_QSTR(MP_QSTR_pan), MP_ROM_PTR(&audiosampler_samplervoice_pan_obj) }, { MP_ROM_QSTR(MP_QSTR_pitch), MP_ROM_PTR(&audiosampler_samplervoice_pitch_obj) }, + { MP_ROM_QSTR(MP_QSTR_cutoff), MP_ROM_PTR(&audiosampler_samplervoice_cutoff_obj) }, + { MP_ROM_QSTR(MP_QSTR_vibrato_depth), + MP_ROM_PTR(&audiosampler_samplervoice_vibrato_depth_obj) }, }; static MP_DEFINE_CONST_DICT(audiosampler_samplervoice_locals_dict, audiosampler_samplervoice_locals_dict_table); diff --git a/src/circuitpython_spike/shared-module/audiosampler/Sampler.h b/src/circuitpython_spike/shared-module/audiosampler/Sampler.h index 0658412c..6e570927 100644 --- a/src/circuitpython_spike/shared-module/audiosampler/Sampler.h +++ b/src/circuitpython_spike/shared-module/audiosampler/Sampler.h @@ -45,6 +45,8 @@ typedef struct { mp_obj_t level; mp_obj_t pan; mp_obj_t pitch; + mp_obj_t cutoff; + mp_obj_t vibrato_depth; } audiosampler_samplervoice_obj_t; void audiosampler_sampler_reset_buffer(audiosampler_sampler_obj_t *self, diff --git a/src/cpython/_audiodsp.c b/src/cpython/_audiodsp.c index f30d635b..4478e7a7 100644 --- a/src/cpython/_audiodsp.c +++ b/src/cpython/_audiodsp.c @@ -2794,7 +2794,25 @@ static PyObject *sampler_state_play(audiodsp_sampler_object_t *self, static char *keywords[] = {"index", "buffer", "rate", "pitch", "loop_start", "loop_end", "loop_mode", "level", "pan", "delay", "attack", "hold", "decay", - "sustain", "release", NULL}; + "sustain", "release", + "cutoff", + "resonance", + "mod_delay", + "mod_attack", + "mod_hold", + "mod_decay", + "mod_sustain", + "mod_release", + "mod_to_pitch", + "mod_to_cutoff", + "vibrato_delay", + "vibrato_rate", + "vibrato_depth", + "lfo_delay", + "lfo_rate", + "lfo_to_pitch", + "lfo_to_cutoff", + "lfo_to_volume", NULL}; PyObject *index_obj; PyObject *buffer_obj; audiodsp_sampler_note_t note; @@ -2803,10 +2821,13 @@ static PyObject *sampler_state_play(audiodsp_sampler_object_t *self, double delay = 0.0, attack = 0.0, hold = 0.0, decay = 0.0, sustain = 1.0, release = 0.0; unsigned long loop_start = 0, loop_end = 0, loop_mode = 0; - if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO|ddkkkdddddddd:play", + double mod[18] = {0.0}; + mod[6] = 1.0; // mod_sustain + if (!PyArg_ParseTupleAndKeywords(args, kwargs, + "OO|ddkkkdddddddddddddddddddddddddd:play", keywords, &index_obj, &buffer_obj, &rate, &pitch, &loop_start, &loop_end, &loop_mode, &level, &pan, &delay, &attack, &hold, &decay, - &sustain, &release)) { + &sustain, &release, &mod[0], &mod[1], &mod[2], &mod[3], &mod[4], &mod[5], &mod[6], &mod[7], &mod[8], &mod[9], &mod[10], &mod[11], &mod[12], &mod[13], &mod[14], &mod[15], &mod[16], &mod[17])) { return NULL; } uint32_t index; @@ -2840,6 +2861,24 @@ static PyObject *sampler_state_play(audiodsp_sampler_object_t *self, note.decay = (float)decay; note.sustain = (float)sustain; note.release = (float)release; + note.cutoff = (float)mod[0]; + note.resonance = (float)mod[1]; + note.mod_delay = (float)mod[2]; + note.mod_attack = (float)mod[3]; + note.mod_hold = (float)mod[4]; + note.mod_decay = (float)mod[5]; + note.mod_sustain = (float)mod[6]; + note.mod_release = (float)mod[7]; + note.mod_to_pitch = (float)mod[8]; + note.mod_to_cutoff = (float)mod[9]; + note.vibrato_delay = (float)mod[10]; + note.vibrato_rate = (float)mod[11]; + note.vibrato_depth = (float)mod[12]; + note.lfo_delay = (float)mod[13]; + note.lfo_rate = (float)mod[14]; + note.lfo_to_pitch = (float)mod[15]; + note.lfo_to_cutoff = (float)mod[16]; + note.lfo_to_volume = (float)mod[17]; // Let go of what this voice was reading only once the new buffer is // known to be good, so a refused play() leaves the voice as it was. // The kernel leaves the voice and its tail untouched on a refusal. @@ -2935,6 +2974,30 @@ static PyObject *sampler_state_set_pitch(audiodsp_sampler_object_t *self, Py_RETURN_NONE; } +static PyObject *sampler_state_set_cutoff(audiodsp_sampler_object_t *self, + PyObject *args) { + PyObject *index_obj; + double value; + if (!PyArg_ParseTuple(args, "Od:set_cutoff", &index_obj, &value)) return NULL; + uint32_t index; + if (sampler_state_voice(self, index_obj, &index) < 0) return NULL; + audiodsp_sampler_set_cutoff(&self->sampler, index, (float)value); + Py_RETURN_NONE; +} + +static PyObject *sampler_state_set_vibrato_depth( + audiodsp_sampler_object_t *self, PyObject *args) { + PyObject *index_obj; + double value; + if (!PyArg_ParseTuple(args, "Od:set_vibrato_depth", &index_obj, &value)) { + return NULL; + } + uint32_t index; + if (sampler_state_voice(self, index_obj, &index) < 0) return NULL; + audiodsp_sampler_set_vibrato_depth(&self->sampler, index, (float)value); + Py_RETURN_NONE; +} + static PyObject *sampler_state_sounding(audiodsp_sampler_object_t *self, PyObject *unused) { uint32_t count = 0; @@ -2969,6 +3032,8 @@ AUDIODSP_LOCKED_ARGS(sampler_state_playing, audiodsp_sampler_object_t) AUDIODSP_LOCKED_ARGS(sampler_state_set_level, audiodsp_sampler_object_t) AUDIODSP_LOCKED_ARGS(sampler_state_set_pan, audiodsp_sampler_object_t) AUDIODSP_LOCKED_ARGS(sampler_state_set_pitch, audiodsp_sampler_object_t) +AUDIODSP_LOCKED_ARGS(sampler_state_set_cutoff, audiodsp_sampler_object_t) +AUDIODSP_LOCKED_ARGS(sampler_state_set_vibrato_depth, audiodsp_sampler_object_t) AUDIODSP_LOCKED_ARGS(sampler_state_sounding, audiodsp_sampler_object_t) AUDIODSP_LOCKED_ARGS(sampler_state_render, audiodsp_sampler_object_t) static PyMethodDef sampler_state_methods[] = { @@ -2980,6 +3045,8 @@ static PyMethodDef sampler_state_methods[] = { {"set_level", (PyCFunction)sampler_state_set_level_locked, METH_VARARGS, NULL}, {"set_pan", (PyCFunction)sampler_state_set_pan_locked, METH_VARARGS, NULL}, {"set_pitch", (PyCFunction)sampler_state_set_pitch_locked, METH_VARARGS, NULL}, + {"set_cutoff", (PyCFunction)sampler_state_set_cutoff_locked, METH_VARARGS, NULL}, + {"set_vibrato_depth", (PyCFunction)sampler_state_set_vibrato_depth_locked, METH_VARARGS, NULL}, {"sounding", (PyCFunction)sampler_state_sounding_locked, METH_NOARGS, NULL}, {"render", (PyCFunction)sampler_state_render_locked, METH_NOARGS, NULL}, {NULL, NULL, 0, NULL}, diff --git a/src/cpython/audiosampler.py b/src/cpython/audiosampler.py index f2b85551..2e92aa24 100644 --- a/src/cpython/audiosampler.py +++ b/src/cpython/audiosampler.py @@ -34,6 +34,22 @@ `pitch` are live: a change lands at the next block, the first two as a ramp across it so a fader move is a slope rather than a click. +A voice can also carry what a SoundFont region asks for besides its +amplitude envelope. `cutoff` (Hz) and `resonance` (dB at the cutoff above +DC) set a two-pole resonant low-pass; a cutoff of 19.9 kHz or more with no +resonance and nothing moving it is no filter, and that is the default. The +modulation envelope (`mod_delay`, `mod_attack`, `mod_hold`, `mod_decay`, +`mod_sustain`, `mod_release`) moves pitch by `mod_to_pitch` cents and the +cutoff by `mod_to_cutoff` cents at full scale. The vibrato LFO +(`vibrato_delay`, `vibrato_rate` in Hz, `vibrato_depth` in cents) moves +pitch; the modulation LFO (`lfo_delay`, `lfo_rate`) moves pitch by +`lfo_to_pitch` cents, the cutoff by `lfo_to_cutoff` cents and the level by +`lfo_to_volume` dB at its peak. Both LFOs are triangles that start at zero, +rising, after their delay. `cutoff` and `vibrato_depth` are live too, which +is how a mod wheel reaches a sounding note. These are worked out every 32 +frames; a voice that uses none of them costs no more than one that never +heard of them. + Every voice is mixed in one C call per block, in float, and quantised once. Interpolation is a 4-point Hermite, so a loop at pitch 1.0 is a wire. """ @@ -64,11 +80,18 @@ def __init__(self, sampler, index): self._level = 1.0 self._pan = 0.0 self._pitch = 1.0 + self._cutoff = None + self._vibrato_depth = 0.0 def play(self, buffer, *, rate=None, pitch=1.0, loop_start=0, loop_end=None, loop_mode=LOOP_NONE, level=1.0, pan=0.0, delay=0.0, attack=0.0, hold=0.0, decay=0.0, sustain=1.0, - release=0.0): + release=0.0, cutoff=None, resonance=0.0, mod_delay=0.0, + mod_attack=0.0, mod_hold=0.0, mod_decay=0.0, mod_sustain=1.0, + mod_release=0.0, mod_to_pitch=0.0, mod_to_cutoff=0.0, + vibrato_delay=0.0, vibrato_rate=0.0, vibrato_depth=0.0, + lfo_delay=0.0, lfo_rate=0.0, lfo_to_pitch=0.0, + lfo_to_cutoff=0.0, lfo_to_volume=0.0): """Start `buffer` (mono signed 16-bit) on this voice, replacing whatever it was playing, which fades out over 5 ms under the new note. `loop_end` left out means the end of the buffer. Raises ValueError for a loop that does not fit the buffer or @@ -82,8 +105,29 @@ def play(self, buffer, *, rate=None, pitch=1.0, loop_start=0, loop_mode=int(loop_mode), level=float(level), pan=float(pan), delay=float(delay), attack=float(attack), hold=float(hold), decay=float(decay), sustain=float(sustain), - release=float(release)) + release=float(release), + cutoff=0.0 if cutoff is None else float(cutoff), + resonance=float(resonance), + mod_delay=float(mod_delay), + mod_attack=float(mod_attack), + mod_hold=float(mod_hold), + mod_decay=float(mod_decay), + mod_sustain=float(mod_sustain), + mod_release=float(mod_release), + mod_to_pitch=float(mod_to_pitch), + mod_to_cutoff=float(mod_to_cutoff), + vibrato_delay=float(vibrato_delay), + vibrato_rate=float(vibrato_rate), + vibrato_depth=float(vibrato_depth), + lfo_delay=float(lfo_delay), + lfo_rate=float(lfo_rate), + lfo_to_pitch=float(lfo_to_pitch), + lfo_to_cutoff=float(lfo_to_cutoff), + lfo_to_volume=float(lfo_to_volume), + ) self._level = float(level) + self._cutoff = None if cutoff is None else float(cutoff) + self._vibrato_depth = float(vibrato_depth) self._pan = float(pan) self._pitch = float(pitch) @@ -147,6 +191,32 @@ def pitch(self, value): self._sampler._state.set_pitch(self._index, self._pitch) + @property + def cutoff(self): + """The low-pass cutoff in Hz, None for no filter. Lands at the next + 32 frames; the filter starts from rest if it was off.""" + return self._cutoff + + @cutoff.setter + def cutoff(self, value): + self._sampler._check() + self._cutoff = None if value is None else float(value) + self._sampler._state.set_cutoff( + self._index, 0.0 if value is None else self._cutoff) + + @property + def vibrato_depth(self): + """The vibrato LFO's depth in cents. Lands at the next 32 frames.""" + return self._vibrato_depth + + @vibrato_depth.setter + def vibrato_depth(self, value): + self._sampler._check() + self._vibrato_depth = float(value) + self._sampler._state.set_vibrato_depth(self._index, + self._vibrato_depth) + + class SampleBuffer(bytearray): """`size` zeroed bytes for a sampler's voices to read. diff --git a/src/shared/audiodsp_sampler.c b/src/shared/audiodsp_sampler.c index bf5d0805..9e77a369 100644 --- a/src/shared/audiodsp_sampler.c +++ b/src/shared/audiodsp_sampler.c @@ -7,6 +7,7 @@ #include +#include "shared/audiodsp_exp2.h" #include "shared/audiodsp_hot.h" #define Q32_ONE 4294967296.0 @@ -70,6 +71,8 @@ void audiodsp_sampler_init(audiodsp_sampler_t *self, uint32_t sample_rate, voices[i].pitch = 1.0f; voices[i].rate_ratio = 1.0f; voices[i].sustain = 1.0f; + voices[i].mod_gain = 1.0f; + voices[i].filter_gain = 1.0f; } } @@ -78,6 +81,7 @@ void audiodsp_sampler_note_init(audiodsp_sampler_note_t *note) { note->pitch = 1.0f; note->level = 1.0f; note->sustain = 1.0f; + note->mod_sustain = 1.0f; } // --- the envelope ------------------------------------------------------------ @@ -166,6 +170,240 @@ static void stage_done(audiodsp_sampler_voice_t *voice) { } } +// --- the modulation envelope --------------------------------------------------- + +// The same six stages as the amplitude envelope, linear in its own units (a +// fraction of full scale), and run once a chunk rather than once a sample. +static void mod_enter(audiodsp_sampler_voice_t *voice, uint32_t stage) { + for (;;) { + voice->mod_stage = stage; + switch (stage) { + case AUDIODSP_SAMPLER_STAGE_DELAY: + voice->mod_env = 0.0f; + voice->mod_step = 0.0f; + if (voice->mod_delay_samples == 0u) { + stage = AUDIODSP_SAMPLER_STAGE_ATTACK; + continue; + } + voice->mod_left = voice->mod_delay_samples; + return; + case AUDIODSP_SAMPLER_STAGE_ATTACK: + voice->mod_env = 0.0f; + if (voice->mod_attack_samples == 0u) { + stage = AUDIODSP_SAMPLER_STAGE_HOLD; + continue; + } + voice->mod_left = voice->mod_attack_samples; + voice->mod_step = 1.0f / (float)voice->mod_attack_samples; + return; + case AUDIODSP_SAMPLER_STAGE_HOLD: + voice->mod_env = 1.0f; + voice->mod_step = 0.0f; + if (voice->mod_hold_samples == 0u) { + stage = AUDIODSP_SAMPLER_STAGE_DECAY; + continue; + } + voice->mod_left = voice->mod_hold_samples; + return; + case AUDIODSP_SAMPLER_STAGE_DECAY: + voice->mod_env = 1.0f; + if (voice->mod_decay_samples == 0u) { + stage = AUDIODSP_SAMPLER_STAGE_SUSTAIN; + continue; + } + voice->mod_left = voice->mod_decay_samples; + voice->mod_step = (voice->mod_sustain - 1.0f) / + (float)voice->mod_decay_samples; + return; + case AUDIODSP_SAMPLER_STAGE_SUSTAIN: + voice->mod_env = voice->mod_sustain; + voice->mod_step = 0.0f; + voice->mod_left = 0; + return; + case AUDIODSP_SAMPLER_STAGE_RELEASE: + if (voice->mod_release_samples == 0u || + voice->mod_env <= 0.0f) { + stage = AUDIODSP_SAMPLER_STAGE_IDLE; + continue; + } + voice->mod_left = voice->mod_release_samples; + voice->mod_step = -voice->mod_env / + (float)voice->mod_release_samples; + return; + case AUDIODSP_SAMPLER_STAGE_IDLE: + default: + voice->mod_stage = AUDIODSP_SAMPLER_STAGE_IDLE; + voice->mod_env = 0.0f; + voice->mod_step = 0.0f; + voice->mod_left = 0; + return; + } + } +} + +static void mod_advance(audiodsp_sampler_voice_t *voice, uint32_t frames) { + while (frames > 0u) { + const uint32_t stage = voice->mod_stage; + if (stage == AUDIODSP_SAMPLER_STAGE_SUSTAIN || + stage == AUDIODSP_SAMPLER_STAGE_IDLE) { + return; + } + uint32_t take = frames < voice->mod_left ? frames : voice->mod_left; + voice->mod_env += voice->mod_step * (float)take; + voice->mod_left -= take; + frames -= take; + if (voice->mod_left == 0u) { + switch (stage) { + case AUDIODSP_SAMPLER_STAGE_DELAY: + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_ATTACK); + break; + case AUDIODSP_SAMPLER_STAGE_ATTACK: + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_HOLD); + break; + case AUDIODSP_SAMPLER_STAGE_HOLD: + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_DECAY); + break; + case AUDIODSP_SAMPLER_STAGE_DECAY: + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_SUSTAIN); + break; + default: + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_IDLE); + break; + } + } + } +} + +// --- the filter and the LFOs ----------------------------------------------------- + +#define SAMPLER_PI 3.14159265f + +// tan(x) for 0 <= x < pi/2, as sine over cosine, each a Taylor series: the +// cutoff never goes past 0.45 of the output rate, so x stays under 1.42, +// where the dropped terms are below 2e-8. libm's tanf would differ in the +// last place between targets. +static float tan_poly(float x) { + const float x2 = x * x; + const float s = x * (1.0f + x2 * (-1.0f / 6.0f + x2 * (1.0f / 120.0f + + x2 * (-1.0f / 5040.0f + x2 * (1.0f / 362880.0f + + x2 * (-1.0f / 39916800.0f)))))); + const float c = 1.0f + x2 * (-0.5f + x2 * (1.0f / 24.0f + + x2 * (-1.0f / 720.0f + x2 * (1.0f / 40320.0f + + x2 * (-1.0f / 3628800.0f + x2 * (1.0f / 479001600.0f)))))); + return s / c; +} + +// 2^(cents / 1200). +static inline float cents_ratio(float cents) { + return audiodsp_exp2_poly(cents * (1.0f / 1200.0f)); +} + +// The trapezoidal state-variable low-pass's three coefficients, for the +// voice's cutoff moved by `cents`. +static void filter_coefficients(const audiodsp_sampler_t *self, + audiodsp_sampler_voice_t *voice, float cents) { + float fc = voice->cutoff; + if (cents != 0.0f) { + fc *= cents_ratio(cents); + } + const float top = 0.45f * (float)self->sample_rate; + if (fc > top) { + fc = top; + } else if (fc < 5.0f) { + fc = 5.0f; + } + const float g = tan_poly(SAMPLER_PI * fc / (float)self->sample_rate); + voice->fa1 = 1.0f / (1.0f + g * (g + voice->filter_k)); + voice->fa2 = g * voice->fa1; + voice->fa3 = g * voice->fa2; + voice->filter_cents = cents; + voice->filter_dirty = false; +} + +// Works out what the voice needs from its settings, and puts the plain +// path's step back when nothing moves the pitch any more. +static void update_mods(audiodsp_sampler_voice_t *voice) { + uint32_t mods = 0; + if (voice->mod_to_pitch != 0.0f || voice->vibrato_depth != 0.0f || + voice->lfo_to_pitch != 0.0f) { + mods |= AUDIODSP_SAMPLER_MOD_PITCH; + } + if (voice->mod_to_cutoff != 0.0f || voice->lfo_to_cutoff != 0.0f) { + mods |= AUDIODSP_SAMPLER_MOD_CUTOFF; + } + if (voice->lfo_to_volume != 0.0f) { + mods |= AUDIODSP_SAMPLER_MOD_VOLUME; + } + if (voice->mod_to_pitch != 0.0f || voice->mod_to_cutoff != 0.0f) { + mods |= AUDIODSP_SAMPLER_MOD_ENV; + } + if (voice->cutoff > 0.0f && (voice->cutoff < AUDIODSP_SAMPLER_OPEN_CUTOFF || + voice->resonance > 0.0f || (mods & AUDIODSP_SAMPLER_MOD_CUTOFF))) { + mods |= AUDIODSP_SAMPLER_MOD_FILTER; + } else { + mods &= ~AUDIODSP_SAMPLER_MOD_CUTOFF; + } + if ((mods & AUDIODSP_SAMPLER_MOD_FILTER) && + !(voice->mods & AUDIODSP_SAMPLER_MOD_FILTER)) { + voice->ic1 = 0.0f; + voice->ic2 = 0.0f; + voice->filter_dirty = true; + } + if (!(mods & AUDIODSP_SAMPLER_MOD_PITCH)) { + voice->step = voice->step_base; + } + voice->filter_gain = (mods & AUDIODSP_SAMPLER_MOD_FILTER) ? + audiodsp_exp2_poly(-voice->resonance * (0.5f * 0.16609640f)) : 1.0f; + voice->mods = mods; +} + +// The resonance as the filter's 1/Q. Q's response at the cutoff is Q itself, +// so Q = 10^(dB/20) would be the specification read literally; that is +// 1.0 at 0 dB, a small bump, where the specification says 0 dB is not +// resonant. sqrt(10^(dB/10) - 1/2) is Butterworth (0.707) at 0 dB, within +// 0.1 dB of the literal reading from 10 dB up, and smooth between. +static float resonance_k(float db) { + const float power = audiodsp_exp2_poly(db * 0.33219281f) - 0.5f; + // sqrt by Newton's method from a guess that is never short, so the + // result is the same bits everywhere. + float q = power > 1.0f ? power : 1.0f; + for (int n = 0; n < 24; ++n) { + q = 0.5f * (q + power / q); + } + return 1.0f / q; +} + +// A triangle from 0, rising, at the phase of `elapsed` frames. +static inline float triangle(uint32_t elapsed, uint32_t inc) { + const uint32_t phase = elapsed * inc; + const float t = (float)phase * (1.0f / 4294967296.0f); + if (t < 0.25f) { + return 4.0f * t; + } + if (t < 0.75f) { + return 2.0f - 4.0f * t; + } + return 4.0f * t - 4.0f; +} + +static uint32_t lfo_inc(float hz, uint32_t sample_rate) { + float cycles = hz / (float)sample_rate; + if (!(cycles > 0.0f)) { + return 0; + } + if (cycles > 0.5f) { + cycles = 0.5f; + } + return (uint32_t)((double)cycles * Q32_ONE); +} + +// `base` times `factor`, in Q32.32: the factor to 24 bits, about 1e-4 cents, +// with no double on the way (a board's FPU is single precision). +static inline uint64_t scale_step(uint64_t base, float factor) { + const uint64_t fq = (uint64_t)(factor * 16777216.0f + 0.5f); + return (((base >> 32) * fq) << 8) + (((base & 0xffffffffu) * fq) >> 24); +} + // --- the voice table ----------------------------------------------------------- static bool audible(const audiodsp_sampler_voice_t *voice) { @@ -228,6 +466,40 @@ bool audiodsp_sampler_play(audiodsp_sampler_t *self, uint32_t index, voice->pitch = clampf(note->pitch, AUDIODSP_SAMPLER_MIN_PITCH, AUDIODSP_SAMPLER_MAX_PITCH); voice->step = step_for(voice); + voice->step_base = voice->step; + voice->age = 0; + + voice->cutoff = note->cutoff > 0.0f ? note->cutoff : 0.0f; + voice->resonance = clampf(note->resonance, 0.0f, + AUDIODSP_SAMPLER_MAX_RESONANCE); + voice->filter_k = resonance_k(voice->resonance); + voice->mod_delay_samples = seconds_to_samples(note->mod_delay, + self->sample_rate); + voice->mod_attack_samples = seconds_to_samples(note->mod_attack, + self->sample_rate); + voice->mod_hold_samples = seconds_to_samples(note->mod_hold, + self->sample_rate); + voice->mod_decay_samples = seconds_to_samples(note->mod_decay, + self->sample_rate); + voice->mod_release_samples = seconds_to_samples(note->mod_release, + self->sample_rate); + voice->mod_sustain = clampf(note->mod_sustain, 0.0f, 1.0f); + const float most = AUDIODSP_SAMPLER_MAX_CENTS; + voice->mod_to_pitch = clampf(note->mod_to_pitch, -most, most); + voice->mod_to_cutoff = clampf(note->mod_to_cutoff, -most, most); + voice->vibrato_delay = seconds_to_samples(note->vibrato_delay, + self->sample_rate); + voice->vibrato_inc = lfo_inc(note->vibrato_rate, self->sample_rate); + voice->vibrato_depth = clampf(note->vibrato_depth, -most, most); + voice->lfo_delay = seconds_to_samples(note->lfo_delay, self->sample_rate); + voice->lfo_inc = lfo_inc(note->lfo_rate, self->sample_rate); + voice->lfo_to_pitch = clampf(note->lfo_to_pitch, -most, most); + voice->lfo_to_cutoff = clampf(note->lfo_to_cutoff, -most, most); + voice->lfo_to_volume = clampf(note->lfo_to_volume, -96.0f, 96.0f); + voice->mods = 0; + update_mods(voice); + voice->mod_gain = voice->filter_gain; + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_DELAY); voice->attack_samples = seconds_to_samples(note->attack, self->sample_rate); voice->hold_samples = seconds_to_samples(note->hold, self->sample_rate); @@ -274,6 +546,9 @@ void audiodsp_sampler_release(audiodsp_sampler_t *self, uint32_t index) { voice->looping = false; } enter_stage(voice, AUDIODSP_SAMPLER_STAGE_RELEASE); + if (voice->mod_stage != AUDIODSP_SAMPLER_STAGE_IDLE) { + mod_enter(voice, AUDIODSP_SAMPLER_STAGE_RELEASE); + } } void audiodsp_sampler_stop(audiodsp_sampler_t *self, uint32_t index) { @@ -330,6 +605,27 @@ void audiodsp_sampler_set_pitch(audiodsp_sampler_t *self, uint32_t index, voice->pitch = clampf(pitch, AUDIODSP_SAMPLER_MIN_PITCH, AUDIODSP_SAMPLER_MAX_PITCH); voice->step = step_for(voice); + voice->step_base = voice->step; + } +} + +void audiodsp_sampler_set_cutoff(audiodsp_sampler_t *self, uint32_t index, + float cutoff) { + if (index < self->voice_count) { + audiodsp_sampler_voice_t *voice = &self->voices[index]; + voice->cutoff = cutoff > 0.0f ? cutoff : 0.0f; + voice->filter_dirty = true; + update_mods(voice); + } +} + +void audiodsp_sampler_set_vibrato_depth(audiodsp_sampler_t *self, + uint32_t index, float cents) { + if (index < self->voice_count) { + audiodsp_sampler_voice_t *voice = &self->voices[index]; + voice->vibrato_depth = clampf(cents, -AUDIODSP_SAMPLER_MAX_CENTS, + AUDIODSP_SAMPLER_MAX_CENTS); + update_mods(voice); } } @@ -504,9 +800,17 @@ static uint32_t render_run(audiodsp_sampler_t *self, return i; } +static void render_voice_mod(audiodsp_sampler_t *self, + audiodsp_sampler_voice_t *voice); + AUDIODSP_HOT static void render_voice(audiodsp_sampler_t *self, audiodsp_sampler_voice_t *voice) { + if (voice->mods != 0u) { + render_voice_mod(self, voice); + return; + } + voice->age += self->frames; const uint32_t frames = self->frames; const uint32_t channels = self->channel_count; float *mix = self->mix; @@ -598,6 +902,213 @@ static void render_voice(audiodsp_sampler_t *self, voice->gain[1] = target[1]; } +// --- the modulated voice --------------------------------------------------------- + +// `run` clean frames from frame `i` into `out`, the plain path's numbers in +// its order, without the gains. +static uint32_t make_run(audiodsp_sampler_voice_t *voice, float *out, + uint32_t i, uint32_t run) { + const int16_t *data = voice->data; + const uint64_t step = voice->step; + const bool ramp = voice->stage != AUDIODSP_SAMPLER_STAGE_SUSTAIN; + const float env_step = voice->env_step; + uint64_t position = voice->position; + float env = voice->env; + const uint32_t end = i + run; + for (; i < end; ++i) { + out[i] = tap_run(data, position) * env; + if (ramp) { + env += env_step; + } + position += step; + } + voice->position = position; + voice->env = env; + if (ramp) { + voice->stage_left -= run; + } + return i; +} + +// `frames` frames of the voice, enveloped, into `out`: the plain path's +// per-frame rules (delay, taps, stages, wraps and the end), with silence +// after the voice stops. +static void make_frames(audiodsp_sampler_voice_t *voice, float *out, + uint32_t frames) { + uint32_t i = 0; + while (i < frames && voice->stage == AUDIODSP_SAMPLER_STAGE_DELAY) { + uint32_t run = frames - i; + if (run > voice->stage_left) { + run = voice->stage_left; + } + for (uint32_t j = 0; j < run; ++j) { + out[i + j] = 0.0f; + } + i += run; + voice->stage_left -= run; + if (voice->stage_left == 0u) { + enter_stage(voice, AUDIODSP_SAMPLER_STAGE_ATTACK); + } + } + const int16_t *data = voice->data; + for (; i < frames; ++i) { + if (voice->stage == AUDIODSP_SAMPLER_STAGE_IDLE) { + break; + } + uint32_t run = clean_run(voice, frames - i); + if (run >= 2u) { + i = make_run(voice, out, i, run); + if (i >= frames) { + break; + } + } + const uint32_t index = (uint32_t)(voice->position >> 32); + const float x = (float)(uint32_t)voice->position * + (1.0f / 4294967296.0f); + const uint32_t limit = voice->looping ? voice->loop_end : voice->length; + float sample; + if (index >= 1u && index + 2u < limit) { + sample = hermite((float)data[index - 1u], (float)data[index], + (float)data[index + 1u], (float)data[index + 2u], x); + } else { + sample = hermite(tap(voice, (int64_t)index - 1), + tap(voice, (int64_t)index), tap(voice, (int64_t)index + 1), + tap(voice, (int64_t)index + 2), x); + } + out[i] = sample * voice->env; + + if (voice->stage != AUDIODSP_SAMPLER_STAGE_SUSTAIN) { + voice->env += voice->env_step; + if (--voice->stage_left == 0u) { + stage_done(voice); + if (voice->stage == AUDIODSP_SAMPLER_STAGE_IDLE) { + ++i; + break; + } + } + } + + voice->position += voice->step; + if (voice->looping) { + wrap(voice); + } else if ((voice->position >> 32) >= voice->length) { + enter_stage(voice, AUDIODSP_SAMPLER_STAGE_IDLE); + ++i; + break; + } + } + for (; i < frames; ++i) { + out[i] = 0.0f; + } +} + +// Sets the chunk's step, filter and target volume from the modulators where +// the voice is now, and returns the target volume. +static float modulate(const audiodsp_sampler_t *self, + audiodsp_sampler_voice_t *voice) { + const uint32_t mods = voice->mods; + const uint32_t age = voice->age; + const float env = (mods & AUDIODSP_SAMPLER_MOD_ENV) ? voice->mod_env : 0.0f; + const float vibrato = (voice->vibrato_inc != 0u && + age >= voice->vibrato_delay) ? + triangle(age - voice->vibrato_delay, voice->vibrato_inc) : 0.0f; + const float lfo = (voice->lfo_inc != 0u && age >= voice->lfo_delay) ? + triangle(age - voice->lfo_delay, voice->lfo_inc) : 0.0f; + if (mods & AUDIODSP_SAMPLER_MOD_PITCH) { + float cents = env * voice->mod_to_pitch + + vibrato * voice->vibrato_depth + lfo * voice->lfo_to_pitch; + cents = clampf(cents, -7200.0f, 7200.0f); + voice->step = cents == 0.0f ? voice->step_base : + scale_step(voice->step_base, cents_ratio(cents)); + } + if (mods & AUDIODSP_SAMPLER_MOD_FILTER) { + float cents = 0.0f; + if (mods & AUDIODSP_SAMPLER_MOD_CUTOFF) { + cents = clampf(env * voice->mod_to_cutoff + + lfo * voice->lfo_to_cutoff, -AUDIODSP_SAMPLER_MAX_CENTS, + AUDIODSP_SAMPLER_MAX_CENTS); + } + if (voice->filter_dirty || cents != voice->filter_cents) { + filter_coefficients(self, voice, cents); + } + } + float gain = voice->filter_gain; + if (mods & AUDIODSP_SAMPLER_MOD_VOLUME) { + // dB to a ratio: 2^(dB * log2(10) / 20). + gain *= audiodsp_exp2_poly(lfo * voice->lfo_to_volume * 0.16609640f); + } + return gain; +} + +AUDIODSP_HOT +static void render_voice_mod(audiodsp_sampler_t *self, + audiodsp_sampler_voice_t *voice) { + const uint32_t frames = self->frames; + const uint32_t channels = self->channel_count; + float *mix = self->mix; + float row[AUDIODSP_SAMPLER_CONTROL]; + + float target[2]; + target_gains(voice, channels, target); + float g0 = voice->gain[0]; + float g1 = voice->gain[1]; + const float per_frame = 1.0f / (float)frames; + const float dg0 = (target[0] - g0) * per_frame; + const float dg1 = (target[1] - g1) * per_frame; + + for (uint32_t c = 0; c < frames;) { + if (voice->stage == AUDIODSP_SAMPLER_STAGE_IDLE) { + break; + } + uint32_t n = frames - c; + if (n > AUDIODSP_SAMPLER_CONTROL) { + n = AUDIODSP_SAMPLER_CONTROL; + } + const float want = modulate(self, voice); + make_frames(voice, row, n); + if (voice->mods & AUDIODSP_SAMPLER_MOD_FILTER) { + const float a1 = voice->fa1, a2 = voice->fa2, a3 = voice->fa3; + float ic1 = voice->ic1, ic2 = voice->ic2; + for (uint32_t j = 0; j < n; ++j) { + const float v3 = row[j] - ic2; + const float v1 = a1 * ic1 + a2 * v3; + const float v2 = ic2 + a2 * ic1 + a3 * v3; + ic1 = 2.0f * v1 - ic1; + ic2 = 2.0f * v2 - ic2; + row[j] = v2; + } + voice->ic1 = ic1; + voice->ic2 = ic2; + } + float gain = voice->mod_gain; + const float dgain = (want - gain) / (float)n; + if (channels == 2u) { + float *out = mix + 2u * c; + for (uint32_t j = 0; j < n; ++j) { + const float a = row[j] * gain; + out[2u * j] += a * g0; + out[2u * j + 1u] += a * g1; + g0 += dg0; + g1 += dg1; + gain += dgain; + } + } else { + float *out = mix + c; + for (uint32_t j = 0; j < n; ++j) { + out[j] += row[j] * gain * g0; + g0 += dg0; + gain += dgain; + } + } + voice->mod_gain = want; + voice->age += n; + mod_advance(voice, n); + c += n; + } + voice->gain[0] = target[0]; + voice->gain[1] = target[1]; +} + AUDIODSP_HOT uint32_t audiodsp_sampler_render_s16(audiodsp_sampler_t *self, int16_t *out) { const uint32_t words = self->frames * self->channel_count; diff --git a/src/shared/audiodsp_sampler.h b/src/shared/audiodsp_sampler.h index 4aa291e1..cd08ff5c 100644 --- a/src/shared/audiodsp_sampler.h +++ b/src/shared/audiodsp_sampler.h @@ -54,6 +54,34 @@ // slot per voice rather than a pool so the binding knows, without asking, // which buffer each one is reading. // +// WHAT A SOUNDFONT VOICE ADDS. A SoundFont region asks for more than an +// amplitude envelope: a resonant low-pass, a second (modulation) envelope +// that sweeps cutoff and pitch, a vibrato LFO on pitch and a modulation LFO +// on pitch, cutoff and volume (SoundFont 2.04, section 8.1.2). A voice that +// asks for none of them -- no cutoff below the open one, no resonance, every +// depth zero -- takes the path above, untouched, and costs what it did +// before any of this existed; the one addition is a counter of its age. +// +// A voice that asks for any of them renders in chunks of +// AUDIODSP_SAMPLER_CONTROL frames. The modulators are worked out once a +// chunk, at its start: the modulation envelope and both LFOs, the pitch they +// add up to, and the cutoff. The samples of the chunk are made as above into +// a scratch row, filtered, and mixed. Control rate is 1.5 kHz at 48 kHz, +// well above the fastest LFO a bank can ask for (about 100 Hz). +// +// The filter is a two-pole low-pass, the trapezoidal state-variable form, so +// a cutoff that moves every chunk never makes it unstable. Its response at +// the cutoff is the resonance asked for, in dB above its response at DC, and +// the DC response is lowered by half the resonance, as the specification +// says. Zero resonance is Butterworth. The cutoff's tangent and every 2^x are +// polynomials, so three interpreters on three architectures render the same +// bits (see audiodsp_exp2.h). +// +// The LFOs are triangles from 0, rising, and start after their delay. Their +// phase is the voice's age times their rate, in 32-bit integer arithmetic, +// so a vibrato's depth turned up from zero halfway through a note picks up +// where it would have been all along. +// // The pulling loop is not here, for the same reason it is not in // audiodsp_modal.c: each runtime reaches its audio graph differently. Nor is // any memory: the binding owns the voice table, the mix accumulator and the @@ -101,6 +129,21 @@ //: voice, then each voice's tail, which plays out a stopped note's fade. #define AUDIODSP_SAMPLER_SLOTS(voice_count) (2u * (voice_count)) +//: Frames between two updates of a modulated voice's envelope, LFOs, pitch +//: and filter. Unmodulated voices never see it. +#define AUDIODSP_SAMPLER_CONTROL 32u + +//: A cutoff at or above this, in Hz, with no resonance and nothing moving it, +//: is no filter at all: the SoundFont default of 13500 cents is 19.9 kHz, and +//: the specification says such a filter "does not affect the signal". +#define AUDIODSP_SAMPLER_OPEN_CUTOFF 19900.0f + +//: The resonance ceiling, in dB: the specification's 960 centibels. +#define AUDIODSP_SAMPLER_MAX_RESONANCE 96.0f + +//: The most a modulator may move pitch or cutoff, either way, in cents. +#define AUDIODSP_SAMPLER_MAX_CENTS 12000.0f + //: How a voice treats `[loop_start, loop_end)`. The numbers are SoundFont's //: `sampleModes` so an instrument reader writes the file's value straight //: in; 2 is unused there and is refused here. @@ -132,8 +175,35 @@ typedef struct { float pan; // -1 (left) .. 1 (right); ignored for mono output float delay, attack, hold, decay, release; // seconds float sustain; // 0..1, linear + // The low-pass. cutoff 0 (or at or above AUDIODSP_SAMPLER_OPEN_CUTOFF + // with no resonance and nothing moving it) is no filter. + float cutoff; // Hz + float resonance; // dB at the cutoff above DC, 0..96 + // The modulation envelope, in the amplitude envelope's units, and how + // far it moves pitch and cutoff at full scale. + float mod_delay, mod_attack, mod_hold, mod_decay, mod_release; // seconds + float mod_sustain; // 0..1, linear + float mod_to_pitch; // cents + float mod_to_cutoff; // cents + // The vibrato LFO: pitch only. + float vibrato_delay; // seconds + float vibrato_rate; // Hz + float vibrato_depth; // cents + // The modulation LFO: pitch, cutoff and volume. + float lfo_delay; // seconds + float lfo_rate; // Hz + float lfo_to_pitch; // cents + float lfo_to_cutoff; // cents + float lfo_to_volume; // dB at the LFO's peak } audiodsp_sampler_note_t; +//: What a voice's `mods` says it needs, besides the plain path. +#define AUDIODSP_SAMPLER_MOD_FILTER 1u // the low-pass runs +#define AUDIODSP_SAMPLER_MOD_PITCH 2u // something moves the pitch +#define AUDIODSP_SAMPLER_MOD_CUTOFF 4u // something moves the cutoff +#define AUDIODSP_SAMPLER_MOD_VOLUME 8u // the LFO moves the volume +#define AUDIODSP_SAMPLER_MOD_ENV 16u // the modulation envelope is used + //: One voice. Written by the kernel only; a binding reads `stage` to answer //: `playing` and to know when to let go of the buffer it handed over. typedef struct { @@ -161,6 +231,40 @@ typedef struct { float gain[2]; // Where a run of clean frames left the gains, mid-block (render only). float gain_run[2]; + // Frames since play(), for the LFOs. Wraps after a day at 48 kHz. + uint32_t age; + // AUDIODSP_SAMPLER_MOD_*: zero for a voice on the plain path. + uint32_t mods; + // The step pitch alone asks for; a modulated voice scales it each chunk. + uint64_t step_base; + // The low-pass: its cutoff (Hz) and 1/Q, the DC gain the resonance + // asks for, its two states, its coefficients, and the cutoff offset + // (cents) they were worked out for. + float cutoff; + float resonance; + float filter_k; + float filter_gain; + float ic1, ic2; + float fa1, fa2, fa3; + float filter_cents; + bool filter_dirty; + // The modulation envelope, as the amplitude one. + uint32_t mod_stage; + float mod_env; + float mod_step; + uint32_t mod_left; + uint32_t mod_delay_samples, mod_attack_samples, mod_hold_samples, + mod_decay_samples, mod_release_samples; + float mod_sustain; + float mod_to_pitch, mod_to_cutoff; + // The two LFOs: delay in frames, phase step per frame (Q0.32), depths. + uint32_t vibrato_delay, vibrato_inc; + float vibrato_depth; + uint32_t lfo_delay, lfo_inc; + float lfo_to_pitch, lfo_to_cutoff, lfo_to_volume; + // The volume factor the last chunk ended on (the filter's DC gain times + // the LFO's tremolo), so the next ramps from it. + float mod_gain; } audiodsp_sampler_voice_t; typedef struct { @@ -223,6 +327,13 @@ void audiodsp_sampler_set_pan(audiodsp_sampler_t *self, uint32_t index, float pan); void audiodsp_sampler_set_pitch(audiodsp_sampler_t *self, uint32_t index, float pitch); +// The cutoff (Hz) and the vibrato depth (cents) land at the next chunk. A +// cutoff that opens the filter fully takes the voice back to the plain path; +// one that closes it starts the filter from rest. +void audiodsp_sampler_set_cutoff(audiodsp_sampler_t *self, uint32_t index, + float cutoff); +void audiodsp_sampler_set_vibrato_depth(audiodsp_sampler_t *self, + uint32_t index, float cents); // Mixes every sounding voice, and every fading tail, into `out`: `frames` // interleaved frames of `channel_count` signed 16-bit samples. Silent when diff --git a/tests/parity/sampler_cost.py b/tests/parity/sampler_cost.py new file mode 100644 index 00000000..c0122cea --- /dev/null +++ b/tests/parity/sampler_cost.py @@ -0,0 +1,107 @@ +"""What a block of audiosampler costs, by voice count and by what each voice does. + + sampler_cost.py [voices ...] + +Runs on CPython (with `_audiodsp` built) and MicroPython (desktop or board), +so one script times the desktop and a board alike. Every voice loops a +2,048-frame buffer at 48 kHz, stereo, 256-frame blocks, at a pitch of 1.06 +so the interpolator works, and every voice is sounding. Three cases: + +- plain: no filter, no LFO -- the path a voice took before any of these + existed. +- filter: a 2 kHz low-pass with 6 dB of resonance, nothing moving it. +- full: the filter plus the modulation envelope on cutoff and pitch, the + vibrato LFO and the modulation LFO on pitch, cutoff and volume. + +It prints the best of five runs of 200 blocks, in microseconds per block, +beside the block's own length. +""" + +import sys +from array import array + +import audiocore +import audiosampler + +try: + from time import ticks_diff, ticks_us + + def now(): + return ticks_us() + + def elapsed(a, b): + return ticks_diff(b, a) +except ImportError: + from time import perf_counter + + def now(): + return perf_counter() + + def elapsed(a, b): + return (b - a) * 1e6 + +RATE = 48000 +BLOCKS = 200 +LENGTH = 2048 + +CASES = { + "plain": {}, + "filter": {"cutoff": 2000.0, "resonance": 6.0}, + "full": {"cutoff": 2000.0, "resonance": 6.0, "mod_attack": 0.01, + "mod_decay": 0.5, "mod_sustain": 0.3, "mod_to_pitch": 50.0, + "mod_to_cutoff": 2400.0, "vibrato_rate": 5.0, + "vibrato_depth": 20.0, "lfo_rate": 3.0, "lfo_to_pitch": 10.0, + "lfo_to_cutoff": 600.0, "lfo_to_volume": 3.0}, +} + + +def tone(): + # A sum of three integer triangles, so it has partials for a filter to + # take, and no math module is needed. + data = array("h", [0] * LENGTH) + for i in range(LENGTH): + a = (i * 7) % 256 + b = (i * 31) % 128 + c = (i * 3) % 512 + data[i] = (abs(a - 128) - 64) * 60 + (abs(b - 64) - 32) * 50 + ( + abs(c - 256) - 128) * 20 + return data + + +def cost(voices, case, data): + sampler = audiosampler.Sampler(voice_count=voices, channel_count=2, + sample_rate=RATE) + for v in range(voices): + sampler.voice[v].play(data, pitch=1.06, loop_start=0, + loop_end=LENGTH, + loop_mode=audiosampler.LOOP_FOREVER, + level=0.02, **CASES[case]) + for _ in range(4): + audiocore.get_buffer(sampler) + best = None + for _ in range(5): + start = now() + for _ in range(BLOCKS): + audiocore.get_buffer(sampler) + us = elapsed(start, now()) / BLOCKS + if best is None or us < best: + best = us + sampler.deinit() + return best + + +def main(): + counts = [int(a) for a in sys.argv[1:]] or [32, 48, 64] + data = tone() + print("block %d us (256 frames at %d Hz)" % (256 * 1000000 // RATE, RATE)) + for voices in counts: + row = [] + for case in ("plain", "filter", "full"): + try: + row.append("%s %.0f" % (case, cost(voices, case, data))) + except TypeError: # an audiosampler from before the filter + row.append("%s -" % case) + print("%3d voices: %s us" % (voices, ", ".join(row))) + + +main() diff --git a/tests/parity/sampler_probe.py b/tests/parity/sampler_probe.py index 8afa2275..6866589d 100644 --- a/tests/parity/sampler_probe.py +++ b/tests/parity/sampler_probe.py @@ -18,6 +18,8 @@ sounded: the fade is 5 ms, 40 frames at 8 kHz, so 39 is the claim and 255 would be no fade at all, a cut a block late. Its PCM, and that of a `play()` over a sounding voice, is emitted beside it. +- **modulated** is PCM from voices with the filter, the modulation + envelope and both LFOs, and from a filter and a vibrato set live. - **held** prints how many frames of the buffer's tail were heard while a mode-3 voice was still held: zero is the claim. @@ -194,6 +196,32 @@ def main(): ends() fade() choke() + modulated() + + +def modulated(): + # The filter, the modulation envelope and both LFOs on one voice, and a + # filter moved live on another: the cutoff's tangent and every 2^x are + # polynomials so that this is the same bits everywhere. Times are whole + # samples at 8 kHz and depths whole cents. + sampler = build(2, 2) + sampler.voice[0].play(triangle(), pitch=1.5, + loop_mode=sampler_module.LOOP_FOREVER, + cutoff=1000.0, resonance=9.0, mod_attack=0.0625, + mod_decay=0.125, mod_sustain=0.25, + mod_to_pitch=200.0, mod_to_cutoff=1200.0, + vibrato_delay=0.03125, vibrato_rate=6.0, + vibrato_depth=30.0, lfo_rate=2.0, lfo_to_pitch=10.0, + lfo_to_cutoff=300.0, lfo_to_volume=3.0) + sampler.voice[1].play(triangle(), pitch=0.75, pan=0.5, + loop_mode=sampler_module.LOOP_FOREVER) + emit("modulated", sampler, 6) + sampler.voice[1].cutoff = 500.0 + sampler.voice[1].vibrato_depth = 40.0 + emit("closed", sampler, 4) + sampler.voice[0].release() + sampler.voice[1].cutoff = None + emit("opened", sampler, 3) main() diff --git a/tests/test_cpython_audiosampler_modulation.py b/tests/test_cpython_audiosampler_modulation.py new file mode 100644 index 00000000..e05a8190 --- /dev/null +++ b/tests/test_cpython_audiosampler_modulation.py @@ -0,0 +1,317 @@ +"""audiosampler's filter, modulation envelope and LFOs, measured. + +What a SoundFont region asks of a voice besides its amplitude envelope +(SoundFont 2.04, section 8.1.2): a resonant low-pass, a modulation envelope +on pitch and cutoff, a vibrato LFO on pitch, and a modulation LFO on pitch, +cutoff and volume. `test_cpython_audiosampler.py` holds the plain voice's +traits; these are the ones a modulated voice adds. + +## The traits, with their bars + +| ID | Trait | Bar | +|---|---|---| +| M1 | A voice with the filter open (19.9 kHz, no resonance, nothing moving it) renders the same bits as one with no filter | exact | +| M2 | At its cutoff the filter answers 20*log10(Q) - R/2 dB, where R is the resonance and Q = sqrt(10^(R/10) - 1/2); far below it, -R/2 dB | 0.3 dB | +| M3 | Two octaves above the cutoff the filter is down about 24 dB | 21 to 27 dB | +| M4 | The modulation envelope sweeps pitch: at full scale up by `mod_to_pitch`, then down linearly in cents through the decay to the sustain | 1% | +| M5 | The modulation envelope sweeps the cutoff: two octaves of `mod_to_cutoff` from 500 Hz lets a 2 kHz tone through as a 2 kHz cutoff would | 0.5 dB | +| M6 | The vibrato LFO swings pitch by its depth either way at its rate, and not before its delay | depth 5%, rate 2%, delay one chunk | +| M7 | The modulation LFO swings volume by its depth in dB either way at its rate | 0.5 dB, rate 2% | +| M8 | A vibrato depth set live from zero starts the vibrato; set back to zero, the pitch is exactly the plain voice's again | 5%; exact | +| M9 | A cutoff set live closes the filter on a sounding voice, and set to None opens it again | 0.3 dB; 0.05 dB | +| M10 | 32, 48 and 64 looping stereo voices, every filter on, render inside the 5333 us block | measured | + +## The planted faults + +Recorded rather than automated, because the file they break is C. Each was +made by hand in `src/shared/audiodsp_sampler.c`, rebuilt, and the suite run. + +| Fault | Breaks | What was measured | +|---|---|---| +| `resonance_k()` works out Q for half the resonance asked | M2 | 500 Hz with 6 dB of resonance answered -1.25 dB at the cutoff against 2.42. Measured 2026-10-29. | +| `triangle()` runs at twice the phase | M6, M7 | the vibrato measured 10.0 Hz against 5; the tremolo's peaks came every 0.125 s against 0.25. | +| `mod_advance()` moves the modulation envelope at half speed | M4 | 0.1 s into a one-second sweep from an octave up, 966 Hz against 933; half-way through a release, 841 Hz against 707. | +""" + +import math +import time +import unittest +from array import array + +import audiocore +import audiosampler + +RATE = 48000 +BLOCK = audiosampler.FRAMES + + +def sine(hz, frames, peak=16000): + """A loop of `frames` holding a whole number of cycles of `hz`.""" + return array("h", [int(round(peak * math.sin(2 * math.pi * hz * i / RATE))) + for i in range(frames)]) + + +def mono(): + return audiosampler.Sampler(voice_count=1, channel_count=1, + sample_rate=RATE) + + +def render(sampler, blocks): + out = [] + for _ in range(blocks): + _, data = audiocore.get_buffer(sampler) + out.extend(array("h", bytes(data))) + return out + + +def rms(samples): + return math.sqrt(sum(v * v for v in samples) / float(len(samples))) + + +def db(a, b): + return 20.0 * math.log10(a / b) + + +def crossings(samples): + """Fractional positions of the rising zero crossings.""" + out = [] + for i in range(1, len(samples)): + a, b = samples[i - 1], samples[i] + if a < 0 <= b: + out.append(i - 1 + (-a) / float(b - a)) + return out + + +def frequency_track(samples): + """(time in frames, Hz) per rising crossing, from the period around it.""" + xs = crossings(samples) + return [((xs[i] + xs[i - 1]) / 2.0, RATE / (xs[i] - xs[i - 1])) + for i in range(1, len(xs))] + + +def hz_between(samples, start, end): + xs = [x for x in crossings(samples) if start <= x < end] + return (len(xs) - 1) * RATE / (xs[-1] - xs[0]) + + +def loop(voice, data, **kw): + voice.play(data, loop_start=0, loop_end=len(data), + loop_mode=audiosampler.LOOP_FOREVER, **kw) + + +def through(hz, blocks=60, **kw): + """RMS of a looped `hz` sine through a voice with `kw`, after a settle.""" + sampler = mono() + frames = int(round(RATE / hz * 40)) + # Quiet enough that 20 dB of resonance stays clear of the rails. + data = sine(RATE * 40.0 / frames, frames, peak=3000) + loop(sampler.voice[0], data, **kw) + out = render(sampler, blocks) + sampler.deinit() + return rms(out[20 * BLOCK:]), RATE * 40.0 / frames + + +def expected_db(resonance): + q = math.sqrt(10 ** (resonance / 10.0) - 0.5) + return 20 * math.log10(q) - resonance / 2.0 + + +class Filter(unittest.TestCase): + + def test_m1_open_filter_is_the_plain_voice(self): + data = sine(1000, 480) + outs = [] + for kw in ({}, {"cutoff": 19900.0}, {"cutoff": 22000.0}, + {"cutoff": None}): + sampler = mono() + loop(sampler.voice[0], data, pitch=1.07, attack=0.01, **kw) + outs.append(render(sampler, 20)) + for other in outs[1:]: + self.assertEqual(outs[0], other) + + def test_m2_response_at_and_below_the_cutoff(self): + for cutoff in (500.0, 2000.0, 6000.0): + for resonance in (0.0, 6.0, 12.0, 20.0): + plain, hz = through(cutoff) + # The filter's cutoff is put on the tone's exact frequency. + at, _ = through(cutoff, cutoff=hz, resonance=resonance) + self.assertAlmostEqual( + db(at, plain), expected_db(resonance), delta=0.3, + msg="cutoff %g, resonance %g" % (cutoff, resonance)) + low, lhz = through(cutoff / 16) + below, _ = through(cutoff / 16, cutoff=lhz * 16, + resonance=resonance) + self.assertAlmostEqual(db(below, low), -resonance / 2.0, + delta=0.3) + + def test_m3_two_octaves_up_is_about_24_db_down(self): + plain, hz = through(4000.0) + above, _ = through(4000.0, cutoff=hz / 4) + self.assertTrue(-27 < db(above, plain) < -21, db(above, plain)) + + +class ModulationEnvelope(unittest.TestCase): + + def test_m4_sweeps_pitch(self): + sampler = mono() + data = sine(500, 960) + loop(sampler.voice[0], data, mod_decay=1.0, mod_sustain=0.0, + mod_to_pitch=1200.0) + out = render(sampler, int(1.3 * RATE / BLOCK)) + for at in (0.1, 0.25, 0.5, 0.75): + got = hz_between(out, at * RATE - 600, at * RATE + 600) + want = 500 * 2 ** (1.0 - at) + self.assertAlmostEqual(got, want, delta=want * 0.01, + msg="at %g s" % at) + got = hz_between(out, 1.1 * RATE, 1.25 * RATE) + self.assertAlmostEqual(got, 500, delta=5) + + def test_m4_release_takes_pitch_back(self): + sampler = mono() + loop(sampler.voice[0], sine(500, 960), mod_to_pitch=1200.0, + mod_release=0.5, release=2.0) + out = render(sampler, 20) + self.assertAlmostEqual(hz_between(out, 1000, 5000), 1000, delta=10) + sampler.voice[0].release() + out = render(sampler, int(0.4 * RATE / BLOCK)) + # 0.25 s into a 0.5 s release: half the envelope, half an octave. + mid = 0.25 * RATE + self.assertAlmostEqual(hz_between(out, mid - 600, mid + 600), + 500 * 2 ** 0.5, delta=10) + + def test_m5_sweeps_the_cutoff(self): + tone, hz = through(2000.0) + shut, _ = through(2000.0, cutoff=hz / 4) + swept, _ = through(2000.0, cutoff=hz / 4, mod_to_cutoff=2400.0) + self.assertAlmostEqual(db(swept, tone), expected_db(0.0), delta=0.5) + self.assertLess(db(shut, tone), -20) + + +class Lfos(unittest.TestCase): + + def test_m6_vibrato_rate_depth_and_delay(self): + sampler = mono() + loop(sampler.voice[0], sine(1500, 960), vibrato_delay=0.1, + vibrato_rate=5.0, vibrato_depth=100.0) + out = render(sampler, int(1.2 * RATE / BLOCK)) + track = frequency_track(out) + before = [f for t, f in track if t < 0.1 * RATE - 64] + self.assertAlmostEqual(min(before), 1500, delta=1) + self.assertAlmostEqual(max(before), 1500, delta=1) + after = [(t, f) for t, f in track if t > 0.1 * RATE + 64] + cents = [1200 * math.log(f / 1500.0, 2) for _, f in after] + self.assertAlmostEqual(max(cents), 100, delta=5) + self.assertAlmostEqual(min(cents), -100, delta=5) + # The rate: upward crossings of zero cents, over a second. + ups = [after[i][0] for i in range(1, len(after)) + if cents[i - 1] < 0 <= cents[i]] + period = (ups[-1] - ups[0]) / (len(ups) - 1) / RATE + self.assertAlmostEqual(1 / period, 5.0, delta=0.1) + # The first thing after the delay is up: a triangle from 0, rising. + first = [c for (t, f), c in zip(after, cents) + if t < 0.1 * RATE + 2400] + self.assertGreater(sum(first), 0) + + def test_m7_tremolo(self): + sampler = mono() + loop(sampler.voice[0], sine(1500, 960), lfo_rate=4.0, + lfo_to_volume=6.0) + out = render(sampler, int(1.1 * RATE / BLOCK)) + window = 160 + levels = [] + for start in range(0, len(out) - window, window): + levels.append(rms(out[start:start + window])) + quiet = min(levels[30:]) + loud = max(levels[30:]) + plain = 16000 / math.sqrt(2) + self.assertAlmostEqual(db(loud, plain), 6.0, delta=0.5) + self.assertAlmostEqual(db(quiet, plain), -6.0, delta=0.5) + # Rate: the loud windows come every 0.25 s. + peaks = [i for i in range(1, len(levels) - 1) + if levels[i] >= levels[i - 1] and levels[i] > levels[i + 1] + and db(levels[i], plain) > 5.0] + gaps = [peaks[i] - peaks[i - 1] for i in range(1, len(peaks))] + gaps = [g for g in gaps if g > 10] + mean = sum(gaps) * window / float(len(gaps)) / RATE + self.assertAlmostEqual(mean, 0.25, delta=0.005) + + def test_m7_lfo_on_pitch(self): + sampler = mono() + loop(sampler.voice[0], sine(1500, 960), lfo_rate=6.0, + lfo_to_pitch=50.0) + out = render(sampler, int(0.6 * RATE / BLOCK)) + cents = [1200 * math.log(f / 1500.0, 2) + for t, f in frequency_track(out)] + self.assertAlmostEqual(max(cents), 50, delta=3) + self.assertAlmostEqual(min(cents), -50, delta=3) + + def test_m8_live_vibrato_depth(self): + data = sine(1500, 960) + plain = mono() + loop(plain.voice[0], data, pitch=1.03) + sampler = mono() + loop(sampler.voice[0], data, pitch=1.03, vibrato_rate=5.0) + first = render(sampler, 40) + self.assertEqual(first, render(plain, 40)) + sampler.voice[0].vibrato_depth = 50.0 + self.assertEqual(sampler.voice[0].vibrato_depth, 50.0) + swung = render(sampler, int(0.5 * RATE / BLOCK)) + render(plain, int(0.5 * RATE / BLOCK)) + cents = [1200 * math.log(f / (1500.0 * 1.03), 2) + for t, f in frequency_track(swung)] + self.assertAlmostEqual(max(cents), 50, delta=3) + sampler.voice[0].vibrato_depth = 0.0 + render(sampler, 1) + render(plain, 1) + # The step is the plain voice's again; the position may differ, so + # compare frequency rather than bits. + back = render(sampler, 40) + steady = [f for t, f in frequency_track(back)] + self.assertAlmostEqual(min(steady), 1545, delta=0.5) + self.assertAlmostEqual(max(steady), 1545, delta=0.5) + + def test_m9_live_cutoff(self): + tone_rms, hz = through(2000.0) + data = sine(hz, int(round(RATE / hz * 40)), peak=3000) + sampler = mono() + loop(sampler.voice[0], data) + render(sampler, 10) + sampler.voice[0].cutoff = hz + self.assertEqual(sampler.voice[0].cutoff, hz) + out = render(sampler, 40) + self.assertAlmostEqual(db(rms(out[20 * BLOCK:]), tone_rms), + expected_db(0.0), delta=0.3) + sampler.voice[0].cutoff = None + self.assertIsNone(sampler.voice[0].cutoff) + render(sampler, 1) + # Open again: the tone comes through at its own level. + out = render(sampler, 20) + self.assertAlmostEqual(db(rms(out), tone_rms), 0.0, delta=0.05) + + +class Cost(unittest.TestCase): + + def test_m10_filtered_voices_inside_the_block(self): + data = sine(1000, 2400) + budget = BLOCK / RATE * 1e6 + for voices in (32, 48, 64): + sampler = audiosampler.Sampler(voice_count=voices, + channel_count=2, sample_rate=RATE) + for v in range(voices): + loop(sampler.voice[v], data, pitch=1.06, level=0.02, + cutoff=2000.0, resonance=6.0) + render(sampler, 4) + best = float("inf") + for _ in range(3): + started = time.perf_counter() + for _ in range(100): + audiocore.get_buffer(sampler) + best = min(best, (time.perf_counter() - started) / 100 * 1e6) + print("\n%d filtered stereo voices: %.1f us per block " + "(budget %.0f us)" % (voices, best, budget)) + self.assertLess(best, budget) + sampler.deinit() + + +if __name__ == "__main__": + unittest.main() From 9f078c4d37a7909e8a37d1a07ae9ad674e5ec4fb Mon Sep 17 00:00:00 2001 From: Brad Barnett <127794626+bdbarnett@users.noreply.github.com> Date: Thu, 29 Oct 2026 15:05:50 -0500 Subject: [PATCH 2/2] docs: what the sampler's filter and LFOs cost on an ESP32-P4 --- docs/guides/play-recorded-notes.md | 8 +++++--- docs/how-it-works/upstream-diff.md | 14 +++++++++++++- 2 files changed, 18 insertions(+), 4 deletions(-) diff --git a/docs/guides/play-recorded-notes.md b/docs/guides/play-recorded-notes.md index ba665ceb..188c118f 100644 --- a/docs/guides/play-recorded-notes.md +++ b/docs/guides/play-recorded-notes.md @@ -78,9 +78,11 @@ LFOs are triangles that start at zero, rising, once their delay is over. `cutoff` and `vibrato_depth` are live properties. These are worked out every 32 frames, which at 48 kHz is about fourteen -times a cycle of the fastest LFO a SoundFont can ask for (110 Hz). A voice that uses none of them -renders exactly as before and costs the same. One that does costs about -1.7 times as much on a desktop; the numbers for a board are in +times a cycle of the fastest LFO a SoundFont can ask for (110 Hz). A voice +that uses none of them renders exactly as before and costs the same. One +with a filter costs about 1.7 times as much on a desktop and twice as much +on an ESP32-P4, where 48 filtered voices fit a 256-frame block and 64 only +just do: see [what the filter and the LFOs cost](../how-it-works/upstream-diff.md#what-the-filter-and-the-lfos-cost). ## How it renders diff --git a/docs/how-it-works/upstream-diff.md b/docs/how-it-works/upstream-diff.md index a6a91de6..96e3ced1 100644 --- a/docs/how-it-works/upstream-diff.md +++ b/docs/how-it-works/upstream-diff.md @@ -111,7 +111,19 @@ pitch, cutoff and volume. (x86-64, one core, gcc -O2.) -P4_TABLE +On an ESP32-P4 at 400 MHz (MicroPython, the script run over the REPL): + +| Voices | plain before | plain | filter | full | +|---|---|---|---|---| +| 32 | 1383 | 1384 | 2626 | 3465 | +| 48 | 2039 | 2040 | 3904 | 5179 | +| 64 | 2745 | 2804 | 5290 | 6970 | + +*Plain before* is the same firmware built from the commit before the +filter. A board fits 48 filtered voices with room to spare, and 64 with +almost none; with every LFO and the modulation envelope moving as well, 48 +still fit and 64 don't. A bank rarely asks for all of it at once on every +voice, but size a board's `voice_count` with this table in mind. ### How it is verified