Repository navigation
audiofeatures: speech features for a wake word model - #334
Merged
Merged
Conversation
… rate TensorFlow Lite Micro's audio microfrontend with microWakeWord's settings, carried in third_party with the kissfft it calls and built as one C translation unit. 16 kHz audio gives the reference build's bytes exactly; other rates go through a windowed-sinc resampler to 16 kHz first. feed() from Python anywhere, attach() to an audiopump.Tap on a board. Not built into CircuitPython, like audiometer.
The resampler's taps came from sin(), which may differ in the last bit between C libraries, and the unix MicroPython build in CI made different 48 kHz features from the CPython wheel. sin(pi x) is now a series in exact IEEE arithmetic, and both files take audiodsp_fp_contract.h like the rest of src/shared. The symbol test needs GNU nm, so it runs on Linux only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A new module,
audiofeatures: every 10 ms of audio becomes a slice of 40 mel-band values, the input a microWakeWord model takes.third_party/microfrontend) with the kissfft it uses (BSD-3-Clause,third_party/kissfft), both unchanged.src/shared/audiodsp_microfrontend.ccompiles them as one C file with every symbol renamed, and replaces the three C++ glue files with C, so no target needs a C++ compiler.readinto()gives the int8 those models take,readinto_raw()the frontend's uint16.tests/data/audiofeatureswere written by that build.attach(tap, rate)reads anaudiopump.Tapin C on MicroPython.examples/audiofeatures_simpletest.pyprints a WAV's slices;docs/guides/speech-features.mdexplains the module;tests/parity/features_probe.pyholds every interpreter to the same bytes.