Skip to content

Feat/kawaii eyes - #108

Closed
blakelton wants to merge 12 commits into
jamro:mainfrom
blakelton:feat/kawaii_eyes
Closed

blakelton wants to merge 12 commits into
jamro:mainfrom
blakelton:feat/kawaii_eyes

Conversation

@blakelton

@blakelton blakelton commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds a new OLED eye style, eyes_style=kawaii: round eyes with big pupils, a small talking mouth, and blush. It is opt-in, and the default stays classic. This supersedes #107, which I opened too early and closed.

The face

  • Big dark pupils with white catch-lights. They look where the existing choreography points (the task, the human, up while thinking).
  • Pupils keep one size: lids, smiles, and blinks cover them like eyelids.
  • Moods ease between each other (springs on eye size, smiles, and mouth).
  • Shapes that cannot morph (> <, x x, hearts) swap behind a quick blink.
  • The classic pose still drives position, height, glances, and blinks, so every eye mode works with it, including the dead flicker and sleep.

Per animation

Animation Face
idle Mostly smiles; now and then a grin, wink, excitement, or heart eyes
typing / reading / thinking A focused / absorbed / pondering base face, with occasional positive accents (pleased, "ooh", "aha!") that come more often the longer it has been since an error. Typing reads code: word by word along a few lines, then a dart to a new spot
welcome, attention, error, abort, dead The mouth says the stock clip's words in sync. It is keyed to each clip's loudness, with lips closing on p, b, m, t, d, and k. A custom clip with its own .cue gets syllable babble instead
talking The mouth follows /play loudness with vowel shapes, or babbles without audio
ring A celebration: a deep breath through the wind-up, an excited yell and a star burst on the ding, and pupils shaking with the bell's measured loudness until it fades

Code

  • Location: everything kawaii lives in src/display/eyes/styles/kawaii/ as plain C++ with no Arduino dependencies, and builds in the native env.
  • Layout: one module per job (renderer, expressions, animator, mood director, speech, lip-sync lines, gaze, star burst); styles/kawaii.cpp is the only firmware glue.
  • Docs: the layout, tuning, and adding a mood are documented in kawaii/README.md.
  • Performance: the ESP32-C3 has no FPU, and audio clips are fed from the same loop() as the eyes. The renderer works in per-row runs with pixel-centered discs. On an emulated C3, the busiest frame while a clip plays costs ~550 k instructions; the README documents the 600 k budget.
  • Style registration: the same pattern as the other styles: registry.cpp, eye_style_ids.cpp (settings validation and its test), the Config dropdown, docs/api.md, and the web UI API page.

Two small fixes outside kawaii, in their own commits

  • fix(anim) — 8b60ab0. The scripted blinks in the welcome and attention eye modes had their easing upside down: the eye snapped shut, opened, shut, then snapped open. Each window is now one blink. This changes classic, dots, and cover slightly at those two moments.
  • fix(scripts) — 3885522. The post-upload LittleFS hook did not pass --upload-port, so with several serial devices plugged in it uploaded to the wrong one.

Happy to split either fix into its own PR if you prefer.

No HTTP routes, pins, NVS layout, or servo behavior change. eyes_style gains one accepted value, so nothing breaks.

Checks

  • Title is type(scope): summary
  • Breaking HTTP / pins / NVS / servo defaults / hook CLI called out as type!: plus a BREAKING CHANGE: footer (N/A: additive only)
  • Firmware: pio run (no warnings) and pio test -e native (133/133; the new test_kawaii suite covers rendering, easing, moods, lip sync, gaze, and the bell)
  • HTTP: web UI API page (ui/src/views/ApiReference.tsx) + docs/api.md updated for the new eyes_style value; npm test --prefix ui and npm run lint --prefix ui pass
  • Settings: kawaii added to validation and its test; no raw access_token in logs
  • Servo ranges not widened without a real robot (untouched)
  • Hardware tested: robot
  • No .env, tokens, or Wi-Fi passwords in logs or screenshots

Summary by CodeRabbit

  • New Features
    • Added the Kawaii eye style, with expressive animated faces, mood changes, reading gaze, speech-synced mouth movements, and bell celebrations.
    • Added Kawaii to the available eye-style options and settings reference.
  • Bug Fixes
    • Improved blink animation timing in attention and welcome screens.
    • Applied the configured upload port when uploading filesystem data.
  • Documentation
    • Documented Kawaii expressions, animations, speech behavior, and wakeup appearance.

Add `eyes_style=kawaii`: round eyes with sparkle highlights, a small
mouth between them, and dithered blush under the outer corners.

- Parametric face renderer (eye shape, lids, happy crescents, mouth,
  blush, twinkles) so every feature eases between moods; springs give
  eye size, smiles and mouth opening a little overshoot
- Shape swaps that cannot morph (> <, x x, hearts) happen behind a blink
- Idle personality: mostly smiles, with occasional grins, winks,
  excitement, and rare heart eyes
- Each animation maps to a mood; talking opens the mouth with speech
  loudness from /play
- Classic pose, glances and blinks still drive position and openness
- Renderer and animator are Arduino-free and covered by native tests
- Register the style id, Config select option, API docs and web UI
  API page
The kawaii mouth only moved with /play audio, so a plain
`POST /anim talking` showed a still smile while the head and arms
moved. Each syllable now opens the mouth into an "ah", "ee" or "o"
shape and then returns to a small resting smile. With audio, loudness
sets how far it opens and each syllable onset picks a new shape.
Without audio, it babbles in short phrases with pauses.

Also, from the review:
- split mood.* into expression, animator, idle, speech and compose
  modules, with shared helpers in util.h
- name every tuning value; idle accent odds and vowel shapes are
  tables with compile-time checks
- break up drawRoundEye, drawMouth, drawKawaii and the animator's
  shape-swap blink to cut cognitive complexity
- remove the static scratch buffer from render(); the eyes now clear
  their own gap around the blush, so render() keeps no state
- rename test_kawaii_face to test_kawaii and add tests for vowel
  shapes, babble rhythm, millis wrap, audio follow and render purity
- document kawaii in the dead and talking rows of docs/api.md
- add kawaii/README.md with the module layout and tuning guide

Every mood except talking renders pixel-identical to before.
The welcome clip clipped with kawaii eyes. Audio clips and eye
drawing share loop(): each pass feeds 23 ms of audio, and the
refactor's two-pass blush gap made a welcome frame (Excited at the
welcome pose) cost about 2.45 M instructions on the C3. With the
~12 ms OLED push, that overran the budget.

- Draw each eye once; blush picks its dots against the finished eyes
  before drawing any, so render() still keeps no state
- Draw eyes, lids, carve, sparkle holes and > < / x x strokes as
  per-row runs: only edge pixels are tested in float, with the exact
  original test, so output is unchanged
- Precompute blush and heart terms per row or column, hoist loop
  bounds, skip the mouth wobble when there is none
- Cache the animator's easing weights per frame time

A welcome frame now takes about 510 k instructions (1.15 M before
the refactor). Every mood renders pixel-identical. The frame budget
is documented in kawaii/README.md.
The scripted blinks in the welcome and attention eye modes had their
easing upside down: blinkT runs 0 → 1 → 0 across the window, and
open = ease(blinkT) snapped the eyes shut at the start, opened them
in the middle, shut them again, and snapped them open at the end.

At 33 ms frames the 100 ms welcome blink after "Hello, human" showed
as shut → open → half shut → open, a visible flicker (most obvious
with round kawaii eyes). Use ease(1 - blinkT) so each window is one
blink: open at the edges, closed in the middle.
A round kawaii eye squashed to a couple of pixels drew as a flat bar,
which reads as a glitch rather than a blink. Draw the closed "‿" lid
line once the eye is flatter than 0.22 of its width radius, compared
with the width because scripted blinks squash the pose height itself.

Only the deepest blink frames change.
The dark sparkle holes read as pupils on the robot, so make them real
pupils: a dark disc that looks where the eye choreography points (the
task or the human, down at the keyboard while typing, up while
thinking), with a white catch-light inside it that cannot be mistaken
for a second pupil.

- Gaze comes from the shared eye-mode glances; moods add their own
  pupil size and look (small when surprised or angry, big and
  looking down when sad)
- The pupil keeps a rim of eye around it: when a lid, tilt, smile or
  blink leaves no room where it looks, it slides toward the middle,
  then shrinks, so the eye outline never notches
- Gaze eases quickly, like a saccade
- Tests cover the catch-light, gaze direction, the rim under lids
  and smiles, and no pupil in a full ^ ^ smile
Replace the dark sparkle holes, which read as small pupils, with big
dark pupils that look where the robot is looking, and white
catch-lights inside them (a round light and two dots) that cannot be
mistaken for a pupil.

- Gaze follows the shared eye-mode glances (the task or the human,
  up while thinking); moods set pupil size and their own look
- Pupils keep one size: lids, smiles and blinks cover them like
  eyelids, a 2 px rim always stays white, and centers snap to whole
  pixels so a moving pupil never changes shape
- Typing reads code: a few lines word by word, then a dart to another
  spot, instead of following classic's typing jitter
- Thinking follows its glances halfway, for a calmer look aside
- Gaze glides (150 ms); catch-lights no longer pulse
- Discs centered on pixels precompute their rows: a welcome frame
  drops to ~420 k instructions on the C3
- Tests cover gaze, constant pupil size, the rim under lids and
  smiles, and the code-reading gaze
Working modes held one face the whole time. Give typing, reading and
thinking the same personality idle has: mostly the working face, with
a short accent now and then before settling back.

- Typing: focused, with a pleased smile, a quick grin, or a wink;
  the eyes keep reading code through the accents
- Reading: absorbed, with a pleased read or an "ooh, interesting"
- Thinking: pondering, with an "aha!" (wide eyes looking up, glints,
  a hop, an open smile)
- Accents come more often the longer work has gone well: the gap
  shrinks over ~90 s without an error, abort or dead, and resets on
  one
- Generalize the idle director into MoodDirector with one
  personality table per activity; idle behaves as before
- Add Pleased and Aha moods
- Tests: working modes stay mostly on their base face with only
  their own accents, and going well brings more accents
…bell

The kawaii mouth now says the words of every stock voice clip in sync,
and the bell gets a celebration instead of squashed surprised eyes.

Spoken lines (keyed to each stock clip's loudness envelope):
- welcome: "Hello, human. What are we building today?"
- attention: "Psst... human... you might want to take a look."
- error: "Uh-oh. Human, we have a problem."
- abort: "Fine. I didn't want to finish it anyway."
- dead: "Insufficient resources... shutting doooowwwnnn"
Keyframes close the lips on p, b, m, t, d and k and hold pauses shut;
tracks lead the audio by 40 ms. While a clip plays, the face uses the
talking rest mouth. One Line table holds every track and its stock cue
times; a custom clip with its own .cue gets syllable babble instead.

Bell (ring): an eager wind-up, a burst of joy the moment bell.wav
starts, bliss while it rings (^ ^ squeezed shut, blush, open smile, a
delighted shiver), then a dreamy afterglow. The ring no longer squashes
the eyes; without a bell clip the ding comes after 1.5 s.

Adds Bliss and Afterglow moods and a quiver (shiver) parameter.
The bell's delighted shiver was hidden behind squeezed-shut eyes and
stopped abruptly, and the wind-up before the ding had little to say.

- Wind-up: eager, then suspense builds over 0.8 s and holds until the
  strike: eyes narrow onto the bell, the mouth holds its breath, and
  a nervous tremble grows
- Bliss keeps the eyes open (soft happy eyes, big sparkling pupils),
  so the shiver shows for as long as bell.wav plays
- When the ring ends, the shiver winds down over 500 ms, smaller and
  slower (14 Hz to 3 Hz), while the face blends into the afterglow
- The shiver's phase now advances in the animator, so its speed can
  change without a jump
- BellCelebration returns the face itself; Expression gains a blend()
  helper and quiverHz; adds a Suspense mood
…hake

The bell's shake moved the whole eye, trembled through the wind-up,
and ran for the full clip, including its silent tail. Rework the
celebration around the ding.

- Only the pupils shake, starting on the ding and following the stock
  bell.wav loudness (measured every 10 ms): hardest and fastest on the
  strike, fading and slowing to the last of the ring at ~1.4 s
- Wind-up is a deep breath that deepens until the strike: the face
  rises, eyes widen, pupils tighten, cheeks flush, and the mouth draws
  in a growing "o"; the ding lets it out as an excited yell
- Stars burst from the outer sides of the eyes on the ding and swirl
  off-screen, with trails that shorten as the bell fades
- Adds the Yell mood, a breath "rise", per-eye pupil shake, and face
  sparks; the busiest bell frame is ~550 k instructions on the C3
- Docs: kawaii summaries, the lip-sync paragraph, the pose exceptions,
  and the frame budget brought up to date
The post-upload hook runs `pio run -t uploadfs` as a child process,
which does not inherit --upload-port. With more than one serial device
plugged in, the filesystem upload auto-detected another port and
failed after the firmware had flashed fine.

Pass the port the firmware upload used ($UPLOAD_PORT, given or
auto-detected) on to uploadfs.
@coderabbitai

coderabbitai Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The pull request adds a Kawaii eye style with mood-driven expressions, animated face rendering, gaze tracking, speech synchronization, and bell effects. It also updates style configuration and documentation, reverses two blink curves, and passes a configured port to filesystem uploads.

Changes: Kawaii eye style

Layer / File(s) Summary
Face and expression contracts
src/display/eyes/styles/kawaii/util.h, face.h, expression.h, expression.cpp, compose.h
New types define face features, poses, moods, and expressions. Mood presets and shared utilities provide values for the renderer.
Face composition and rendering
src/display/eyes/styles/kawaii/compose.cpp, face.cpp, burst.h, burst.cpp
Face composition derives eye, mouth, gaze, blush, and glint values. The renderer draws them into a frame, and the burst effect adds timed sparks.
Mood, animation, and reading gaze
src/display/eyes/styles/kawaii/animator.*, director.*, tracker.*
The animator eases expressions and eye-kind changes. The mood director schedules activity accents and bell celebrations, while the code reader updates gaze over time.
Speech and mouth animation
src/display/eyes/styles/kawaii/speech.*, lines.*
Voice data, babbling, and audio-following shape the mouth. Timed viseme tracks animate five stock lines and check whether audio cues match stock timings.
Style integration and validation
src/display/eyes/styles/kawaii.cpp, registry.cpp, eye_style_ids.cpp, platformio.ini, ui/src/config/ConfigForm.tsx, ui/src/views/ApiReference.tsx, test/*, docs/api.md, docs/hardware/testing.md, src/display/eyes/styles/README.md, src/display/eyes/styles/kawaii/README.md
The style connects mood selection, speech, animation, composition, and drawing to the OLED pipeline. Registration, native test sources, configuration options, tests, and documentation are updated.

Attention and welcome blink curves

Layer / File(s) Summary
Reverse blink openness curves
src/display/eyes/modes/attention.cpp, src/display/eyes/modes/welcome.cpp
Both blink calculations use the inverse of blink progress for openness, so the eyes close toward the window midpoint.

Filesystem upload port

Layer / File(s) Summary
Add port to filesystem upload command
scripts/upload_fs_after_upload.py
The helper builds the PlatformIO command as an argument vector and adds the upload-port option only when the port is set and the target is uploadfs.

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Suggested reviewers: jamro, mechazawa, jtruland

Merge Risk: Minimal · up to 38855

This PR adds an opt-in kawaii OLED eye style, adjusts two blink curves, and forwards the upload port to the LittleFS upload. No concrete defect was found. The default style is unchanged, and the new code is covered by native tests and documentation. It looks safe to merge.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title "Feat/kawaii eyes" names the main change. The PR adds an opt-in kawaii eye style to the OLED display, and the changeset implements that style. The title uses a branch-style prefix and is bri…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title "Feat/kawaii eyes" names the main change. The PR adds an opt-in kawaii eye style to the OLED display, and the changeset implements that style. The title uses a branch-style prefix and is brief, but it still identifies the primary change for a reader scanning history.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Repository UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d9cbb1bb-3480-4111-87f5-714c7eb4e5a8
Commits

Reviewing files that changed from the base of the PR and between 567fcc8 and 3885522.

Files selected for processing: 34

  • docs/api.md
  • docs/hardware/testing.md
  • platformio.ini
  • scripts/upload_fs_after_upload.py
  • src/display/eyes/modes/attention.cpp
  • src/display/eyes/modes/welcome.cpp
  • src/display/eyes/styles/README.md
  • src/display/eyes/styles/eye_style_ids.cpp
  • src/display/eyes/styles/kawaii.cpp
  • src/display/eyes/styles/kawaii/README.md
  • src/display/eyes/styles/kawaii/animator.cpp
  • src/display/eyes/styles/kawaii/animator.h
  • src/display/eyes/styles/kawaii/burst.cpp
  • src/display/eyes/styles/kawaii/burst.h
  • src/display/eyes/styles/kawaii/compose.cpp
  • src/display/eyes/styles/kawaii/compose.h
  • src/display/eyes/styles/kawaii/director.cpp
  • src/display/eyes/styles/kawaii/director.h
  • src/display/eyes/styles/kawaii/expression.cpp
  • src/display/eyes/styles/kawaii/expression.h
  • src/display/eyes/styles/kawaii/face.cpp
  • src/display/eyes/styles/kawaii/face.h
  • src/display/eyes/styles/kawaii/lines.cpp
  • src/display/eyes/styles/kawaii/lines.h
  • src/display/eyes/styles/kawaii/speech.cpp
  • src/display/eyes/styles/kawaii/speech.h
  • src/display/eyes/styles/kawaii/tracker.cpp
  • src/display/eyes/styles/kawaii/tracker.h
  • src/display/eyes/styles/kawaii/util.h
  • src/display/eyes/styles/registry.cpp
  • test/test_kawaii/test_kawaii.cpp
  • test/test_settings_validate/test_settings_validate.cpp
  • ui/src/config/ConfigForm.tsx
  • ui/src/views/ApiReference.tsx

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 3 remain after this review.

@blakelton

Copy link
Copy Markdown
Contributor Author

Hope you love it!

@blakelton blakelton closed this Nov 14, 2026
@blakelton blakelton mentioned this pull request Nov 14, 2026
8 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant