From be841aa9a945af1f8f46a8421ca6d6d30dd818d9 Mon Sep 17 00:00:00 2001 From: Victor Radu Date: Wed, 18 Nov 2026 00:05:37 +0000 Subject: [PATCH 1/2] feat(anim): add waiting animation Continuous. The torso turns to the robot's left, the left hand taps the desk in groups of three at random times, and the head drifts slowly between four wide corners. Classic eyes lead the head to each corner. Co-Authored-By: Claude Opus 5.5 --- docs/api.md | 20 +- docs/hardware/components.md | 2 +- docs/hardware/testing.md | 2 +- docs/hooks.md | 2 +- docs/integration.md | 1 + src/animation.h | 3 +- src/animation/registry.cpp | 13 ++ src/animation/waiting.cpp | 291 ++++++++++++++++++++++++++++ src/animation/waiting.h | 17 ++ src/display/eyes.h | 3 +- src/display/eyes/modes/waiting.cpp | 86 ++++++++ src/display/eyes/modes/waiting.h | 6 + src/display/eyes/styles/kaomoji.cpp | 1 + src/display/eyes/styles/kawaii.cpp | 2 + src/hardware/rgb.cpp | 1 + src/http/play_handlers.cpp | 2 +- src/serial/play.cpp | 2 +- ui/src/config/ConfigForm.tsx | 2 +- ui/src/lib/animations.ts | 1 + ui/src/views/ApiReference.tsx | 12 ++ 20 files changed, 452 insertions(+), 17 deletions(-) create mode 100644 src/animation/waiting.cpp create mode 100644 src/animation/waiting.h create mode 100644 src/display/eyes/modes/waiting.cpp create mode 100644 src/display/eyes/modes/waiting.h diff --git a/docs/api.md b/docs/api.md index f0fa8058..6b402643 100644 --- a/docs/api.md +++ b/docs/api.md @@ -194,7 +194,7 @@ curl http://tiny-engineer.local/settings | `serial_log` | USB serial debug logging (default **false**). When off, diagnostic `Serial` output is suppressed | | `push_button` | Extension-port push-button reader (default **false**). When on, publishes immediate `button.down` / `button.up` plus `button.click` / `button.double_click` / `button.long_click` (see [Outbound events](#outbound-events)). Applies **immediately**. Local reactions use [Actions](#get-settingsactions) | | `push_button_pin` | GPIO on the Extension port for the push-button: **6**, **7**, **20**, or **21** (default **20**). Persists when `push_button` is off. Applies **immediately** when the extension is on | -| `continuous_timeout` | Minutes a continuous animation (`typing` / `reading` / `thinking` / `talking`) may run before the firmware switches to `attention` then idle (default **5**) | +| `continuous_timeout` | Minutes a continuous animation (`typing` / `reading` / `thinking` / `talking` / `waiting`) may run before the firmware switches to `attention` then idle (default **5**) | | `loading` | Boot loading screen: `progress` (bar + large IP for 3 s) or `sleep_inertia` (eyes wake; no bar/IP). Default **`progress`**. Applies on **next reboot** | | `eyes_style` | OLED eyes look: `classic` (procedural rounded eyes), `kaomoji` (animated faces), `cover` (full-half bars for an eye-hole mask), `dots` (12×12 circles), or `kawaii` (round eyes with big moving pupils, a talking mouth, and blush). Default **`classic`**. Applies **immediately**. Classic, cover, and dots sequences are described under [`POST /anim`](#post-anim); kaomoji face mapping is under [Kaomoji eyes](#kaomoji-eyes); cover drawing is under [Cover eyes](#cover-eyes); dots drawing is under [Dots eyes](#dots-eyes); kawaii moods are under [Kawaii eyes](#kawaii-eyes) | | `access_token_set` | `true` when a non-empty access token is stored (secret itself is never returned) | @@ -593,7 +593,7 @@ curl http://tiny-engineer.local/anim | Field | Meaning | | --- | --- | -| `animation` | `none`, `typing`, `reading`, `thinking`, `talking`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | +| `animation` | `none`, `typing`, `reading`, `thinking`, `talking`, `waiting`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | ### `POST /anim` @@ -601,7 +601,7 @@ Request an animation switch. Each animation runs at least **1s**; if the current | Param | Type | Values | | --- | --- | --- | -| `name` | string | `none`, `typing`, `reading`, `thinking`, `talking`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, `sleep` | +| `name` | string | `none`, `typing`, `reading`, `thinking`, `talking`, `waiting`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, `sleep` | ```bash curl -X POST "http://tiny-engineer.local/anim?name=typing" @@ -634,6 +634,7 @@ Durations and quoted lines below are the stock clips in [`assets/`](../assets/). | `reading` | **Continuous.** Hands/body park as in `none`. Head nods on same lowest 10° band as typing, but slower. Neck sweeps ±10° around mid; right (angle down) slower than left (angle up). Occasional right-hand down-arrow bursts (1–3 presses, same 15° band as typing). Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | | `thinking` | **Continuous.** Hands/body park as in `none`. Head (pitch) and neck (yaw) ease from the current pose into thinking poses (up + slight left/right). Move → pause → optional micro-adjust (sometimes chained) → pause; nearby pose drift with occasional larger shifts after ~2.2 s, more often over time. Axes stagger start/duration; no periodic sway. Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | | `talking` | **Continuous.** Speaking fidget with no sound of its own. Also the default pose for [`POST /play`](#post-play). Via `/anim`, the head faces the user just above mid with soft random nods, slow neck drift, occasional small torso shifts, and occasional right-hand lifts. During `/play` with pose `talking`, motion follows speech energy from the PCM stream (no extra metadata): emphasis triggers nods, silence settles the pose, phrase starts may raise the hand or shift the torso (not both). **Classic eyes:** a little taller than idle; glances away when ambient or quiet, gaze locked on the listener while speech energy is high. Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | +| `waiting` | **Continuous.** Impatient wait with no sound. The torso turns to the robot's left (body `+0.8`), the right hand parks at rest, and the left hand taps the desk in groups of three (each tap lifts 0.18–0.28 of its range, groups every 2.5–8 s at random). Every 3–7 s the head eases (1.4–2.3 s) to one of four wide corners, never the same one twice: down at the tapping hand (most often), up-left, up-right, or down-right. **Classic eyes:** glance to the same corner about 180 ms before the head gets there (±5 px sideways, 3 px up / 2 px down), a little wider looking up and a little lower looking down. The Claude Code hooks use it when Claude has waited about a minute for the user (`Notification` with `idle_prompt`). Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | | `ring` | **One-shot** service-bell gesture; does not loop. Wind-up (body `min`, neck mid, head mid+10°, both hands `max`) → fast right-hand strike to `min+5°` with head to `min` → plays `bell.wav` once on strike (LittleFS; same `uploadfs` requirement as `/test/audio/bell`) → slower bounce to `min+20°` → return to `none` pose and stop. After completion, `GET /anim` reports `none`. | | `welcome` | **One-shot** hello gesture synced to `welcome.wav` (~2.7 s). Right hand raises during "Hello, human.", holds through the pause, wiggles during "What are we building today?", then lowers. Head nods to mid+10° and returns. Plays automatically after successful Wi-Fi connect at boot; also via API. Requires `welcome.wav` on LittleFS (same `uploadfs` flow as `bell.wav`). After completion, `GET /anim` reports `none`. | | `attention` | Friendly input-request gesture synced to `attention.wav` (~3.0 s, "pst... human.... you might want to take a look"). Moves into a calm prompt pose first (centered body/neck, head slightly up, right hand raised partway), waits until all servos stop, then plays audio with light neck/head/hand motion during playback (whisper hold → lean toward user → glance/point on "take a look"). **Classic eyes:** phased blink/look cues during audio. After audio ends (or if audio fails to start), holds a gentle waiting loop for **1 minute** (soft head/neck drifts plus occasional slight right-hand waves), then returns to `none`. Requires `attention.wav` on LittleFS (same `uploadfs` flow as `bell.wav`). | @@ -649,7 +650,7 @@ When `eyes_style=kaomoji`, the OLED shows looping faces from the expression libr | Animation / phase | Face | | --- | --- | -| `none` (idle) | Idle | +| `none` (idle), `waiting` | Idle | | `typing`, `thinking` | Thinking | | `reading`, `attention` | Curious | | `ring` | Surprise | @@ -688,6 +689,7 @@ While `welcome`, `attention`, `error`, `abort`, or `dead` plays its stock clip, | `abort` | Panic: `> <` eyes, wavy mouth; says "Fine. I didn't want to finish it anyway." | | `wakeup` | Sleepy: heavy lids, yawn | | `talking` | The mouth talks in syllables, shaped as `ah` (tall), `ee` (wide), or `o` (round). With `/play` audio it opens with the speech loudness and changes shape on each syllable onset. Without audio it babbles in short phrases with pauses | +| `waiting` | The idle smile (`ω`). The pupils follow the classic eyes to each corner the head looks at | | `sleep` / sleep-close | Lids close, small `ω` | | `dead` | Sad (lids tilted down at the outer corners, frown), then `x x` eyes during the X hold. Says "Insufficient resources... shutting doooowwwnnn" | @@ -696,15 +698,15 @@ Wrong params return **400**: | `error` | When | | --- | --- | | `missing name` | Query param `name` absent | -| `unknown animation` | `name` not `none`, `typing`, `reading`, `thinking`, `talking`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | +| `unknown animation` | `name` not `none`, `typing`, `reading`, `thinking`, `talking`, `waiting`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | ### `POST /play` -Streams a WAV to the speaker while one animation runs, then returns to the continuous animation (`typing`, `reading`, `thinking`, `talking`) that was running before the request, otherwise to `none`. The body is played as it arrives, so clips of any length need no storage on the robot. The response comes when the clip ends. +Streams a WAV to the speaker while one animation runs, then returns to the continuous animation (`typing`, `reading`, `thinking`, `talking`, `waiting`) that was running before the request, otherwise to `none`. The body is played as it arrives, so clips of any length need no storage on the robot. The response comes when the clip ends. | Part | Value | | --- | --- | -| Path | `/play` runs `talking`; `/play/{name}` runs `talking`, `typing`, `reading`, `thinking`, or `none`. Animations with their own sound are rejected so two clips never overlap | +| Path | `/play` runs `talking`; `/play/{name}` runs `talking`, `typing`, `reading`, `thinking`, `waiting`, or `none`. Animations with their own sound are rejected so two clips never overlap | | Body | WAV, 16-bit mono PCM at 22050 Hz (the firmware `SAMPLE_RATE`). `Content-Length` is required. Chunks other than `fmt ` and `data` are skipped | | `Authorization` | `Bearer ` when `access_token` is set | @@ -725,7 +727,7 @@ Errors arrive before any audio plays: | Status | `error` | When | | --- | --- | --- | -| `400` | `name must be talking, typing, reading, thinking or none` | `{name}` is unknown or has its own sound | +| `400` | `name must be talking, typing, reading, thinking, waiting or none` | `{name}` is unknown or has its own sound | | `415` | `expected 16-bit mono PCM at 22050 Hz` | WAV format or sample rate differs | | `415` | `not a WAV file`, `no audio data` | Body is not a RIFF/WAVE file, or has no `data` chunk | @@ -759,7 +761,7 @@ Onboard WS2812 on **GPIO10** (`RGB_LED_PIN`). Byte order is the saved `rgb_order | Animation | LED color | | --- | --- | -| `typing`, `reading`, `thinking`, `talking`, `welcome`, `ring`, `wakeup` | White (full intensity) | +| `typing`, `reading`, `thinking`, `talking`, `waiting`, `welcome`, `ring`, `wakeup` | White (full intensity) | | `attention`, `error`, `dead` | Red pulse: 500 ms 10%→100%, 500 ms hold 100%, 500 ms 100%→10%, repeat | | `abort` | Red (full intensity, solid) | | `none`, `sleep` | Off | diff --git a/docs/hardware/components.md b/docs/hardware/components.md index 8cf5517e..f0bb61b3 100644 --- a/docs/hardware/components.md +++ b/docs/hardware/components.md @@ -34,7 +34,7 @@ Constants cited below match [`include/pins.h`](../../include/pins.h). | USB | Native USB on GPIO18/19 — reached via the main control board USB-C (onboard USB-C unused when assembled). Advanced path: Adafruit 5993 — [wiring.md](wiring.md#usb-connector-5993) | | Flash | 4 MB stacked flash on GPIO12–GPIO17 (those GPIOs are **not exposed**) | | Onboard RGB | WS2812 on **GPIO10** | -| Firmware notes | Boot: dim green status. Animations: white (typing/reading/thinking/talking/welcome/ring/wakeup), pulsing red (attention/error/dead), solid red (abort), off (idle/sleep). 1 s fade between non-pulse states. Fatal init errors: hang and blink red (1 = PCA9685, 2 = I2S). See [blink codes](testing.md#boot-failure-blink-codes) and [`docs/api.md`](../api.md#rgb-led) | +| Firmware notes | Boot: dim green status. Animations: white (typing/reading/thinking/talking/waiting/welcome/ring/wakeup), pulsing red (attention/error/dead), solid red (abort), off (idle/sleep). 1 s fade between non-pulse states. Fatal init errors: hang and blink red (1 = PCA9685, 2 = I2S). See [blink codes](testing.md#boot-failure-blink-codes) and [`docs/api.md`](../api.md#rgb-led) | | BOOT | Button / strapping on **GPIO9** | | RESET | Separate RESET button (CHIP_EN), not a GPIO | | UART0 silkscreen | TX = GPIO21, RX = GPIO20 | diff --git a/docs/hardware/testing.md b/docs/hardware/testing.md index 98d8aa16..638f01ac 100644 --- a/docs/hardware/testing.md +++ b/docs/hardware/testing.md @@ -70,7 +70,7 @@ During normal operation the onboard WS2812 tracks the active animation (not boot | Animation | LED | | --- | --- | -| `typing`, `reading`, `thinking`, `talking`, `welcome`, `ring`, `wakeup` | White | +| `typing`, `reading`, `thinking`, `talking`, `waiting`, `welcome`, `ring`, `wakeup` | White | | `attention`, `error`, `dead` | Pulsing red (10%↔100%, 1.5 s cycle) | | `abort` | Solid red | | `none`, `sleep` | Off | diff --git a/docs/hooks.md b/docs/hooks.md index bc217c55..21e540e7 100644 --- a/docs/hooks.md +++ b/docs/hooks.md @@ -197,6 +197,6 @@ Config: [`.cursor/hooks.json`](../.cursor/hooks.json). Local anim hook: [`.curso - Robot offline → hook still exits 0; no agent stall. - Animation API holds each pose ≥1s and keeps only the latest pending switch — see [`api.md`](api.md). -- The onboard RGB LED follows the active animation (white for typing/reading/thinking/talking/welcome/ring/wakeup, pulsing red for attention/error/dead, solid red for abort, off for `none`/`sleep`) with 1 s fades between non-pulse states — see [RGB LED](api.md#rgb-led). +- The onboard RGB LED follows the active animation (white for typing/reading/thinking/talking/waiting/welcome/ring/wakeup, pulsing red for attention/error/dead, solid red for abort, off for `none`/`sleep`) with 1 s fades between non-pulse states — see [RGB LED](api.md#rgb-led). - To change the map, edit [`packages/tiny-engineer-cli/src/integrations/cursor/map.js`](../packages/tiny-engineer-cli/src/integrations/cursor/map.js). - Custom agents or non-Cursor IDEs: call the [REST API](integration.md#2-rest-api-advanced--custom). diff --git a/docs/integration.md b/docs/integration.md index 6635fe0d..5fc38115 100644 --- a/docs/integration.md +++ b/docs/integration.md @@ -150,6 +150,7 @@ curl -X POST "http://tiny-engineer.local/anim?name=typing" | `welcome` | Session start / greeting | | `wakeup` | Sleep-inertia wake (eyes/head) | | `sleep` | Force sleep (eye close + OLED off) | +| `waiting` | Turn finished and the user has not answered for a while (quiet, before `attention`) | | `attention` | Needs user input | | `error` / `abort` | Failure / cancel | | `dead` | Out of power (`dead.wav`, then X X hold) | diff --git a/src/animation.h b/src/animation.h index 8c7a144e..30735e68 100644 --- a/src/animation.h +++ b/src/animation.h @@ -15,7 +15,8 @@ enum class AnimationId { Wakeup, Sleep, Dead, - Talking + Talking, + Waiting }; void setAnimation(AnimationId id); diff --git a/src/animation/registry.cpp b/src/animation/registry.cpp index 2c3c3ce7..333fc245 100644 --- a/src/animation/registry.cpp +++ b/src/animation/registry.cpp @@ -15,6 +15,7 @@ #include "animation/thinking.h" #include "animation/typing.h" #include "animation/util.h" +#include "animation/waiting.h" #include "animation/wakeup.h" #include "animation/welcome.h" #include "display/eyes/modes/abort.h" @@ -27,6 +28,7 @@ #include "display/eyes/modes/talking.h" #include "display/eyes/modes/thinking.h" #include "display/eyes/modes/typing.h" +#include "display/eyes/modes/waiting.h" #include "display/eyes/modes/wakeup.h" #include "display/eyes/modes/welcome.h" #include "hardware/servo_wrapper.h" @@ -275,6 +277,17 @@ constexpr ModeEntry kModes[] = { startTalkingEyes, updateTalkingEyes, }, + { + AnimationId::Waiting, + EyeMode::Waiting, + "waiting", + true, + -1, + startWaiting, + updateWaiting, + startWaitingEyes, + updateWaitingEyes, + }, }; constexpr size_t kModeCount = sizeof(kModes) / sizeof(kModes[0]); diff --git a/src/animation/waiting.cpp b/src/animation/waiting.cpp new file mode 100644 index 00000000..dc4b91c0 --- /dev/null +++ b/src/animation/waiting.cpp @@ -0,0 +1,291 @@ +#include + +#include "animation/constants.h" +#include "animation/util.h" +#include "animation/waiting.h" +#include "hardware/servo_wrapper.h" +#include "servos.h" +#include "serial_log.h" + +using anim::beginEasedMove; +using anim::easedMoveDone; +using anim::easedMoveValue; +using anim::EasedMove; +using anim::randChance; +using anim::randRangeMs; +using anim::randUnit; + +namespace { + +// Torso turned to the robot's left, away from the bell (body + = left). +constexpr float WAIT_BODY = 0.8f; +constexpr float WAIT_SETTLE_SPEED_DEG_S = 30.0f; + +// Left hand rests on the desk (+1, its scale is inverted) and lifts toward - +// a little for each tap. +constexpr int WAIT_TAP_SERVO = SERVO_HAND_LEFT; +constexpr float WAIT_HAND_REST = 1.0f; +constexpr float WAIT_TAP_DIR = -1.0f; +constexpr float WAIT_TAP_LIFT_MIN = 0.18f; +constexpr float WAIT_TAP_LIFT_SPAN = 0.10f; +constexpr float WAIT_TAP_UP_SPEED_DEG_S = 70.0f; +constexpr float WAIT_TAP_DOWN_SPEED_DEG_S = 100.0f; +constexpr uint32_t WAIT_TAP_GROUP_GAP_MIN_MS = 2500; +constexpr uint32_t WAIT_TAP_GROUP_GAP_MAX_MS = 8000; +constexpr uint32_t WAIT_TAP_IN_GROUP_GAP_MIN_MS = 90; +constexpr uint32_t WAIT_TAP_IN_GROUP_GAP_MAX_MS = 220; +// Every group is three taps; the groups start at random times. +constexpr uint8_t WAIT_TAPS_PER_GROUP = 3; + +// Head corners. Neck: - = the robot's right, + = its left. Head: + = up. +constexpr float WAIT_HEAD_JITTER = 0.06f; +constexpr float WAIT_NECK_JITTER = 0.05f; +constexpr uint32_t WAIT_LOOK_HOLD_MIN_MS = 3000; +constexpr uint32_t WAIT_LOOK_HOLD_MAX_MS = 7000; +// Longer moves for the wide corners keep the head speed calm. +constexpr uint32_t WAIT_LOOK_MOVE_MIN_MS = 1400; +constexpr uint32_t WAIT_LOOK_MOVE_MAX_MS = 2300; +constexpr uint32_t WAIT_LOOK_STAGGER_MAX_MS = 200; + +struct WaitCorner { + const char* name; + float headNorm; + float neckNorm; + float gazeX; // eyes: -1 = robot's right, +1 = robot's left + float gazeY; // eyes: -1 = down, +1 = up + uint8_t weight; +}; + +// Index 0 looks at the tapping hand: down and a little to the left. +constexpr WaitCorner WAIT_CORNERS[] = { + {"hand (SW)", -0.90f, 0.30f, 0.6f, -1.0f, 34}, + {"up-left (NW)", 0.60f, 0.70f, 1.0f, 1.0f, 22}, + {"up-right (NE)", 0.60f, -0.70f, -1.0f, 1.0f, 22}, + {"down-right (SE)", -0.70f, -0.65f, -1.0f, -1.0f, 22}, +}; +constexpr uint8_t WAIT_CORNER_COUNT = + sizeof(WAIT_CORNERS) / sizeof(WAIT_CORNERS[0]); +constexpr uint8_t WAIT_HAND_CORNER = 0; + +enum class WaitPhase { + Settle, + Run, +}; + +enum class TapPhase { + Idle, + Lift, + Drop, + Gap, +}; + +WaitPhase g_phase = WaitPhase::Settle; +TapPhase g_tapPhase = TapPhase::Idle; +uint32_t g_nextTapMs = 0; +uint8_t g_tapsLeft = 0; + +uint8_t g_corner = WAIT_HAND_CORNER; +uint32_t g_nextLookMs = 0; +EasedMove g_headMove = {}; +EasedMove g_neckMove = {}; +WaitingGaze g_gaze = {0.0f, 0.0f, 0}; + +float jitter(float amount) { + return (randUnit() * 2.0f - 1.0f) * amount; +} + +uint8_t pickNextCorner() { + uint32_t total = 0; + for (uint8_t i = 0; i < WAIT_CORNER_COUNT; i++) { + if (i != g_corner) { + total += WAIT_CORNERS[i].weight; + } + } + + uint32_t roll = randRangeMs(0, total - 1); + for (uint8_t i = 0; i < WAIT_CORNER_COUNT; i++) { + if (i == g_corner) { + continue; + } + if (roll < WAIT_CORNERS[i].weight) { + return i; + } + roll -= WAIT_CORNERS[i].weight; + } + return WAIT_HAND_CORNER; +} + +void setGaze(uint8_t corner, uint32_t now) { + g_gaze.x = WAIT_CORNERS[corner].gazeX; + g_gaze.y = WAIT_CORNERS[corner].gazeY; + g_gaze.sinceMs = now == 0 ? 1 : now; +} + +void beginLook(uint8_t corner, uint32_t now) { + const WaitCorner& c = WAIT_CORNERS[corner]; + const float headFrom = + servoDegToNorm(SERVO_HEAD, servoAt(SERVO_HEAD).angle()); + const float neckFrom = + servoDegToNorm(SERVO_NECK, servoAt(SERVO_NECK).angle()); + const float headTo = servoSaturateNorm(c.headNorm + jitter(WAIT_HEAD_JITTER)); + const float neckTo = servoSaturateNorm(c.neckNorm + jitter(WAIT_NECK_JITTER)); + + // Eyes lead, then head and neck follow; one of them starts a bit later. + const uint32_t stagger = randRangeMs(0, WAIT_LOOK_STAGGER_MAX_MS); + const bool headFirst = randChance(50); + beginEasedMove( + g_headMove, + headFrom, + headTo, + headFirst ? now : now + stagger, + randRangeMs(WAIT_LOOK_MOVE_MIN_MS, WAIT_LOOK_MOVE_MAX_MS) + ); + beginEasedMove( + g_neckMove, + neckFrom, + neckTo, + headFirst ? now + stagger : now, + randRangeMs(WAIT_LOOK_MOVE_MIN_MS, WAIT_LOOK_MOVE_MAX_MS) + ); + + g_corner = corner; + setGaze(corner, now); + g_nextLookMs = now + randRangeMs(WAIT_LOOK_HOLD_MIN_MS, WAIT_LOOK_HOLD_MAX_MS); + + serialLogPrint("[anim] waiting look "); + serialLogPrintln(c.name); +} + +void tickLook(uint32_t now) { + if (g_headMove.active) { + servoAt(SERVO_HEAD).setNormPosition(easedMoveValue(g_headMove, now)); + if (easedMoveDone(g_headMove, now)) { + g_headMove.active = false; + } + } + if (g_neckMove.active) { + servoAt(SERVO_NECK).setNormPosition(easedMoveValue(g_neckMove, now)); + if (easedMoveDone(g_neckMove, now)) { + g_neckMove.active = false; + } + } + + const bool moving = g_headMove.active || g_neckMove.active; + if (!moving && now >= g_nextLookMs) { + beginLook(pickNextCorner(), now); + } +} + +void scheduleTapGroup(uint32_t now) { + g_tapPhase = TapPhase::Idle; + g_nextTapMs = + now + randRangeMs(WAIT_TAP_GROUP_GAP_MIN_MS, WAIT_TAP_GROUP_GAP_MAX_MS); +} + +void tickTaps(uint32_t now) { + ServoWrapper& hand = servoAt(WAIT_TAP_SERVO); + + switch (g_tapPhase) { + case TapPhase::Idle: + if (now >= g_nextTapMs) { + g_tapsLeft = WAIT_TAPS_PER_GROUP; + g_tapPhase = TapPhase::Lift; + hand.setNormTarget( + WAIT_HAND_REST + + WAIT_TAP_DIR * (WAIT_TAP_LIFT_MIN + WAIT_TAP_LIFT_SPAN * randUnit()), + WAIT_TAP_UP_SPEED_DEG_S + ); + } + break; + + case TapPhase::Lift: + if (!hand.isMoving()) { + g_tapPhase = TapPhase::Drop; + hand.setNormTarget(WAIT_HAND_REST, WAIT_TAP_DOWN_SPEED_DEG_S); + } + break; + + case TapPhase::Drop: + if (!hand.isMoving()) { + g_tapsLeft--; + if (g_tapsLeft == 0) { + scheduleTapGroup(now); + } else { + g_tapPhase = TapPhase::Gap; + g_nextTapMs = now + randRangeMs( + WAIT_TAP_IN_GROUP_GAP_MIN_MS, + WAIT_TAP_IN_GROUP_GAP_MAX_MS + ); + } + } + break; + + case TapPhase::Gap: + if (now >= g_nextTapMs) { + g_tapPhase = TapPhase::Lift; + hand.setNormTarget( + WAIT_HAND_REST + + WAIT_TAP_DIR * (WAIT_TAP_LIFT_MIN + WAIT_TAP_LIFT_SPAN * randUnit()), + WAIT_TAP_UP_SPEED_DEG_S + ); + } + break; + } +} + +bool settled() { + return !servoAt(SERVO_BODY).isMoving() + && !servoAt(SERVO_HAND_LEFT).isMoving() + && !servoAt(SERVO_HAND_RIGHT).isMoving() + && !servoAt(SERVO_HEAD).isMoving() + && !servoAt(SERVO_NECK).isMoving(); +} + +} // namespace + +void startWaiting(uint32_t animationStartedMs) { + serialLogPrintln("[anim] startWaiting"); + anim::stopAnimServos(); + + // Turn to the left, park the hands, and look at the tapping hand. + const WaitCorner& hand = WAIT_CORNERS[WAIT_HAND_CORNER]; + servoAt(SERVO_BODY).setNormTarget(WAIT_BODY, WAIT_SETTLE_SPEED_DEG_S); + anim::parkHands(anim::TRANSITION_HAND_SPEED_DEG_S); + servoAt(SERVO_HEAD).setNormTarget(hand.headNorm, WAIT_SETTLE_SPEED_DEG_S); + servoAt(SERVO_NECK).setNormTarget(hand.neckNorm, WAIT_SETTLE_SPEED_DEG_S); + + g_phase = WaitPhase::Settle; + g_tapPhase = TapPhase::Idle; + g_tapsLeft = 0; + g_headMove.active = false; + g_neckMove.active = false; + g_corner = WAIT_HAND_CORNER; + setGaze(WAIT_HAND_CORNER, animationStartedMs); +} + +void updateWaiting(uint32_t now) { + servoAt(SERVO_BODY).update(); + servoAt(SERVO_HAND_LEFT).update(); + servoAt(SERVO_HAND_RIGHT).update(); + + if (g_phase == WaitPhase::Settle) { + servoAt(SERVO_HEAD).update(); + servoAt(SERVO_NECK).update(); + if (settled()) { + g_phase = WaitPhase::Run; + g_nextLookMs = + now + randRangeMs(WAIT_LOOK_HOLD_MIN_MS, WAIT_LOOK_HOLD_MAX_MS); + // The first tap comes soon, so the pose reads as "waiting" at once. + g_nextTapMs = now + randRangeMs(600, 1500); + serialLogPrintln("[anim] waiting settled"); + } + return; + } + + tickLook(now); + tickTaps(now); +} + +WaitingGaze waitingGaze() { + return g_gaze; +} diff --git a/src/animation/waiting.h b/src/animation/waiting.h new file mode 100644 index 00000000..58f12aae --- /dev/null +++ b/src/animation/waiting.h @@ -0,0 +1,17 @@ +#pragma once + +#include + +// Where the waiting head looks, for the eyes to follow. +// x: -1 = the robot's right, +1 = the robot's left. +// y: -1 = down, +1 = up. +// sinceMs changes each time the head picks a new corner. +struct WaitingGaze { + float x; + float y; + uint32_t sinceMs; +}; + +void startWaiting(uint32_t animationStartedMs); +void updateWaiting(uint32_t now); +WaitingGaze waitingGaze(); diff --git a/src/display/eyes.h b/src/display/eyes.h index b2cd0541..aba324fd 100644 --- a/src/display/eyes.h +++ b/src/display/eyes.h @@ -21,7 +21,8 @@ enum class EyeMode { Abort, Wakeup, Dead, - Talking + Talking, + Waiting }; enum class SleepEyeResult { diff --git a/src/display/eyes/modes/waiting.cpp b/src/display/eyes/modes/waiting.cpp new file mode 100644 index 00000000..e417d158 --- /dev/null +++ b/src/display/eyes/modes/waiting.cpp @@ -0,0 +1,86 @@ +#include "display/eyes/modes/waiting.h" + +#include "animation/util.h" +#include "animation/waiting.h" +#include "display/eyes.h" +#include "display/eyes/core/constants.h" +#include "display/eyes/core/internal.h" +#include "display/eyes/core/util.h" + +namespace { + +// The eyes look to the corner the head turns to. They get there first +// (a quick glance), then the head follows over about a second. +constexpr float GAZE_RANGE_X_PX = 5.0f; // same full glance as error/attention +constexpr float GAZE_UP_PX = 3.0f; +constexpr float GAZE_DOWN_PX = 2.0f; +constexpr uint32_t GAZE_MOVE_MS = 180; + +// Lids follow the gaze: a little wider looking up, a little lower looking down. +constexpr int16_t HEIGHT_UP = 15; +constexpr int16_t HEIGHT_DOWN = 12; + +float g_fromX = 0.0f; +float g_fromY = 0.0f; +float g_toX = 0.0f; +float g_toY = 0.0f; +float g_x = 0.0f; +float g_y = 0.0f; +uint32_t g_gazeSinceMs = 0; +uint32_t g_moveStartMs = 0; + +} // namespace + +void startWaitingEyes(uint32_t now) { + g_fromX = 0.0f; + g_fromY = 0.0f; + g_toX = 0.0f; + g_toY = 0.0f; + g_x = 0.0f; + g_y = 0.0f; + g_gazeSinceMs = 0; + g_moveStartMs = now; +} + +void updateWaitingEyes(uint32_t now) { + const WaitingGaze gaze = waitingGaze(); + if (gaze.sinceMs != g_gazeSinceMs) { + g_gazeSinceMs = gaze.sinceMs; + g_fromX = g_x; + g_fromY = g_y; + g_toX = gaze.x; + g_toY = gaze.y; + g_moveStartMs = now; + } + + const float t = anim::easeInOutCubic( + eyes::moveProgress(now, g_moveStartMs, GAZE_MOVE_MS) + ); + g_x = anim::lerp(g_fromX, g_toX, t); + g_y = anim::lerp(g_fromY, g_toY, t); + + // Screen x follows the neck sign: the robot's right is -x (see error eyes). + const int16_t offsetX = (int16_t)(g_x * GAZE_RANGE_X_PX); + const int16_t offsetY = (int16_t)(g_y >= 0.0f + ? -g_y * GAZE_UP_PX + : -g_y * GAZE_DOWN_PX); + const int16_t height = g_y >= 0.0f + ? eyes::lerpInt(eyes::DEFAULT_LEFT.height, HEIGHT_UP, g_y) + : eyes::lerpInt(eyes::DEFAULT_LEFT.height, HEIGHT_DOWN, -g_y); + + Eye& left = mutableLeftEye(); + Eye& right = mutableRightEye(); + + left = eyes::eyeWithHeight( + (int16_t)(eyes::DEFAULT_LEFT.x + offsetX), + eyes::DEFAULT_LEFT.width, + height + ); + right = eyes::eyeWithHeight( + (int16_t)(eyes::DEFAULT_RIGHT.x + offsetX), + eyes::DEFAULT_RIGHT.width, + height + ); + left.y += offsetY; + right.y += offsetY; +} diff --git a/src/display/eyes/modes/waiting.h b/src/display/eyes/modes/waiting.h new file mode 100644 index 00000000..54326902 --- /dev/null +++ b/src/display/eyes/modes/waiting.h @@ -0,0 +1,6 @@ +#pragma once + +#include + +void startWaitingEyes(uint32_t now); +void updateWaitingEyes(uint32_t now); diff --git a/src/display/eyes/styles/kaomoji.cpp b/src/display/eyes/styles/kaomoji.cpp index 217507bb..a9bf9d75 100644 --- a/src/display/eyes/styles/kaomoji.cpp +++ b/src/display/eyes/styles/kaomoji.cpp @@ -32,6 +32,7 @@ expressions::Expression expressionFor( switch (mode) { case EyeMode::Idle: + case EyeMode::Waiting: return expressions::Expression::Idle; case EyeMode::Typing: case EyeMode::Thinking: diff --git a/src/display/eyes/styles/kawaii.cpp b/src/display/eyes/styles/kawaii.cpp index 7a7593eb..ce0e46f5 100644 --- a/src/display/eyes/styles/kawaii.cpp +++ b/src/display/eyes/styles/kawaii.cpp @@ -89,6 +89,8 @@ kawaii::Mood modeMood(EyeMode mode) { return deadShowingX() ? kawaii::Mood::Dizzy : kawaii::Mood::Sad; case EyeMode::Talking: return kawaii::Mood::Talking; + case EyeMode::Waiting: + return kawaii::Mood::Content; // calm; the pupils follow the head's corners } return kawaii::Mood::Content; diff --git a/src/hardware/rgb.cpp b/src/hardware/rgb.cpp index 0157d825..9a40c3e6 100644 --- a/src/hardware/rgb.cpp +++ b/src/hardware/rgb.cpp @@ -110,6 +110,7 @@ void resolveAnimationTarget(AnimationId id, uint8_t& r, uint8_t& g, uint8_t& b) case AnimationId::Reading: case AnimationId::Thinking: case AnimationId::Talking: + case AnimationId::Waiting: case AnimationId::Welcome: case AnimationId::Ring: case AnimationId::Wakeup: diff --git a/src/http/play_handlers.cpp b/src/http/play_handlers.cpp index 3fbb03d7..560c831d 100644 --- a/src/http/play_handlers.cpp +++ b/src/http/play_handlers.cpp @@ -48,7 +48,7 @@ class PlaySession : public WavStreamParser::Sink { // Continuous animations and none play no sound of their own, so they cannot talk over the clip. if (!parseAnimationName(name, animation_) || (animation_ != AnimationId::None && !animationIsContinuous(animation_))) { - refusal_ = Refusal{400, "name must be talking, typing, reading, thinking or none"}; + refusal_ = Refusal{400, "name must be talking, typing, reading, thinking, waiting or none"}; releaseBusy_(); return; } diff --git a/src/serial/play.cpp b/src/serial/play.cpp index 04b22d61..dc97b2ce 100644 --- a/src/serial/play.cpp +++ b/src/serial/play.cpp @@ -33,7 +33,7 @@ class SerialPlaySession : public WavStreamParser::Sink { } if (!parseAnimationName(name, animation_) || (animation_ != AnimationId::None && !animationIsContinuous(animation_))) { - error = "name must be talking, typing, reading, thinking or none"; + error = "name must be talking, typing, reading, thinking, waiting or none"; release_(); return false; } diff --git a/ui/src/config/ConfigForm.tsx b/ui/src/config/ConfigForm.tsx index b1b7bf64..381e1f34 100644 --- a/ui/src/config/ConfigForm.tsx +++ b/ui/src/config/ConfigForm.tsx @@ -146,7 +146,7 @@ export function ConfigForm({ settings }: { settings: Settings }) { value={draft.continuousTimeout} onInput={(event) => set({ continuousTimeout: event.currentTarget.value })} /> -

Max time for typing / reading / thinking / talking

+

Max time for typing / reading / thinking / talking / waiting

diff --git a/ui/src/lib/animations.ts b/ui/src/lib/animations.ts index c56ab42c..eff08934 100644 --- a/ui/src/lib/animations.ts +++ b/ui/src/lib/animations.ts @@ -5,6 +5,7 @@ export const ANIMATIONS = [ ["reading", "Reading", "Continuous"], ["thinking", "Thinking", "Continuous"], ["talking", "Talking", "Continuous"], + ["waiting", "Waiting", "Continuous"], ["ring", "Bell", "One-shot"], ["welcome", "Welcome", "One-shot"], ["wakeup", "Wakeup", "Sleep inertia"], diff --git a/ui/src/views/ApiReference.tsx b/ui/src/views/ApiReference.tsx index 7c425e07..b737aeff 100644 --- a/ui/src/views/ApiReference.tsx +++ b/ui/src/views/ApiReference.tsx @@ -275,6 +275,12 @@ export function ApiReference() { Speaking gesture; soft torso shifts (audio-reactive during /play) + + + waiting + + Turned to the left, taps the desk with the left hand, looks around + ring @@ -361,6 +367,12 @@ export function ApiReference() { Thinking gesture + + + waiting + + Waiting gesture + none From f73b4f9ce376398828aac38542240f22213d595d Mon Sep 17 00:00:00 2001 From: Victor Radu Date: Wed, 18 Nov 2026 00:26:33 +0000 Subject: [PATCH 2/2] feat(integrations): play waiting when a Claude Code session goes idle Map the Notification hook with notification_type idle_prompt (Claude finished about a minute ago and the user has not typed) to the new waiting animation. Other notification types keep attention. Add waiting to the CLI name lists for /anim and /play. Co-Authored-By: Claude Opus 5.5 --- packages/tiny-engineer-claude-code/src/cli.js | 5 +++-- packages/tiny-engineer-claude-code/src/map.js | 9 ++++++++- .../tiny-engineer-claude-code/test/map.test.js | 14 ++++++++++++++ .../src/integrations/claude-code/map.js | 9 ++++++++- packages/tiny-engineer-cli/src/robot/client.js | 2 ++ packages/tiny-engineer-cli/test/client.test.js | 2 ++ .../tiny-engineer-cli/test/map-claude-code.test.js | 14 ++++++++++++++ skills/tiny-engineer-play/SKILL.md | 2 +- skills/tiny-engineer-play/scripts/play | 2 +- 9 files changed, 53 insertions(+), 6 deletions(-) diff --git a/packages/tiny-engineer-claude-code/src/cli.js b/packages/tiny-engineer-claude-code/src/cli.js index ce1f2414..69967fbb 100644 --- a/packages/tiny-engineer-claude-code/src/cli.js +++ b/packages/tiny-engineer-claude-code/src/cli.js @@ -47,14 +47,15 @@ Event map: PreCompact → thinking PermissionRequest → attention PermissionDenied → abort - Notification → attention + Notification + idle_prompt → waiting + other Notification → attention Stop → ring StopFailure → error Play mode: Streams a 16-bit mono PCM WAV (at the robot's sample rate) to POST /play and runs one animation while it plays: talking (default), typing, reading, - thinking or none. The robot then returns to the loop it was in. + thinking, waiting or none. The robot then returns to the loop it was in. Prints the robot's JSON reply when the clip ends; exits 1 on any error. Examples: diff --git a/packages/tiny-engineer-claude-code/src/map.js b/packages/tiny-engineer-claude-code/src/map.js index 00810c14..478a795d 100644 --- a/packages/tiny-engineer-claude-code/src/map.js +++ b/packages/tiny-engineer-claude-code/src/map.js @@ -29,7 +29,7 @@ const EVENT_ANIM = new Map([ /** * Map a Claude Code hook payload to an animation name, or null to skip. - * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string }} event + * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string, notification_type?: string }} event * @returns {string | null} */ export function animationForEvent(event) { @@ -56,5 +56,12 @@ export function animationForEvent(event) { return event.reason === "clear" || event.reason === "resume" ? null : "sleep"; } + // Claude finished about a minute ago and the user has not typed: wait + // quietly. The firmware's continuous_timeout turns it into attention later. + // Every other notification asks for action and keeps attention. + if (name === "Notification" && event.notification_type === "idle_prompt") { + return "waiting"; + } + return EVENT_ANIM.get(name) ?? null; } diff --git a/packages/tiny-engineer-claude-code/test/map.test.js b/packages/tiny-engineer-claude-code/test/map.test.js index b450afd5..a666b7e7 100644 --- a/packages/tiny-engineer-claude-code/test/map.test.js +++ b/packages/tiny-engineer-claude-code/test/map.test.js @@ -48,6 +48,20 @@ test("lifecycle events map to their poses", () => { }), "attention" ); + assert.equal( + animationForEvent({ + hook_event_name: "Notification", + notification_type: "idle_prompt", + }), + "waiting" + ); + for (const notification_type of ["elicitation_dialog", "auth_success", undefined]) { + assert.equal( + animationForEvent({ hook_event_name: "Notification", notification_type }), + "attention", + String(notification_type) + ); + } assert.equal(animationForEvent({ hook_event_name: "Stop" }), "ring"); assert.equal(animationForEvent({ hook_event_name: "StopFailure" }), "error"); }); diff --git a/packages/tiny-engineer-cli/src/integrations/claude-code/map.js b/packages/tiny-engineer-cli/src/integrations/claude-code/map.js index 00810c14..478a795d 100644 --- a/packages/tiny-engineer-cli/src/integrations/claude-code/map.js +++ b/packages/tiny-engineer-cli/src/integrations/claude-code/map.js @@ -29,7 +29,7 @@ const EVENT_ANIM = new Map([ /** * Map a Claude Code hook payload to an animation name, or null to skip. - * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string }} event + * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string, notification_type?: string }} event * @returns {string | null} */ export function animationForEvent(event) { @@ -56,5 +56,12 @@ export function animationForEvent(event) { return event.reason === "clear" || event.reason === "resume" ? null : "sleep"; } + // Claude finished about a minute ago and the user has not typed: wait + // quietly. The firmware's continuous_timeout turns it into attention later. + // Every other notification asks for action and keeps attention. + if (name === "Notification" && event.notification_type === "idle_prompt") { + return "waiting"; + } + return EVENT_ANIM.get(name) ?? null; } diff --git a/packages/tiny-engineer-cli/src/robot/client.js b/packages/tiny-engineer-cli/src/robot/client.js index 835a1558..53b3f2b7 100644 --- a/packages/tiny-engineer-cli/src/robot/client.js +++ b/packages/tiny-engineer-cli/src/robot/client.js @@ -5,6 +5,7 @@ export const ANIM_NAMES = new Set([ "reading", "thinking", "talking", + "waiting", "ring", "welcome", "attention", @@ -21,6 +22,7 @@ export const PLAY_ANIM_NAMES = new Set([ "typing", "reading", "thinking", + "waiting", "none", ]); diff --git a/packages/tiny-engineer-cli/test/client.test.js b/packages/tiny-engineer-cli/test/client.test.js index 147198d6..8294bcc2 100644 --- a/packages/tiny-engineer-cli/test/client.test.js +++ b/packages/tiny-engineer-cli/test/client.test.js @@ -38,6 +38,7 @@ test("validateAnimName accepts firmware names", () => { assert.equal(validateAnimName("ring"), null); assert.equal(validateAnimName("typing"), null); assert.ok(ANIM_NAMES.has("sleep")); + assert.ok(ANIM_NAMES.has("waiting")); }); test("validateAnimName rejects unknown names", () => { @@ -50,6 +51,7 @@ test("validatePlayAnimName accepts play path names", () => { assert.equal(validatePlayAnimName("talking"), null); assert.equal(validatePlayAnimName("none"), null); assert.ok(PLAY_ANIM_NAMES.has("thinking")); + assert.ok(PLAY_ANIM_NAMES.has("waiting")); }); test("validatePlayAnimName rejects ring", () => { diff --git a/packages/tiny-engineer-cli/test/map-claude-code.test.js b/packages/tiny-engineer-cli/test/map-claude-code.test.js index dce7f116..78e2c31b 100644 --- a/packages/tiny-engineer-cli/test/map-claude-code.test.js +++ b/packages/tiny-engineer-cli/test/map-claude-code.test.js @@ -49,6 +49,20 @@ test("lifecycle events map to their poses", () => { }), "attention" ); + assert.equal( + animationForEvent({ + hook_event_name: "Notification", + notification_type: "idle_prompt", + }), + "waiting" + ); + for (const notification_type of ["elicitation_dialog", "auth_success", undefined]) { + assert.equal( + animationForEvent({ hook_event_name: "Notification", notification_type }), + "attention", + String(notification_type) + ); + } assert.equal(animationForEvent({ hook_event_name: "Stop" }), "ring"); assert.equal(animationForEvent({ hook_event_name: "StopFailure" }), "error"); }); diff --git a/skills/tiny-engineer-play/SKILL.md b/skills/tiny-engineer-play/SKILL.md index 57b13f29..1a56690d 100644 --- a/skills/tiny-engineer-play/SKILL.md +++ b/skills/tiny-engineer-play/SKILL.md @@ -33,6 +33,6 @@ It reads `TINY_ENGINEER_URL` (default `http://tiny-engineer.local`) and `TINY_EN | Reply | Fix | | --- | --- | | `415 expected 16-bit mono PCM at N Hz` | Re-run the `ffmpeg` step with `-ar N` | -| `400 name must be ...` | Use `talking`, `typing`, `reading`, `thinking` or `none` | +| `400 name must be ...` | Use `talking`, `typing`, `reading`, `thinking`, `waiting` or `none` | | `401 unauthorized` | Set `TINY_ENGINEER_TOKEN` to the robot's `access_token` | | `robot unreachable` | Check `TINY_ENGINEER_URL`; the robot must be on the same network | diff --git a/skills/tiny-engineer-play/scripts/play b/skills/tiny-engineer-play/scripts/play index 09e51e95..4a14a19f 100755 --- a/skills/tiny-engineer-play/scripts/play +++ b/skills/tiny-engineer-play/scripts/play @@ -1,5 +1,5 @@ #!/bin/sh -# Usage: play [talking|typing|reading|thinking|none] +# Usage: play [talking|typing|reading|thinking|waiting|none] # Streams the clip to the robot's POST /play and prints its JSON reply; exits 1 on any error. set -u