diff --git a/docs/api.md b/docs/api.md index f0fa8058..6b402643 100644 --- a/docs/api.md +++ b/docs/api.md @@ -194,7 +194,7 @@ curl http://tiny-engineer.local/settings | `serial_log` | USB serial debug logging (default **false**). When off, diagnostic `Serial` output is suppressed | | `push_button` | Extension-port push-button reader (default **false**). When on, publishes immediate `button.down` / `button.up` plus `button.click` / `button.double_click` / `button.long_click` (see [Outbound events](#outbound-events)). Applies **immediately**. Local reactions use [Actions](#get-settingsactions) | | `push_button_pin` | GPIO on the Extension port for the push-button: **6**, **7**, **20**, or **21** (default **20**). Persists when `push_button` is off. Applies **immediately** when the extension is on | -| `continuous_timeout` | Minutes a continuous animation (`typing` / `reading` / `thinking` / `talking`) may run before the firmware switches to `attention` then idle (default **5**) | +| `continuous_timeout` | Minutes a continuous animation (`typing` / `reading` / `thinking` / `talking` / `waiting`) may run before the firmware switches to `attention` then idle (default **5**) | | `loading` | Boot loading screen: `progress` (bar + large IP for 3 s) or `sleep_inertia` (eyes wake; no bar/IP). Default **`progress`**. Applies on **next reboot** | | `eyes_style` | OLED eyes look: `classic` (procedural rounded eyes), `kaomoji` (animated faces), `cover` (full-half bars for an eye-hole mask), `dots` (12×12 circles), or `kawaii` (round eyes with big moving pupils, a talking mouth, and blush). Default **`classic`**. Applies **immediately**. Classic, cover, and dots sequences are described under [`POST /anim`](#post-anim); kaomoji face mapping is under [Kaomoji eyes](#kaomoji-eyes); cover drawing is under [Cover eyes](#cover-eyes); dots drawing is under [Dots eyes](#dots-eyes); kawaii moods are under [Kawaii eyes](#kawaii-eyes) | | `access_token_set` | `true` when a non-empty access token is stored (secret itself is never returned) | @@ -593,7 +593,7 @@ curl http://tiny-engineer.local/anim | Field | Meaning | | --- | --- | -| `animation` | `none`, `typing`, `reading`, `thinking`, `talking`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | +| `animation` | `none`, `typing`, `reading`, `thinking`, `talking`, `waiting`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | ### `POST /anim` @@ -601,7 +601,7 @@ Request an animation switch. Each animation runs at least **1s**; if the current | Param | Type | Values | | --- | --- | --- | -| `name` | string | `none`, `typing`, `reading`, `thinking`, `talking`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, `sleep` | +| `name` | string | `none`, `typing`, `reading`, `thinking`, `talking`, `waiting`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, `sleep` | ```bash curl -X POST "http://tiny-engineer.local/anim?name=typing" @@ -634,6 +634,7 @@ Durations and quoted lines below are the stock clips in [`assets/`](../assets/). | `reading` | **Continuous.** Hands/body park as in `none`. Head nods on same lowest 10° band as typing, but slower. Neck sweeps ±10° around mid; right (angle down) slower than left (angle up). Occasional right-hand down-arrow bursts (1–3 presses, same 15° band as typing). Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | | `thinking` | **Continuous.** Hands/body park as in `none`. Head (pitch) and neck (yaw) ease from the current pose into thinking poses (up + slight left/right). Move → pause → optional micro-adjust (sometimes chained) → pause; nearby pose drift with occasional larger shifts after ~2.2 s, more often over time. Axes stagger start/duration; no periodic sway. Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | | `talking` | **Continuous.** Speaking fidget with no sound of its own. Also the default pose for [`POST /play`](#post-play). Via `/anim`, the head faces the user just above mid with soft random nods, slow neck drift, occasional small torso shifts, and occasional right-hand lifts. During `/play` with pose `talking`, motion follows speech energy from the PCM stream (no extra metadata): emphasis triggers nods, silence settles the pose, phrase starts may raise the hand or shift the torso (not both). **Classic eyes:** a little taller than idle; glances away when ambient or quiet, gaze locked on the listener while speech energy is high. Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | +| `waiting` | **Continuous.** Impatient wait with no sound. The torso turns to the robot's left (body `+0.8`), the right hand parks at rest, and the left hand taps the desk in groups of three (each tap lifts 0.18–0.28 of its range, groups every 2.5–8 s at random). Every 3–7 s the head eases (1.4–2.3 s) to one of four wide corners, never the same one twice: down at the tapping hand (most often), up-left, up-right, or down-right. **Classic eyes:** glance to the same corner about 180 ms before the head gets there (±5 px sideways, 3 px up / 2 px down), a little wider looking up and a little lower looking down. The Claude Code hooks use it when Claude has waited about a minute for the user (`Notification` with `idle_prompt`). Runs until replaced, or until `continuous_timeout` triggers `attention`. Same-name re-`POST` refreshes that timeout without restarting motion. | | `ring` | **One-shot** service-bell gesture; does not loop. Wind-up (body `min`, neck mid, head mid+10°, both hands `max`) → fast right-hand strike to `min+5°` with head to `min` → plays `bell.wav` once on strike (LittleFS; same `uploadfs` requirement as `/test/audio/bell`) → slower bounce to `min+20°` → return to `none` pose and stop. After completion, `GET /anim` reports `none`. | | `welcome` | **One-shot** hello gesture synced to `welcome.wav` (~2.7 s). Right hand raises during "Hello, human.", holds through the pause, wiggles during "What are we building today?", then lowers. Head nods to mid+10° and returns. Plays automatically after successful Wi-Fi connect at boot; also via API. Requires `welcome.wav` on LittleFS (same `uploadfs` flow as `bell.wav`). After completion, `GET /anim` reports `none`. | | `attention` | Friendly input-request gesture synced to `attention.wav` (~3.0 s, "pst... human.... you might want to take a look"). Moves into a calm prompt pose first (centered body/neck, head slightly up, right hand raised partway), waits until all servos stop, then plays audio with light neck/head/hand motion during playback (whisper hold → lean toward user → glance/point on "take a look"). **Classic eyes:** phased blink/look cues during audio. After audio ends (or if audio fails to start), holds a gentle waiting loop for **1 minute** (soft head/neck drifts plus occasional slight right-hand waves), then returns to `none`. Requires `attention.wav` on LittleFS (same `uploadfs` flow as `bell.wav`). | @@ -649,7 +650,7 @@ When `eyes_style=kaomoji`, the OLED shows looping faces from the expression libr | Animation / phase | Face | | --- | --- | -| `none` (idle) | Idle | +| `none` (idle), `waiting` | Idle | | `typing`, `thinking` | Thinking | | `reading`, `attention` | Curious | | `ring` | Surprise | @@ -688,6 +689,7 @@ While `welcome`, `attention`, `error`, `abort`, or `dead` plays its stock clip, | `abort` | Panic: `> <` eyes, wavy mouth; says "Fine. I didn't want to finish it anyway." | | `wakeup` | Sleepy: heavy lids, yawn | | `talking` | The mouth talks in syllables, shaped as `ah` (tall), `ee` (wide), or `o` (round). With `/play` audio it opens with the speech loudness and changes shape on each syllable onset. Without audio it babbles in short phrases with pauses | +| `waiting` | The idle smile (`ω`). The pupils follow the classic eyes to each corner the head looks at | | `sleep` / sleep-close | Lids close, small `ω` | | `dead` | Sad (lids tilted down at the outer corners, frown), then `x x` eyes during the X hold. Says "Insufficient resources... shutting doooowwwnnn" | @@ -696,15 +698,15 @@ Wrong params return **400**: | `error` | When | | --- | --- | | `missing name` | Query param `name` absent | -| `unknown animation` | `name` not `none`, `typing`, `reading`, `thinking`, `talking`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | +| `unknown animation` | `name` not `none`, `typing`, `reading`, `thinking`, `talking`, `waiting`, `ring`, `welcome`, `attention`, `error`, `abort`, `dead`, `wakeup`, or `sleep` | ### `POST /play` -Streams a WAV to the speaker while one animation runs, then returns to the continuous animation (`typing`, `reading`, `thinking`, `talking`) that was running before the request, otherwise to `none`. The body is played as it arrives, so clips of any length need no storage on the robot. The response comes when the clip ends. +Streams a WAV to the speaker while one animation runs, then returns to the continuous animation (`typing`, `reading`, `thinking`, `talking`, `waiting`) that was running before the request, otherwise to `none`. The body is played as it arrives, so clips of any length need no storage on the robot. The response comes when the clip ends. | Part | Value | | --- | --- | -| Path | `/play` runs `talking`; `/play/{name}` runs `talking`, `typing`, `reading`, `thinking`, or `none`. Animations with their own sound are rejected so two clips never overlap | +| Path | `/play` runs `talking`; `/play/{name}` runs `talking`, `typing`, `reading`, `thinking`, `waiting`, or `none`. Animations with their own sound are rejected so two clips never overlap | | Body | WAV, 16-bit mono PCM at 22050 Hz (the firmware `SAMPLE_RATE`). `Content-Length` is required. Chunks other than `fmt ` and `data` are skipped | | `Authorization` | `Bearer ` when `access_token` is set | @@ -725,7 +727,7 @@ Errors arrive before any audio plays: | Status | `error` | When | | --- | --- | --- | -| `400` | `name must be talking, typing, reading, thinking or none` | `{name}` is unknown or has its own sound | +| `400` | `name must be talking, typing, reading, thinking, waiting or none` | `{name}` is unknown or has its own sound | | `415` | `expected 16-bit mono PCM at 22050 Hz` | WAV format or sample rate differs | | `415` | `not a WAV file`, `no audio data` | Body is not a RIFF/WAVE file, or has no `data` chunk | @@ -759,7 +761,7 @@ Onboard WS2812 on **GPIO10** (`RGB_LED_PIN`). Byte order is the saved `rgb_order | Animation | LED color | | --- | --- | -| `typing`, `reading`, `thinking`, `talking`, `welcome`, `ring`, `wakeup` | White (full intensity) | +| `typing`, `reading`, `thinking`, `talking`, `waiting`, `welcome`, `ring`, `wakeup` | White (full intensity) | | `attention`, `error`, `dead` | Red pulse: 500 ms 10%→100%, 500 ms hold 100%, 500 ms 100%→10%, repeat | | `abort` | Red (full intensity, solid) | | `none`, `sleep` | Off | diff --git a/docs/hardware/components.md b/docs/hardware/components.md index 8cf5517e..f0bb61b3 100644 --- a/docs/hardware/components.md +++ b/docs/hardware/components.md @@ -34,7 +34,7 @@ Constants cited below match [`include/pins.h`](../../include/pins.h). | USB | Native USB on GPIO18/19 — reached via the main control board USB-C (onboard USB-C unused when assembled). Advanced path: Adafruit 5993 — [wiring.md](wiring.md#usb-connector-5993) | | Flash | 4 MB stacked flash on GPIO12–GPIO17 (those GPIOs are **not exposed**) | | Onboard RGB | WS2812 on **GPIO10** | -| Firmware notes | Boot: dim green status. Animations: white (typing/reading/thinking/talking/welcome/ring/wakeup), pulsing red (attention/error/dead), solid red (abort), off (idle/sleep). 1 s fade between non-pulse states. Fatal init errors: hang and blink red (1 = PCA9685, 2 = I2S). See [blink codes](testing.md#boot-failure-blink-codes) and [`docs/api.md`](../api.md#rgb-led) | +| Firmware notes | Boot: dim green status. Animations: white (typing/reading/thinking/talking/waiting/welcome/ring/wakeup), pulsing red (attention/error/dead), solid red (abort), off (idle/sleep). 1 s fade between non-pulse states. Fatal init errors: hang and blink red (1 = PCA9685, 2 = I2S). See [blink codes](testing.md#boot-failure-blink-codes) and [`docs/api.md`](../api.md#rgb-led) | | BOOT | Button / strapping on **GPIO9** | | RESET | Separate RESET button (CHIP_EN), not a GPIO | | UART0 silkscreen | TX = GPIO21, RX = GPIO20 | diff --git a/docs/hardware/testing.md b/docs/hardware/testing.md index 98d8aa16..638f01ac 100644 --- a/docs/hardware/testing.md +++ b/docs/hardware/testing.md @@ -70,7 +70,7 @@ During normal operation the onboard WS2812 tracks the active animation (not boot | Animation | LED | | --- | --- | -| `typing`, `reading`, `thinking`, `talking`, `welcome`, `ring`, `wakeup` | White | +| `typing`, `reading`, `thinking`, `talking`, `waiting`, `welcome`, `ring`, `wakeup` | White | | `attention`, `error`, `dead` | Pulsing red (10%↔100%, 1.5 s cycle) | | `abort` | Solid red | | `none`, `sleep` | Off | diff --git a/docs/hooks.md b/docs/hooks.md index bc217c55..21e540e7 100644 --- a/docs/hooks.md +++ b/docs/hooks.md @@ -197,6 +197,6 @@ Config: [`.cursor/hooks.json`](../.cursor/hooks.json). Local anim hook: [`.curso - Robot offline → hook still exits 0; no agent stall. - Animation API holds each pose ≥1s and keeps only the latest pending switch — see [`api.md`](api.md). -- The onboard RGB LED follows the active animation (white for typing/reading/thinking/talking/welcome/ring/wakeup, pulsing red for attention/error/dead, solid red for abort, off for `none`/`sleep`) with 1 s fades between non-pulse states — see [RGB LED](api.md#rgb-led). +- The onboard RGB LED follows the active animation (white for typing/reading/thinking/talking/waiting/welcome/ring/wakeup, pulsing red for attention/error/dead, solid red for abort, off for `none`/`sleep`) with 1 s fades between non-pulse states — see [RGB LED](api.md#rgb-led). - To change the map, edit [`packages/tiny-engineer-cli/src/integrations/cursor/map.js`](../packages/tiny-engineer-cli/src/integrations/cursor/map.js). - Custom agents or non-Cursor IDEs: call the [REST API](integration.md#2-rest-api-advanced--custom). diff --git a/docs/integration.md b/docs/integration.md index 6635fe0d..5fc38115 100644 --- a/docs/integration.md +++ b/docs/integration.md @@ -150,6 +150,7 @@ curl -X POST "http://tiny-engineer.local/anim?name=typing" | `welcome` | Session start / greeting | | `wakeup` | Sleep-inertia wake (eyes/head) | | `sleep` | Force sleep (eye close + OLED off) | +| `waiting` | Turn finished and the user has not answered for a while (quiet, before `attention`) | | `attention` | Needs user input | | `error` / `abort` | Failure / cancel | | `dead` | Out of power (`dead.wav`, then X X hold) | diff --git a/packages/tiny-engineer-claude-code/src/cli.js b/packages/tiny-engineer-claude-code/src/cli.js index ce1f2414..69967fbb 100644 --- a/packages/tiny-engineer-claude-code/src/cli.js +++ b/packages/tiny-engineer-claude-code/src/cli.js @@ -47,14 +47,15 @@ Event map: PreCompact → thinking PermissionRequest → attention PermissionDenied → abort - Notification → attention + Notification + idle_prompt → waiting + other Notification → attention Stop → ring StopFailure → error Play mode: Streams a 16-bit mono PCM WAV (at the robot's sample rate) to POST /play and runs one animation while it plays: talking (default), typing, reading, - thinking or none. The robot then returns to the loop it was in. + thinking, waiting or none. The robot then returns to the loop it was in. Prints the robot's JSON reply when the clip ends; exits 1 on any error. Examples: diff --git a/packages/tiny-engineer-claude-code/src/map.js b/packages/tiny-engineer-claude-code/src/map.js index 00810c14..478a795d 100644 --- a/packages/tiny-engineer-claude-code/src/map.js +++ b/packages/tiny-engineer-claude-code/src/map.js @@ -29,7 +29,7 @@ const EVENT_ANIM = new Map([ /** * Map a Claude Code hook payload to an animation name, or null to skip. - * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string }} event + * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string, notification_type?: string }} event * @returns {string | null} */ export function animationForEvent(event) { @@ -56,5 +56,12 @@ export function animationForEvent(event) { return event.reason === "clear" || event.reason === "resume" ? null : "sleep"; } + // Claude finished about a minute ago and the user has not typed: wait + // quietly. The firmware's continuous_timeout turns it into attention later. + // Every other notification asks for action and keeps attention. + if (name === "Notification" && event.notification_type === "idle_prompt") { + return "waiting"; + } + return EVENT_ANIM.get(name) ?? null; } diff --git a/packages/tiny-engineer-claude-code/test/map.test.js b/packages/tiny-engineer-claude-code/test/map.test.js index b450afd5..a666b7e7 100644 --- a/packages/tiny-engineer-claude-code/test/map.test.js +++ b/packages/tiny-engineer-claude-code/test/map.test.js @@ -48,6 +48,20 @@ test("lifecycle events map to their poses", () => { }), "attention" ); + assert.equal( + animationForEvent({ + hook_event_name: "Notification", + notification_type: "idle_prompt", + }), + "waiting" + ); + for (const notification_type of ["elicitation_dialog", "auth_success", undefined]) { + assert.equal( + animationForEvent({ hook_event_name: "Notification", notification_type }), + "attention", + String(notification_type) + ); + } assert.equal(animationForEvent({ hook_event_name: "Stop" }), "ring"); assert.equal(animationForEvent({ hook_event_name: "StopFailure" }), "error"); }); diff --git a/packages/tiny-engineer-cli/src/integrations/claude-code/map.js b/packages/tiny-engineer-cli/src/integrations/claude-code/map.js index 00810c14..478a795d 100644 --- a/packages/tiny-engineer-cli/src/integrations/claude-code/map.js +++ b/packages/tiny-engineer-cli/src/integrations/claude-code/map.js @@ -29,7 +29,7 @@ const EVENT_ANIM = new Map([ /** * Map a Claude Code hook payload to an animation name, or null to skip. - * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string }} event + * @param {{ hook_event_name?: string, tool_name?: string, source?: string, reason?: string, notification_type?: string }} event * @returns {string | null} */ export function animationForEvent(event) { @@ -56,5 +56,12 @@ export function animationForEvent(event) { return event.reason === "clear" || event.reason === "resume" ? null : "sleep"; } + // Claude finished about a minute ago and the user has not typed: wait + // quietly. The firmware's continuous_timeout turns it into attention later. + // Every other notification asks for action and keeps attention. + if (name === "Notification" && event.notification_type === "idle_prompt") { + return "waiting"; + } + return EVENT_ANIM.get(name) ?? null; } diff --git a/packages/tiny-engineer-cli/src/robot/client.js b/packages/tiny-engineer-cli/src/robot/client.js index 835a1558..53b3f2b7 100644 --- a/packages/tiny-engineer-cli/src/robot/client.js +++ b/packages/tiny-engineer-cli/src/robot/client.js @@ -5,6 +5,7 @@ export const ANIM_NAMES = new Set([ "reading", "thinking", "talking", + "waiting", "ring", "welcome", "attention", @@ -21,6 +22,7 @@ export const PLAY_ANIM_NAMES = new Set([ "typing", "reading", "thinking", + "waiting", "none", ]); diff --git a/packages/tiny-engineer-cli/test/client.test.js b/packages/tiny-engineer-cli/test/client.test.js index 147198d6..8294bcc2 100644 --- a/packages/tiny-engineer-cli/test/client.test.js +++ b/packages/tiny-engineer-cli/test/client.test.js @@ -38,6 +38,7 @@ test("validateAnimName accepts firmware names", () => { assert.equal(validateAnimName("ring"), null); assert.equal(validateAnimName("typing"), null); assert.ok(ANIM_NAMES.has("sleep")); + assert.ok(ANIM_NAMES.has("waiting")); }); test("validateAnimName rejects unknown names", () => { @@ -50,6 +51,7 @@ test("validatePlayAnimName accepts play path names", () => { assert.equal(validatePlayAnimName("talking"), null); assert.equal(validatePlayAnimName("none"), null); assert.ok(PLAY_ANIM_NAMES.has("thinking")); + assert.ok(PLAY_ANIM_NAMES.has("waiting")); }); test("validatePlayAnimName rejects ring", () => { diff --git a/packages/tiny-engineer-cli/test/map-claude-code.test.js b/packages/tiny-engineer-cli/test/map-claude-code.test.js index dce7f116..78e2c31b 100644 --- a/packages/tiny-engineer-cli/test/map-claude-code.test.js +++ b/packages/tiny-engineer-cli/test/map-claude-code.test.js @@ -49,6 +49,20 @@ test("lifecycle events map to their poses", () => { }), "attention" ); + assert.equal( + animationForEvent({ + hook_event_name: "Notification", + notification_type: "idle_prompt", + }), + "waiting" + ); + for (const notification_type of ["elicitation_dialog", "auth_success", undefined]) { + assert.equal( + animationForEvent({ hook_event_name: "Notification", notification_type }), + "attention", + String(notification_type) + ); + } assert.equal(animationForEvent({ hook_event_name: "Stop" }), "ring"); assert.equal(animationForEvent({ hook_event_name: "StopFailure" }), "error"); }); diff --git a/skills/tiny-engineer-play/SKILL.md b/skills/tiny-engineer-play/SKILL.md index 57b13f29..1a56690d 100644 --- a/skills/tiny-engineer-play/SKILL.md +++ b/skills/tiny-engineer-play/SKILL.md @@ -33,6 +33,6 @@ It reads `TINY_ENGINEER_URL` (default `http://tiny-engineer.local`) and `TINY_EN | Reply | Fix | | --- | --- | | `415 expected 16-bit mono PCM at N Hz` | Re-run the `ffmpeg` step with `-ar N` | -| `400 name must be ...` | Use `talking`, `typing`, `reading`, `thinking` or `none` | +| `400 name must be ...` | Use `talking`, `typing`, `reading`, `thinking`, `waiting` or `none` | | `401 unauthorized` | Set `TINY_ENGINEER_TOKEN` to the robot's `access_token` | | `robot unreachable` | Check `TINY_ENGINEER_URL`; the robot must be on the same network | diff --git a/skills/tiny-engineer-play/scripts/play b/skills/tiny-engineer-play/scripts/play index 09e51e95..4a14a19f 100755 --- a/skills/tiny-engineer-play/scripts/play +++ b/skills/tiny-engineer-play/scripts/play @@ -1,5 +1,5 @@ #!/bin/sh -# Usage: play [talking|typing|reading|thinking|none] +# Usage: play [talking|typing|reading|thinking|waiting|none] # Streams the clip to the robot's POST /play and prints its JSON reply; exits 1 on any error. set -u diff --git a/src/animation.h b/src/animation.h index 8c7a144e..30735e68 100644 --- a/src/animation.h +++ b/src/animation.h @@ -15,7 +15,8 @@ enum class AnimationId { Wakeup, Sleep, Dead, - Talking + Talking, + Waiting }; void setAnimation(AnimationId id); diff --git a/src/animation/registry.cpp b/src/animation/registry.cpp index 2c3c3ce7..333fc245 100644 --- a/src/animation/registry.cpp +++ b/src/animation/registry.cpp @@ -15,6 +15,7 @@ #include "animation/thinking.h" #include "animation/typing.h" #include "animation/util.h" +#include "animation/waiting.h" #include "animation/wakeup.h" #include "animation/welcome.h" #include "display/eyes/modes/abort.h" @@ -27,6 +28,7 @@ #include "display/eyes/modes/talking.h" #include "display/eyes/modes/thinking.h" #include "display/eyes/modes/typing.h" +#include "display/eyes/modes/waiting.h" #include "display/eyes/modes/wakeup.h" #include "display/eyes/modes/welcome.h" #include "hardware/servo_wrapper.h" @@ -275,6 +277,17 @@ constexpr ModeEntry kModes[] = { startTalkingEyes, updateTalkingEyes, }, + { + AnimationId::Waiting, + EyeMode::Waiting, + "waiting", + true, + -1, + startWaiting, + updateWaiting, + startWaitingEyes, + updateWaitingEyes, + }, }; constexpr size_t kModeCount = sizeof(kModes) / sizeof(kModes[0]); diff --git a/src/animation/waiting.cpp b/src/animation/waiting.cpp new file mode 100644 index 00000000..dc4b91c0 --- /dev/null +++ b/src/animation/waiting.cpp @@ -0,0 +1,291 @@ +#include + +#include "animation/constants.h" +#include "animation/util.h" +#include "animation/waiting.h" +#include "hardware/servo_wrapper.h" +#include "servos.h" +#include "serial_log.h" + +using anim::beginEasedMove; +using anim::easedMoveDone; +using anim::easedMoveValue; +using anim::EasedMove; +using anim::randChance; +using anim::randRangeMs; +using anim::randUnit; + +namespace { + +// Torso turned to the robot's left, away from the bell (body + = left). +constexpr float WAIT_BODY = 0.8f; +constexpr float WAIT_SETTLE_SPEED_DEG_S = 30.0f; + +// Left hand rests on the desk (+1, its scale is inverted) and lifts toward - +// a little for each tap. +constexpr int WAIT_TAP_SERVO = SERVO_HAND_LEFT; +constexpr float WAIT_HAND_REST = 1.0f; +constexpr float WAIT_TAP_DIR = -1.0f; +constexpr float WAIT_TAP_LIFT_MIN = 0.18f; +constexpr float WAIT_TAP_LIFT_SPAN = 0.10f; +constexpr float WAIT_TAP_UP_SPEED_DEG_S = 70.0f; +constexpr float WAIT_TAP_DOWN_SPEED_DEG_S = 100.0f; +constexpr uint32_t WAIT_TAP_GROUP_GAP_MIN_MS = 2500; +constexpr uint32_t WAIT_TAP_GROUP_GAP_MAX_MS = 8000; +constexpr uint32_t WAIT_TAP_IN_GROUP_GAP_MIN_MS = 90; +constexpr uint32_t WAIT_TAP_IN_GROUP_GAP_MAX_MS = 220; +// Every group is three taps; the groups start at random times. +constexpr uint8_t WAIT_TAPS_PER_GROUP = 3; + +// Head corners. Neck: - = the robot's right, + = its left. Head: + = up. +constexpr float WAIT_HEAD_JITTER = 0.06f; +constexpr float WAIT_NECK_JITTER = 0.05f; +constexpr uint32_t WAIT_LOOK_HOLD_MIN_MS = 3000; +constexpr uint32_t WAIT_LOOK_HOLD_MAX_MS = 7000; +// Longer moves for the wide corners keep the head speed calm. +constexpr uint32_t WAIT_LOOK_MOVE_MIN_MS = 1400; +constexpr uint32_t WAIT_LOOK_MOVE_MAX_MS = 2300; +constexpr uint32_t WAIT_LOOK_STAGGER_MAX_MS = 200; + +struct WaitCorner { + const char* name; + float headNorm; + float neckNorm; + float gazeX; // eyes: -1 = robot's right, +1 = robot's left + float gazeY; // eyes: -1 = down, +1 = up + uint8_t weight; +}; + +// Index 0 looks at the tapping hand: down and a little to the left. +constexpr WaitCorner WAIT_CORNERS[] = { + {"hand (SW)", -0.90f, 0.30f, 0.6f, -1.0f, 34}, + {"up-left (NW)", 0.60f, 0.70f, 1.0f, 1.0f, 22}, + {"up-right (NE)", 0.60f, -0.70f, -1.0f, 1.0f, 22}, + {"down-right (SE)", -0.70f, -0.65f, -1.0f, -1.0f, 22}, +}; +constexpr uint8_t WAIT_CORNER_COUNT = + sizeof(WAIT_CORNERS) / sizeof(WAIT_CORNERS[0]); +constexpr uint8_t WAIT_HAND_CORNER = 0; + +enum class WaitPhase { + Settle, + Run, +}; + +enum class TapPhase { + Idle, + Lift, + Drop, + Gap, +}; + +WaitPhase g_phase = WaitPhase::Settle; +TapPhase g_tapPhase = TapPhase::Idle; +uint32_t g_nextTapMs = 0; +uint8_t g_tapsLeft = 0; + +uint8_t g_corner = WAIT_HAND_CORNER; +uint32_t g_nextLookMs = 0; +EasedMove g_headMove = {}; +EasedMove g_neckMove = {}; +WaitingGaze g_gaze = {0.0f, 0.0f, 0}; + +float jitter(float amount) { + return (randUnit() * 2.0f - 1.0f) * amount; +} + +uint8_t pickNextCorner() { + uint32_t total = 0; + for (uint8_t i = 0; i < WAIT_CORNER_COUNT; i++) { + if (i != g_corner) { + total += WAIT_CORNERS[i].weight; + } + } + + uint32_t roll = randRangeMs(0, total - 1); + for (uint8_t i = 0; i < WAIT_CORNER_COUNT; i++) { + if (i == g_corner) { + continue; + } + if (roll < WAIT_CORNERS[i].weight) { + return i; + } + roll -= WAIT_CORNERS[i].weight; + } + return WAIT_HAND_CORNER; +} + +void setGaze(uint8_t corner, uint32_t now) { + g_gaze.x = WAIT_CORNERS[corner].gazeX; + g_gaze.y = WAIT_CORNERS[corner].gazeY; + g_gaze.sinceMs = now == 0 ? 1 : now; +} + +void beginLook(uint8_t corner, uint32_t now) { + const WaitCorner& c = WAIT_CORNERS[corner]; + const float headFrom = + servoDegToNorm(SERVO_HEAD, servoAt(SERVO_HEAD).angle()); + const float neckFrom = + servoDegToNorm(SERVO_NECK, servoAt(SERVO_NECK).angle()); + const float headTo = servoSaturateNorm(c.headNorm + jitter(WAIT_HEAD_JITTER)); + const float neckTo = servoSaturateNorm(c.neckNorm + jitter(WAIT_NECK_JITTER)); + + // Eyes lead, then head and neck follow; one of them starts a bit later. + const uint32_t stagger = randRangeMs(0, WAIT_LOOK_STAGGER_MAX_MS); + const bool headFirst = randChance(50); + beginEasedMove( + g_headMove, + headFrom, + headTo, + headFirst ? now : now + stagger, + randRangeMs(WAIT_LOOK_MOVE_MIN_MS, WAIT_LOOK_MOVE_MAX_MS) + ); + beginEasedMove( + g_neckMove, + neckFrom, + neckTo, + headFirst ? now + stagger : now, + randRangeMs(WAIT_LOOK_MOVE_MIN_MS, WAIT_LOOK_MOVE_MAX_MS) + ); + + g_corner = corner; + setGaze(corner, now); + g_nextLookMs = now + randRangeMs(WAIT_LOOK_HOLD_MIN_MS, WAIT_LOOK_HOLD_MAX_MS); + + serialLogPrint("[anim] waiting look "); + serialLogPrintln(c.name); +} + +void tickLook(uint32_t now) { + if (g_headMove.active) { + servoAt(SERVO_HEAD).setNormPosition(easedMoveValue(g_headMove, now)); + if (easedMoveDone(g_headMove, now)) { + g_headMove.active = false; + } + } + if (g_neckMove.active) { + servoAt(SERVO_NECK).setNormPosition(easedMoveValue(g_neckMove, now)); + if (easedMoveDone(g_neckMove, now)) { + g_neckMove.active = false; + } + } + + const bool moving = g_headMove.active || g_neckMove.active; + if (!moving && now >= g_nextLookMs) { + beginLook(pickNextCorner(), now); + } +} + +void scheduleTapGroup(uint32_t now) { + g_tapPhase = TapPhase::Idle; + g_nextTapMs = + now + randRangeMs(WAIT_TAP_GROUP_GAP_MIN_MS, WAIT_TAP_GROUP_GAP_MAX_MS); +} + +void tickTaps(uint32_t now) { + ServoWrapper& hand = servoAt(WAIT_TAP_SERVO); + + switch (g_tapPhase) { + case TapPhase::Idle: + if (now >= g_nextTapMs) { + g_tapsLeft = WAIT_TAPS_PER_GROUP; + g_tapPhase = TapPhase::Lift; + hand.setNormTarget( + WAIT_HAND_REST + + WAIT_TAP_DIR * (WAIT_TAP_LIFT_MIN + WAIT_TAP_LIFT_SPAN * randUnit()), + WAIT_TAP_UP_SPEED_DEG_S + ); + } + break; + + case TapPhase::Lift: + if (!hand.isMoving()) { + g_tapPhase = TapPhase::Drop; + hand.setNormTarget(WAIT_HAND_REST, WAIT_TAP_DOWN_SPEED_DEG_S); + } + break; + + case TapPhase::Drop: + if (!hand.isMoving()) { + g_tapsLeft--; + if (g_tapsLeft == 0) { + scheduleTapGroup(now); + } else { + g_tapPhase = TapPhase::Gap; + g_nextTapMs = now + randRangeMs( + WAIT_TAP_IN_GROUP_GAP_MIN_MS, + WAIT_TAP_IN_GROUP_GAP_MAX_MS + ); + } + } + break; + + case TapPhase::Gap: + if (now >= g_nextTapMs) { + g_tapPhase = TapPhase::Lift; + hand.setNormTarget( + WAIT_HAND_REST + + WAIT_TAP_DIR * (WAIT_TAP_LIFT_MIN + WAIT_TAP_LIFT_SPAN * randUnit()), + WAIT_TAP_UP_SPEED_DEG_S + ); + } + break; + } +} + +bool settled() { + return !servoAt(SERVO_BODY).isMoving() + && !servoAt(SERVO_HAND_LEFT).isMoving() + && !servoAt(SERVO_HAND_RIGHT).isMoving() + && !servoAt(SERVO_HEAD).isMoving() + && !servoAt(SERVO_NECK).isMoving(); +} + +} // namespace + +void startWaiting(uint32_t animationStartedMs) { + serialLogPrintln("[anim] startWaiting"); + anim::stopAnimServos(); + + // Turn to the left, park the hands, and look at the tapping hand. + const WaitCorner& hand = WAIT_CORNERS[WAIT_HAND_CORNER]; + servoAt(SERVO_BODY).setNormTarget(WAIT_BODY, WAIT_SETTLE_SPEED_DEG_S); + anim::parkHands(anim::TRANSITION_HAND_SPEED_DEG_S); + servoAt(SERVO_HEAD).setNormTarget(hand.headNorm, WAIT_SETTLE_SPEED_DEG_S); + servoAt(SERVO_NECK).setNormTarget(hand.neckNorm, WAIT_SETTLE_SPEED_DEG_S); + + g_phase = WaitPhase::Settle; + g_tapPhase = TapPhase::Idle; + g_tapsLeft = 0; + g_headMove.active = false; + g_neckMove.active = false; + g_corner = WAIT_HAND_CORNER; + setGaze(WAIT_HAND_CORNER, animationStartedMs); +} + +void updateWaiting(uint32_t now) { + servoAt(SERVO_BODY).update(); + servoAt(SERVO_HAND_LEFT).update(); + servoAt(SERVO_HAND_RIGHT).update(); + + if (g_phase == WaitPhase::Settle) { + servoAt(SERVO_HEAD).update(); + servoAt(SERVO_NECK).update(); + if (settled()) { + g_phase = WaitPhase::Run; + g_nextLookMs = + now + randRangeMs(WAIT_LOOK_HOLD_MIN_MS, WAIT_LOOK_HOLD_MAX_MS); + // The first tap comes soon, so the pose reads as "waiting" at once. + g_nextTapMs = now + randRangeMs(600, 1500); + serialLogPrintln("[anim] waiting settled"); + } + return; + } + + tickLook(now); + tickTaps(now); +} + +WaitingGaze waitingGaze() { + return g_gaze; +} diff --git a/src/animation/waiting.h b/src/animation/waiting.h new file mode 100644 index 00000000..58f12aae --- /dev/null +++ b/src/animation/waiting.h @@ -0,0 +1,17 @@ +#pragma once + +#include + +// Where the waiting head looks, for the eyes to follow. +// x: -1 = the robot's right, +1 = the robot's left. +// y: -1 = down, +1 = up. +// sinceMs changes each time the head picks a new corner. +struct WaitingGaze { + float x; + float y; + uint32_t sinceMs; +}; + +void startWaiting(uint32_t animationStartedMs); +void updateWaiting(uint32_t now); +WaitingGaze waitingGaze(); diff --git a/src/display/eyes.h b/src/display/eyes.h index b2cd0541..aba324fd 100644 --- a/src/display/eyes.h +++ b/src/display/eyes.h @@ -21,7 +21,8 @@ enum class EyeMode { Abort, Wakeup, Dead, - Talking + Talking, + Waiting }; enum class SleepEyeResult { diff --git a/src/display/eyes/modes/waiting.cpp b/src/display/eyes/modes/waiting.cpp new file mode 100644 index 00000000..e417d158 --- /dev/null +++ b/src/display/eyes/modes/waiting.cpp @@ -0,0 +1,86 @@ +#include "display/eyes/modes/waiting.h" + +#include "animation/util.h" +#include "animation/waiting.h" +#include "display/eyes.h" +#include "display/eyes/core/constants.h" +#include "display/eyes/core/internal.h" +#include "display/eyes/core/util.h" + +namespace { + +// The eyes look to the corner the head turns to. They get there first +// (a quick glance), then the head follows over about a second. +constexpr float GAZE_RANGE_X_PX = 5.0f; // same full glance as error/attention +constexpr float GAZE_UP_PX = 3.0f; +constexpr float GAZE_DOWN_PX = 2.0f; +constexpr uint32_t GAZE_MOVE_MS = 180; + +// Lids follow the gaze: a little wider looking up, a little lower looking down. +constexpr int16_t HEIGHT_UP = 15; +constexpr int16_t HEIGHT_DOWN = 12; + +float g_fromX = 0.0f; +float g_fromY = 0.0f; +float g_toX = 0.0f; +float g_toY = 0.0f; +float g_x = 0.0f; +float g_y = 0.0f; +uint32_t g_gazeSinceMs = 0; +uint32_t g_moveStartMs = 0; + +} // namespace + +void startWaitingEyes(uint32_t now) { + g_fromX = 0.0f; + g_fromY = 0.0f; + g_toX = 0.0f; + g_toY = 0.0f; + g_x = 0.0f; + g_y = 0.0f; + g_gazeSinceMs = 0; + g_moveStartMs = now; +} + +void updateWaitingEyes(uint32_t now) { + const WaitingGaze gaze = waitingGaze(); + if (gaze.sinceMs != g_gazeSinceMs) { + g_gazeSinceMs = gaze.sinceMs; + g_fromX = g_x; + g_fromY = g_y; + g_toX = gaze.x; + g_toY = gaze.y; + g_moveStartMs = now; + } + + const float t = anim::easeInOutCubic( + eyes::moveProgress(now, g_moveStartMs, GAZE_MOVE_MS) + ); + g_x = anim::lerp(g_fromX, g_toX, t); + g_y = anim::lerp(g_fromY, g_toY, t); + + // Screen x follows the neck sign: the robot's right is -x (see error eyes). + const int16_t offsetX = (int16_t)(g_x * GAZE_RANGE_X_PX); + const int16_t offsetY = (int16_t)(g_y >= 0.0f + ? -g_y * GAZE_UP_PX + : -g_y * GAZE_DOWN_PX); + const int16_t height = g_y >= 0.0f + ? eyes::lerpInt(eyes::DEFAULT_LEFT.height, HEIGHT_UP, g_y) + : eyes::lerpInt(eyes::DEFAULT_LEFT.height, HEIGHT_DOWN, -g_y); + + Eye& left = mutableLeftEye(); + Eye& right = mutableRightEye(); + + left = eyes::eyeWithHeight( + (int16_t)(eyes::DEFAULT_LEFT.x + offsetX), + eyes::DEFAULT_LEFT.width, + height + ); + right = eyes::eyeWithHeight( + (int16_t)(eyes::DEFAULT_RIGHT.x + offsetX), + eyes::DEFAULT_RIGHT.width, + height + ); + left.y += offsetY; + right.y += offsetY; +} diff --git a/src/display/eyes/modes/waiting.h b/src/display/eyes/modes/waiting.h new file mode 100644 index 00000000..54326902 --- /dev/null +++ b/src/display/eyes/modes/waiting.h @@ -0,0 +1,6 @@ +#pragma once + +#include + +void startWaitingEyes(uint32_t now); +void updateWaitingEyes(uint32_t now); diff --git a/src/display/eyes/styles/kaomoji.cpp b/src/display/eyes/styles/kaomoji.cpp index 217507bb..a9bf9d75 100644 --- a/src/display/eyes/styles/kaomoji.cpp +++ b/src/display/eyes/styles/kaomoji.cpp @@ -32,6 +32,7 @@ expressions::Expression expressionFor( switch (mode) { case EyeMode::Idle: + case EyeMode::Waiting: return expressions::Expression::Idle; case EyeMode::Typing: case EyeMode::Thinking: diff --git a/src/display/eyes/styles/kawaii.cpp b/src/display/eyes/styles/kawaii.cpp index 7a7593eb..ce0e46f5 100644 --- a/src/display/eyes/styles/kawaii.cpp +++ b/src/display/eyes/styles/kawaii.cpp @@ -89,6 +89,8 @@ kawaii::Mood modeMood(EyeMode mode) { return deadShowingX() ? kawaii::Mood::Dizzy : kawaii::Mood::Sad; case EyeMode::Talking: return kawaii::Mood::Talking; + case EyeMode::Waiting: + return kawaii::Mood::Content; // calm; the pupils follow the head's corners } return kawaii::Mood::Content; diff --git a/src/hardware/rgb.cpp b/src/hardware/rgb.cpp index 0157d825..9a40c3e6 100644 --- a/src/hardware/rgb.cpp +++ b/src/hardware/rgb.cpp @@ -110,6 +110,7 @@ void resolveAnimationTarget(AnimationId id, uint8_t& r, uint8_t& g, uint8_t& b) case AnimationId::Reading: case AnimationId::Thinking: case AnimationId::Talking: + case AnimationId::Waiting: case AnimationId::Welcome: case AnimationId::Ring: case AnimationId::Wakeup: diff --git a/src/http/play_handlers.cpp b/src/http/play_handlers.cpp index 3fbb03d7..560c831d 100644 --- a/src/http/play_handlers.cpp +++ b/src/http/play_handlers.cpp @@ -48,7 +48,7 @@ class PlaySession : public WavStreamParser::Sink { // Continuous animations and none play no sound of their own, so they cannot talk over the clip. if (!parseAnimationName(name, animation_) || (animation_ != AnimationId::None && !animationIsContinuous(animation_))) { - refusal_ = Refusal{400, "name must be talking, typing, reading, thinking or none"}; + refusal_ = Refusal{400, "name must be talking, typing, reading, thinking, waiting or none"}; releaseBusy_(); return; } diff --git a/src/serial/play.cpp b/src/serial/play.cpp index 04b22d61..dc97b2ce 100644 --- a/src/serial/play.cpp +++ b/src/serial/play.cpp @@ -33,7 +33,7 @@ class SerialPlaySession : public WavStreamParser::Sink { } if (!parseAnimationName(name, animation_) || (animation_ != AnimationId::None && !animationIsContinuous(animation_))) { - error = "name must be talking, typing, reading, thinking or none"; + error = "name must be talking, typing, reading, thinking, waiting or none"; release_(); return false; } diff --git a/ui/src/config/ConfigForm.tsx b/ui/src/config/ConfigForm.tsx index b1b7bf64..381e1f34 100644 --- a/ui/src/config/ConfigForm.tsx +++ b/ui/src/config/ConfigForm.tsx @@ -146,7 +146,7 @@ export function ConfigForm({ settings }: { settings: Settings }) { value={draft.continuousTimeout} onInput={(event) => set({ continuousTimeout: event.currentTarget.value })} /> -

Max time for typing / reading / thinking / talking

+

Max time for typing / reading / thinking / talking / waiting

diff --git a/ui/src/lib/animations.ts b/ui/src/lib/animations.ts index c56ab42c..eff08934 100644 --- a/ui/src/lib/animations.ts +++ b/ui/src/lib/animations.ts @@ -5,6 +5,7 @@ export const ANIMATIONS = [ ["reading", "Reading", "Continuous"], ["thinking", "Thinking", "Continuous"], ["talking", "Talking", "Continuous"], + ["waiting", "Waiting", "Continuous"], ["ring", "Bell", "One-shot"], ["welcome", "Welcome", "One-shot"], ["wakeup", "Wakeup", "Sleep inertia"], diff --git a/ui/src/views/ApiReference.tsx b/ui/src/views/ApiReference.tsx index 7c425e07..b737aeff 100644 --- a/ui/src/views/ApiReference.tsx +++ b/ui/src/views/ApiReference.tsx @@ -275,6 +275,12 @@ export function ApiReference() { Speaking gesture; soft torso shifts (audio-reactive during /play) + + + waiting + + Turned to the left, taps the desk with the left hand, looks around + ring @@ -361,6 +367,12 @@ export function ApiReference() { Thinking gesture + + + waiting + + Waiting gesture + none