#101: port room_view roster recognition path#102
Merged
Conversation
The room_view roster recognition path emits face_recognized events on roster match, completing the named-greet half that PR #93's bare-greet suppression defers to. Without it, known household members get silent walk-ins instead of "Hi Brett!". This commit lands the pure logic — prompt template, closed-set system prompt, builder, and deterministic parser — as a standalone module so it's testable without FastAPI / asyncio. Wiring into /api/vision/ explain follows in the next commit. Ported verbatim from bridge.py (the retired ZeroClaw bridge, still in the repo as the dashboard service): - bridge.py:561-571 (system prompt) - bridge.py:594 (sentinel) - bridge.py:3370-3413 (prompt template + regex + moods) - bridge.py:3416-3440 (builder) - bridge.py:3443-3482 (parser) Adaptations: - Builder takes registry as a parameter (was a module global on bridge.py); enables unit-testing with a small fake instead of a YAML fixture. - Registry parameter typed against a small _RegistryLike Protocol so the module stays decoupled from HouseholdRegistry's full interface. 17 unit tests cover sentinel/constant contract, builder happy path + empty-registry fallback + raising-registry, parser closed-set match, unknown name, off-roster name, format mismatch, missing/invalid mood, trailing punctuation tolerance. All green; full suite 194/194 (was 177 — 17 new tests). Refs #101 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Route layer for the room_view path. On question == "__ROOM_VIEW_V1__":
- build roster-aware prompt from the attached HouseholdRegistry
- gate-check: skip on dance_active / talk_active / cooldown (mirrors
bridge.py:3518-3552); cached no-person + waiter signal on skip
- VLM call with ROOM_VIEW_SYSTEM_PROMPT (closed-set name vocab, doubles
as the kid-mode safety guard since only roster names + "unknown"
can come back)
- parse → cache with source="room_view" + room_match_person_id
- broadcast PerceptionEvent(name="face_recognized", data={"identity":...,
"source": "room_view"}) on roster match — the bus signal FaceGreeter
+ ProactiveGreeter need to fire the named greet
- plumb mood into perception_state[device]["face_mood"] for snapshot
Falls back to the v1 description path when the registry is empty or
absent — keeps the source="room_view" cache attribution so the
dashboard distinguishes capture types.
The gate logic lives in a private _room_view_gates_block helper rather
than inline so the cooldown integration test can exercise the branch
without simulating dance/talk state.
4 integration tests added:
- happy path: roster match → cache populated correctly + face_recognized
on bus + face_mood in perception state
- off-roster: cache populated, NO face_recognized event
- cooldown: second call within window skips VLM, caches sentinel
- empty registry: sentinel falls back to v1 question, no broadcast
Full suite 198/198 (was 194 — 4 new integration tests). Existing v1
route tests untouched.
Closes #101
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The flat-layout Dockerfile explicitly lists each top-level package as a separate COPY line. The new vision/ package (commit 67e9909) wasn't added to that list, so the deployed image lacked the module and container startup hit `ModuleNotFoundError: No module named 'vision'` on the routes/vision.py import. Refs #101 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Ports the bridge.py “room_view sentinel → roster-aware VLM prompt → deterministic parse → face_recognized broadcast” path into dotty-behaviour so PR #93’s bare-greet suppression is backed by a working named-greet flow.
Changes:
- Added
vision/room_view.pywith the__ROOM_VIEW_V1__sentinel contract, roster-aware prompt builder, and parser (plus unit tests). - Updated
/api/vision/explainto detect the sentinel, apply bridge-like talk/dance/cooldown gates, cachesource="room_view", plumb mood, and broadcastface_recognizedon roster match. - Added integration tests covering the sentinel branch behavior (cache/bus/mood, no-match, cooldown, empty-roster fallback).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| dotty-behaviour/vision/room_view.py | New prompt-builder + parser module for room_view roster recognition. |
| dotty-behaviour/vision/init.py | Exposes room_view helpers/constants as a small importable API. |
| dotty-behaviour/tests/test_vision_room_view.py | Unit tests for the room_view prompt/template + parser behavior. |
| dotty-behaviour/tests/test_routes_vision.py | Integration tests for the /api/vision/explain room_view sentinel branch. |
| dotty-behaviour/routes/vision.py | Implements the sentinel branch, gating, caching, mood plumbing, and face_recognized broadcast. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+115
to
+118
| name_choices = "|".join(sorted( | ||
| p.display_name for p in registry.iter() | ||
| if (getattr(p, "appearance", None) or "").strip() | ||
| )) |
Comment on lines
+159
to
+167
| desc = m.group("desc").strip() | ||
| name = m.group("name").strip().lower() | ||
| raw_mood = (m.group("mood") or "").strip().lower() | ||
| mood = raw_mood if raw_mood in ROOM_VIEW_MOODS else None | ||
| if not desc: | ||
| desc = None | ||
| if name == "unknown" or name not in roster_ids: | ||
| return desc, None, mood | ||
| return desc, name, mood |
Comment on lines
+191
to
+205
| roster_ids = ( | ||
| household.roster_ids_with_appearance() | ||
| if household is not None else set() | ||
| ) | ||
| raw = await vlm.describe_image( | ||
| b64_image, | ||
| room_view_question, | ||
| system_prompt=ROOM_VIEW_SYSTEM_PROMPT, | ||
| timeout_s=config.VISION_TIMEOUT_SEC, | ||
| ) | ||
| parsed_desc, room_match_person_id, parsed_mood = ( | ||
| parse_room_view_response(raw, roster_ids) | ||
| ) | ||
| description = parsed_desc or ROOM_VIEW_NO_PERSON | ||
| effective_question = room_view_question |
Comment on lines
+210
to
+213
| def roster_ids_with_appearance(self) -> set[str]: | ||
| return { | ||
| p.display_name.lower() for p in self.people if p.appearance | ||
| } |
Comment on lines
+108
to
+111
|
|
||
| _ROSTER = {"brett", "hudson"} | ||
|
|
||
|
|
Comment on lines
+71
to
+74
| # tolerates trailing punctuation around the name (e.g. `NAME: Hudson.`). | ||
| _ROOM_VIEW_RESP_RE = re.compile( | ||
| r"^\s*DESC:\s*(?P<desc>.+?)\s*" | ||
| r"\|\s*NAME:\s*(?P<name>[A-Za-z_][A-Za-z0-9_-]*)\s*" |
4 tasks
BrettKinny
added a commit
that referenced
this pull request
May 23, 2026
#102's broadcast emit fanned the event out to bus subscribers (FaceGreeter / scene_synthesis) but skipped the per-device state mutation that POST /api/perception/event normally does — so `last_face_id` + `last_face_recognized_t` never landed on state.state[device_id]. Downstream readers that depend on these fields silently no-op'd: - face_identified_refresher (reads last_face_id to keep right-ring pixel 6 green past the firmware's 4 s timeout — pixel was going yellow ↔ green ↔ off instead of holding green while the user stayed in frame) - perception/snapshot.py reading last_face_id for the talk-turn PerceptionSnapshot ("Speaking with: …" hint to the LLM) - state.get_fresh_face_id() — the canonical accessor Caught during the #44 bench sweep right after #102 merged. The fix mirrors what POST /api/perception/event does: update_state then broadcast. The integration test grew assertions on the two state fields so the regression can't reappear silently. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 23, 2026
BrettKinny
added a commit
that referenced
this pull request
May 23, 2026
Tightens PR #93's bare-greet suppression. The previous _roster_is_populated() check suppressed bare "Hi!" whenever the household had ANY member, but the named-greet path (PR #102) requires `appearance:` set to match a person via the room_view VLM. A roster with members configured but no appearances silently walked everyone in forever. Now: suppress only when at least one roster member is identifiable (has non-empty appearance). Members without appearance can't reach the named-greet path, so they correctly fall back to the bare "Hi!". The existing `test_face_detected_suppressed_when_roster_non_empty` was asserting the buggy behaviour; inverted to test the correct semantics. Existing appearance-bearing suppression test preserved. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 23, 2026
BrettKinny
added a commit
that referenced
this pull request
May 23, 2026
* #111: rip ACP/voice path from bridge.py ACPClient + MessageIn/MessageOut + /api/message + /api/message/stream + voice tool helpers + voice escalate/memory_log/remember/remember_person + _voice_preparer + _ConvoLogger are gone. /health drops acp_running / cached_session / session_turns. The dashboard's per-person memory page keeps its three brain.db read/mutate helpers (_voice_memory_person_records_blocking / _voice_memory_approve_blocking / _voice_memory_delete_blocking) since /ui/memory is still live. Perception consumers + vision/audio endpoints stay for commits 2-3 of the #111 rip. _greeter_llm_client is stubbed to RuntimeError so the ProactiveGreeter (still wired into lifespan until commit 2) fails fast on its first prompt() call rather than NameError on a missing acp ref. bridge.py: 6079 → 4564 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * #111: rip perception consumers from bridge.py All 13 in-process perception consumers — face_lost_aborter, sound_turner, wake_word_turner, face_greeter, face_identified_refresher, purr_player, named_acknowledger, scene_synthesis (loop is partial — see commit 3), idle_photographer, sleep_dreamer, dance_reflector, security_capture, ProactiveGreeter wiring — are gone. The in-memory perception bus (_perception_state / _perception_listeners / _perception_recent_events / _sound_balance_history) goes with them, along with /api/perception/event + /api/perception/state + /api/perception/feed, the legacy convo-log last_user_line cache, and the HouseholdRegistry / SpeakerResolver wiring. Calendar machinery (_fetch_weather / _fetch_calendar_events / _refresh_caches / _calendar_poll_loop / summarize_for_prompt / _build_context / _build_perception_block / Event TypedDict) goes too — dotty-behaviour serves /api/calendar/today now (per #100 / dotty-behaviour/routes/calendar.py). The dashboard perception card now reads empty stubs (_dashboard_perception_state_getter / *_recent_getter / *_last_user_line_getter / *_sound_balance_series / *_vision_failures_last_hour all return empty), so the card renders without errors but shows no data until the dashboard ports to dotty-behaviour. _identity_display_name returns None for the same reason. Dispatch helpers kept: _dispatch_abort, _dispatch_set_state, _dispatch_set_tier1slim_model, _dispatch_set_toggle — all four are needed by the dashboard / admin endpoints. The consumer-only dispatchers (face_greeting / say / set_head_angles / set_face_identified / purr_audio) went with the consumers. bridge.py: 4564 → 2868 lines. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * #111: rip VLM / vision / audio code from bridge.py /api/vision/explain, /api/vision/latest/{device_id}, /api/audio/explain are gone. The room_view sentinel branch (VISION_ROOM_VIEW_SYSTEM_PROMPT, __ROOM_VIEW_V1__, _build_room_view_question, _parse_room_view_response, _ROOM_VIEW_* constants) goes with it — dotty-behaviour now owns the room_view roster recognition path (PR #101 / #102 / #103). _call_vision_api + VLM HTTP plumbing + _classify_vision_result + _call_narrative_llm + _call_audio_caption_api + _audio_format_from_upload go too — dotty-behaviour owns every VLM call now (and dotty-pi-ext owns narrative calls from voice tools). Scene-synthesis / idle-photographer / dream / dance NDJSON writers (_scene_synthesis_log_path / _idle_perception_log_path / _write_idle_perception_record / _is_notable_perception / _idle_photographer_pick_device / _dreams_log_path / _dances_log_path / _write_jsonl_record / _write_dream_record / _write_dance_record / _split_dream_text / _compose_scene_synthesis / _write_scene_synthesis_ndjson / _maybe_emit_scene_synthesis / _perception_sleep_dreamer / _perception_dance_reflector / _perception_idle_photographer / _scene_synthesis_loop) are gone — every consumer they fed lives in dotty-behaviour. The legacy /admin/* endpoints that targeted the retired ZeroClaw daemon (_apply_model_swap, _read_voice_model_from_cfg, _voice_profile_for_model, _admin_schedule_restart, _ADMIN_DAEMON_CFG, /admin/model) were retired alongside — /admin/persona stays for external persona-file edits, /admin/safety stays as a static MCP_TOOL_ALLOWLIST mutation surface, /admin/smart-mode keeps just the tier1slim hot-swap path, /admin/state + /admin/kid-mode keep their xiaozhi-admin passthrough. bridge.py: 2868 → 911 lines. Final state: dashboard-only. Cache stubs (_vision_cache / _audio_cache / _scene_synthesis_cache / _perception_state) all empty dicts so the dashboard perception card renders without errors but shows no data until it ports to dotty-behaviour. The brain.db memory funcs (_voice_memory_person_records_blocking / *_approve_blocking / *_delete_blocking) survive because /ui/memory is the only operator surface on per-person memory rows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the room_view →
face_recognizedchain frombridge.pytodotty-behaviour, completing the named-greet path PR #93's bare-greet suppression defers to.Why
Bench-tested 2026-05-23: PR #93 correctly suppressed the bare "Hi!" but the named greet that should replace it never fired —
routes/vision.py:9-13documented the room_view path as "intentionally NOT ported". Net effect for known household members: silent walk-ins, strictly worse UX than pre-#93. See #101.What landed
Commit 1 —
dotty-behaviour/vision/room_view.py(new module):ROOM_VIEW_SENTINEL = "__ROOM_VIEW_V1__"— wire contract with xiaozhi-patchesROOM_VIEW_SYSTEM_PROMPT— closed-set name vocab (doubles as kid-mode safety guard)build_room_view_question(registry)— substitutes household roster into prompt; takes registry as parameter (was module global in bridge.py) so unit tests don't need YAML fixturesparse_room_view_response(raw, roster_ids)— deterministic parser; graceful fallback to description-only on format mismatchCommit 2 —
dotty-behaviour/routes/vision.py(sentinel branch):question == ROOM_VIEW_SENTINEL, builds roster-aware prompt_room_view_gates_blockhelper: skip VLM on dance_active / talk_active (outside kickoff grace) / cooldown — mirrors bridge.py:3518-3552source="room_view"+room_match_person_id; plumbs mood intoperception_state[device]["face_mood"]PerceptionEvent(name="face_recognized", data={"identity": pid, "source": "room_view"})on roster match — the bus signal FaceGreeter / ProactiveGreeter consume to fire "Oh, it's Brett!"source="room_view"cache attribution)Test plan
pytest tests/test_vision_room_view.py tests/test_routes_vision.py -q— 27 passedpytest tests/ -q— 198 passed (was 177 before; +21 new tests, 0 regressions)BEHAVIOUR_HOST=root@tower.local bash scripts/deploy-behaviour.shface_recognizedevent on bus and named greet fires within 1-3s offace_detected(this is the user-visible verification of Port room_view roster recognition path from bridge.py to dotty-behaviour #101)Scope notes
_classify_vision_result/ failure-aggregator metrics (bridge.py:3667-3669) are NOT ported — they're dashboard code not yet present in dotty-behaviour. No regression vs. the current state.Closes #101
🤖 Generated with Claude Code