Repository navigation
feat(agent): the plot follows the site theme - #12505
Merged
Merged
Conversation
The owner's decision of 2026-11-01, after user test E2: on the dark site the first plot rendered light, a bright card on a dark page. Each turn now carries the theme the site shows, and a plot run renders in it unless the user asked for a theme in the chat. Still one render per version; the light and dark switch renders the other theme. - The page sends `POST .../messages?theme=<light|dark>` (the effective theme, so `system` resolves first). A query parameter, not a body field: the BFF's and the agents service's bodies are strict, and an unknown parameter lets the page, the API and the service deploy in any order. - `pipeline.run_theme` picks the run's theme: the one the root passed (the user asked), the asked theme of the version a change builds on (`CodeVersion.theme_asked`), the site theme of the turn, and without a page the previous version's theme, else light. - The fan-out's sessions use the same hook, so they follow the same theme. - The root prompt and the tool description say what an omitted theme now means; the BFF mock follows the same rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
From the Gemini Flash pre-check: the asked theme in both directions, a new plot after an asked theme (`.start_over()` follows the site again), and the theme toggle as a view that never decides the next version's theme. The BFF mock now reads the word "light" as a request too, as the root does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The reviewed changes have no active blocking findings, with remaining feedback confined to documentation nits and a development-mock behavior concern.
No open issues 🎉
What changed in this PR
This pull request makes generated plots follow the site’s effective light or dark theme while preserving explicitly requested themes across refinements. Theme information is propagated from the frontend through the BFF to the agents service, with backward-compatible fallbacks for page-less callers.
Changes:
- Sends
?theme=light|darkwith agent turns and fan-out requests. - Adds theme precedence and persistence for new plots, refinements, toggles, and
.start_over(). - Updates the mock, tests, documentation, prompts, schemas, ledger, and changelog.
| File | Description | Review findings |
|---|---|---|
tests/unit/api/test_agent_router.py |
Tests BFF theme forwarding, omission, and validation. | — |
tests/unit/agents/runtime/test_service_flow.py |
Covers site-theme rendering, requested-theme persistence, changes, toggles, and fallbacks. | — |
tests/unit/agents/runtime/test_pipeline.py |
Tests theme precedence. | — |
docs/reference/api.md |
Documents the messages endpoint’s optional theme query parameter. | — |
docs/concepts/agent-network.md |
Describes site-theme propagation and pipeline behavior. | — |
changelog.d/agent-plot-follows-site-theme.md |
Adds the user-facing changelog entry. | — |
app/src/sections/agent-chat/FanOut.test.tsx |
Verifies fan-out turns use the page theme. | — |
app/src/pages/AgentChatPage.test.tsx |
Updates request matching to tolerate query parameters. | — |
app/src/lib/agent.ts |
Adds optional theme query construction for chat requests. | — |
app/src/hooks/useAgentSession.ts |
Captures and sends the effective theme for each turn. | — |
app/src/hooks/useAgentSession.test.ts |
Tests theme propagation and site-theme changes between turns. | — |
app/scripts/agent-bff-mock.mjs |
Mirrors site-theme handling and requested-theme precedence in the development mock. | Moderate: Refinements can inherit the previous library’s theme instead of the current library’s theme; select the newest version belonging to the active library. |
api/routers/agent.py |
Validates and forwards the theme query parameter to the agents service. | — |
agents/main.py |
Accepts the theme query parameter and stores it on the request ledger. | Nit: Document the 422 response for invalid theme values to match the validation behavior and BFF documentation. |
agents/anyplot/tools/session.py |
Updates the plot_pipeline tool description for omitted themes. |
Nit: Document both theme='light' and theme='dark' as supported explicit values. |
agents/anyplot/services.py |
Records whether a version’s theme was explicitly requested. | — |
agents/anyplot/schemas.py |
Updates PipelineArgs theme documentation. |
— |
agents/anyplot/prompts/root.md |
Instructs the root agent on site themes and explicit light/dark requests. | — |
agents/anyplot/plugins/ledger.py |
Adds the current site theme to request state. | — |
agents/anyplot/pipeline.py |
Implements theme precedence, persistence, rendering, and version storage. | Nit: Update the related agents/anyplot/agent.py module documentation to distinguish theme_asked from the current turn’s site theme. |
Review effort: Lite
- The BFF mock's refinement changes the current library's latest version, as the service's `latest_rendered(library=)` does, not the session's latest of any library. - The `plot_pipeline` description names `theme='light'` as well. - The route tables name the `422` for a theme other than light or dark. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
The lead's ask of 2026-11-01: every committed matrix baseline renders light, so nothing measures dark first renders yet. `--theme dark` sends the site theme with every turn, as the chat page does, and the stamp records it. The model-free preflight stays light: it checks only the renderer. No dark matrix runs in this PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
6 tasks done
MarkusNeusinger
added a commit
that referenced
this pull request
Nov 1, 2026
Brings in #12505 and #12503. The one conflict, the messages row of docs/reference/api.md, takes main's `?theme=` wording and keeps this branch's summary clause. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
4 of 5 tasks
MarkusNeusinger
added a commit
that referenced
this pull request
Nov 1, 2026
## Summary - Owner decision of 2026-11-01 after the user test: "Replies should be short and to the point, no waffle, and follow-up questions only when they really matter." `root.md` gets a "Replies" section. A reply leads with the result in two or three sentences, asks a question only when the turn cannot go on without the answer or the answer changes the data or the plot materially, closes with no invitation, and acts on the user's answer in the same turn. Brevity never drops a notice the data rules require: the loader's "reads", "keeps time" and "converts" lines, data changes with their counts, `requested_not_found`, binding warnings, tool errors and limits. The step-1 summary no longer asks a question by default and lists nothing the data does not have. - The "what changed" notes are now written in the language the user chats in and in plain words, without the code's own names such as `load_user_data()`, without what stayed as it was, and a fix lists only itself (`adapter.md`, and the self-review line in `pipeline.py`). - The prep eval measures every reply: words, questions, and whether it closes with a question. A closing question in a turn that was not blocked, or a reply over 100 words, is a reply miss, gated at 10 % (`--max-reply-misses`). A turn is blocked when the case is tagged `ask_back`, a data call failed, or a required role is left without a column. A case can send the page's `summarize_data` action, and four summary cases join the set. Decided and documented here: - **"Offer to ..." in `root.md`:** the information stays, said as one short statement, through one rule in "Replies" rather than an edit of every passage. This keeps the diff away from the lines #12503 and #12504 change. Only `needs_attention` drops its "offer to fix one" outright: the user asks when they want a fix. - **Notes language (lead, after #12504 merged):** the reply language, since the notes explain the plot to the user. `AdaptRequest.reply_language` carries the session language only when it differs from the plot language, fenced as `<reply_language>` after the plot language. The adapter then writes `changes` in it and keeps the plot text in the plot language. Without the field the two languages are the same. - **No server-side deduplication of notes:** the repeats in the test were paraphrases from the self-review call, which no string comparison catches. The prompt and the self-review line now address the cause, and the final chat run showed no repeated note. - **Length bound:** 100 words, about the five sentences `root.md` allows at most. ## Measurements Claude Haiku 5.5 on Vertex AI `eu`, $0.985 in all. Seven scripted chats rebuilt from the user test (A1, A3, B1, B4 and three summaries, two of them run twice), through `/v1` in process with the deployed renderer, 20 replies per arm. The baseline arm ran on the `origin/main` prompts. | | main | this branch | |---|---|---| | Words per reply, mean (median, max) | 58.7 (55, 80) | 50.4 (52, 71) | | Questions per reply | 0.75 | 0 | | Replies that close with a question | 13 of 20 | 0 of 20 | | Summaries: words, with a question | 60.6, 11 of 11 | 55.1, 0 of 11 | | Plot replies: words | 50.2 | 37.0 | - **A3, "The scores are out of 100.":** on main the root called no tool and answered "If you want the value axis to run from 0 to 100, tell me". The next plot kept the data's maximum. On this branch the root calls `plot_pipeline` with "Set the value axis to run from 0 to 100 ...", and the axis runs from 0 to 100. - **B1 notes:** on main they were English under a German reply and named `loadUserData()`. On this branch they are German, with no code names. The prep eval (55 cases) ran with the same eval code on both arms: | | main | this branch | |---|---|---| | Exact operation match | 53 of 55 | 55 of 55 | | Reply words, median (max) | 69 (123) | 57 (121) | | Questions per reply | 0.50 | 0.04 | | Replies that close with a question | 20 of 52 | 2 of 52 | | Reply misses (gate 10 %) | 21 of 52, FAIL | 3 of 52, PASS | The main arm ran before the open-role rule joined the blocked definition, so its reply-miss count is not comparable one to one. The closing-question and question counts are. Pre-PR Gemini 3.8 Flash check ($0.21, three findings): - Applied: a statement of what is possible may close a reply, and only an invitation may not. Applied: the policy test covers all seven loader prefixes. - Skipped: counting a question anywhere in the reply as a reply miss. The brief defines the check as a closing question, and the report still shows questions per reply. Copilot review (one round, three findings): - Applied: the true median for an even number of replies, with a test. Applied: the golden adapter-prompt test is renamed for what it now pins. - Skipped, as a false positive: the "high" `NameError` on `PrepCase`'s return annotation. The module has `from __future__ import annotations` and runs on Python 3.14, and its tests pass. ## Plan N/A (brief from the lead session; owner decision of 2026-11-01) ## Test plan - [x] `ruff check .`, `ruff format --check .`, `mypy api core agents` - [x] `pytest tests/unit tests/integration -n 6`: 25,857 passed. New tests cover the "Replies" rules and every protected notice, the action-or-message case model, the reply measurement (closing question, long reply, and blocked by `ask_back`, by a failed call or by an open role), the reply gate, and a scripted summary run. - [x] Live chat scenarios and the prep eval, as above. - [x] After merging `origin/main` (#12505, #12503, #12504) and the reply-language commit: `ruff`, `mypy api core agents`, and `pytest tests/unit tests/integration -n 6` with 25,934 passed. Two flow tests through `/v1` pin that a German plot under English replies gets its notes in English and the reverse, and that no `<reply_language>` block appears while the two languages agree. - [ ] Not measured live: the last notes rule ("never what stayed as it was") and the reply-language notes, since the $1 measurement budget was spent ($0.985). ## Open points - The server's own loader lines ("R reads ...") stay English in the notes list, because the root reads them by their English prefixes. - The regression matrix does not read replies, so plot-turn replies were measured with a scratch driver only. Adding a reply measurement to `matrix.py` would collide with #12503. - Haiku still adds "confirm the data in the data panel" to about half of the summaries: one short sentence, left as it is. https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
POST .../messages?theme=light|dark.systemmode resolves before it is sent. The BFF validates the value and forwards it. The agents service keeps it on the request ledger, andpipeline.run_themepicks the run's theme in this order:CodeVersion.theme_asked, which the change inherits).adk web, the evals, a BFF deployed earlier), the previous version's theme, else light. This is the old behaviour..plot_all()plots every library in the site theme. The root prompt and the tool description say what an omitted theme means now. The dev BFF mock follows the same rule.--theme darkfor the eval matrix (agents/evals/matrix.py, 13 lines; the lead's ask). Every turn of the run sends the site theme, as the page does. The stamp records it astheme. The model-free preflight stays light, because it only checks the renderer.Decisions (decide-and-document)
extra="forbid"), and they deploy on separate triggers. A body field would make every turn a422whenever the page or the BFF deploys first. An unknown query parameter is ignored. Gemini Flash went through all six deploy orders: none fails, and the theme only takes effect once all three tiers are live. There is a precedent:GET .../artifacts?theme=..start_over()) follows the site theme again.Open points
reviewer.mdtakes one labelled render in either theme, and VQ-07 names both backgrounds. R1 to R3 and the G-probes are geometric or colour-share checks, and grounding reads the drawn texts. But every eval measurement so far renders light: ladder_lab, stress, score, and every committed matrix baseline. So the agent quality numbers say nothing yet about the dark first renders that dark-site users now get. This PR adds--theme darkto the matrix and runs no dark matrix. One dark smoke or full run, with--no-baseline, should happen before the public release.keep_plot_request) carries no theme. "Make it dark" typed before.confirm()is lost on a light site, as before this change.Plan
N/A (lead brief of 2026-11-01)
Test plan
uv run ruff check .,uv run ruff format --check .,mypy api core agents: cleanpytest tests/unit/agents tests/unit/api -n auto: 20,683 passed. One render-cancellation test flaked once under xdist and passed alone; it is unrelated.run_themeprecedence table.422.422with no upstream call.?theme=follows the site theme at the start of the turn.yarn lint,fm:check,type-check,test(1,353 passed) andbuild./verify-frontendwith the BFF mock (Playwright, own tab), at both viewports:plot-dark.png, with no render call. The toggle to light renders on demand. After switching the site to light, the next change gets a light v2 (?theme=light; pixel#FAF8F1).eu, fake renderer, about 322k tokens, roughly $0.5 at list price.create_ploton?theme=darkgaveplot-dark.png. "make the markers bigger" on?theme=lightgaveplot-light.png, so the real root leftthemeout of the change.themestays out of refinements. The live check covered one case, and an eval case is out of scope.theme='light', and the route tables name the422.test_theme_dark_sends_the_site_theme_and_renders_dark, and the eval suite passes (429 tests). No live dark matrix was run.https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8