[MarkusNeusinger]

Summary

• Owner decision of 2026-11-01 after the user test: "Replies should be short and to the point, no waffle, and follow-up questions only when they really matter." root.md gets a "Replies" section. A reply leads with the result in two or three sentences, asks a question only when the turn cannot go on without the answer or the answer changes the data or the plot materially, closes with no invitation, and acts on the user's answer in the same turn. Brevity never drops a notice the data rules require: the loader's "reads", "keeps time" and "converts" lines, data changes with their counts, requested_not_found, binding warnings, tool errors and limits. The step-1 summary no longer asks a question by default and lists nothing the data does not have.
• The "what changed" notes are now written in the language the user chats in and in plain words, without the code's own names such as load_user_data(), without what stayed as it was, and a fix lists only itself (adapter.md, and the self-review line in pipeline.py).
• The prep eval measures every reply: words, questions, and whether it closes with a question. A closing question in a turn that was not blocked, or a reply over 100 words, is a reply miss, gated at 10 % (--max-reply-misses). A turn is blocked when the case is tagged ask_back, a data call failed, or a required role is left without a column. A case can send the page's summarize_data action, and four summary cases join the set.

Decided and documented here:

• "Offer to ..." in root.md: the information stays, said as one short statement, through one rule in "Replies" rather than an edit of every passage. This keeps the diff away from the lines fix(agent): read the drawn texts of every library for the reply #12503 and fix(agent): a follow-up message keeps the plot's language #12504 change. Only needs_attention drops its "offer to fix one" outright: the user asks when they want a fix.
• Notes language (lead, after fix(agent): a follow-up message keeps the plot's language #12504 merged): the reply language, since the notes explain the plot to the user. AdaptRequest.reply_language carries the session language only when it differs from the plot language, fenced as <reply_language> after the plot language. The adapter then writes changes in it and keeps the plot text in the plot language. Without the field the two languages are the same.
• No server-side deduplication of notes: the repeats in the test were paraphrases from the self-review call, which no string comparison catches. The prompt and the self-review line now address the cause, and the final chat run showed no repeated note.
• Length bound: 100 words, about the five sentences root.md allows at most.

Measurements

Claude Haiku 5.5 on Vertex AI eu, $0.985 in all.

Seven scripted chats rebuilt from the user test (A1, A3, B1, B4 and three summaries, two of them run twice), through /v1 in process with the deployed renderer, 20 replies per arm. The baseline arm ran on the origin/main prompts.

	main	this branch
Words per reply, mean (median, max)	58.7 (55, 80)	50.4 (52, 71)
Questions per reply	0.75	0
Replies that close with a question	13 of 20	0 of 20
Summaries: words, with a question	60.6, 11 of 11	55.1, 0 of 11
Plot replies: words	50.2	37.0

• A3, "The scores are out of 100.": on main the root called no tool and answered "If you want the value axis to run from 0 to 100, tell me". The next plot kept the data's maximum. On this branch the root calls plot_pipeline with "Set the value axis to run from 0 to 100 ...", and the axis runs from 0 to 100.
• B1 notes: on main they were English under a German reply and named loadUserData(). On this branch they are German, with no code names.

The prep eval (55 cases) ran with the same eval code on both arms:

	main	this branch
Exact operation match	53 of 55	55 of 55
Reply words, median (max)	69 (123)	57 (121)
Questions per reply	0.50	0.04
Replies that close with a question	20 of 52	2 of 52
Reply misses (gate 10 %)	21 of 52, FAIL	3 of 52, PASS

The main arm ran before the open-role rule joined the blocked definition, so its reply-miss count is not comparable one to one. The closing-question and question counts are.

Pre-PR Gemini 3.8 Flash check ($0.21, three findings):

• Applied: a statement of what is possible may close a reply, and only an invitation may not. Applied: the policy test covers all seven loader prefixes.
• Skipped: counting a question anywhere in the reply as a reply miss. The brief defines the check as a closing question, and the report still shows questions per reply.

Copilot review (one round, three findings):

• Applied: the true median for an even number of replies, with a test. Applied: the golden adapter-prompt test is renamed for what it now pins.
• Skipped, as a false positive: the "high" NameError on PrepCase's return annotation. The module has from __future__ import annotations and runs on Python 3.14, and its tests pass.

Plan

N/A (brief from the lead session; owner decision of 2026-11-01)

Test plan

• ruff check ., ruff format --check ., mypy api core agents
• pytest tests/unit tests/integration -n 6: 25,857 passed. New tests cover the "Replies" rules and every protected notice, the action-or-message case model, the reply measurement (closing question, long reply, and blocked by ask_back, by a failed call or by an open role), the reply gate, and a scripted summary run.
• Live chat scenarios and the prep eval, as above.
• After merging origin/main (feat(agent): the plot follows the site theme #12505, fix(agent): read the drawn texts of every library for the reply #12503, fix(agent): a follow-up message keeps the plot's language #12504) and the reply-language commit: ruff, mypy api core agents, and pytest tests/unit tests/integration -n 6 with 25,934 passed. Two flow tests through /v1 pin that a German plot under English replies gets its notes in English and the reverse, and that no <reply_language> block appears while the two languages agree.
• Not measured live: the last notes rule ("never what stayed as it was") and the reply-language notes, since the $1 measurement budget was spent ($0.985).

Open points

• The server's own loader lines ("R reads ...") stay English in the notes list, because the root reads them by their English prefixes.
• The regression matrix does not read replies, so plot-turn replies were measured with a scratch driver only. Adding a reply measurement to matrix.py would collide with fix(agent): read the drawn texts of every library for the reply #12503.
• Haiku still adds "confirm the data in the data panel" to about half of the summaries: one short sentence, left as it is.

https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8

[Copilot]

🟡 Changes recommended

An import-blocking NameError in PrepCase plus an incorrect even-count median calculation and stale golden-test documentation leave active correctness and maintainability issues.

3 open issues

• Quote the PrepCase forward annotation · New
• Calculate the true median for even-sized reply sets · New
• Rename the prompt golden test to match its current purpose · New

What changed in this PR

This pull request makes agent replies shorter and more purposeful, asks follow-up questions only when necessary, and acts on user answers in the same turn. It also extends prep evaluation with summary actions, reply metrics, blocked-turn handling, and a reply-miss gate.

Changes:

• Adds concise reply and meaningful-question guidance.
• Updates adapter notes to use plain, plot-language descriptions without code-specific names.
• Adds summary-action fixtures and reply behavior measurements.
• Updates tests, documentation, README guidance, and the changelog.

This pull request requires changes.

File	Description
tests/​unit/​agents/​runtime/​test_policy.py	Tests the updated reply and adapter-note prompt rules.
tests/​unit/​agents/​evals/​test_prep.py	Tests summary fixtures, reply measurement, blocked turns, and reply gating.
docs/​reference/​api.md	Documents the updated summary behavior and current bindings.
docs/​concepts/​agent-network.md	Describes concise replies, required notices, and current bindings.
changelog.d/​agent-replies-short.md	Records the reply-policy and prep-evaluation changes.
agents/​README.md	Documents prep evaluation cases, actions, and reply metrics.
agents/​evals/​prep.py	Adds summary actions, reply measurement, blocked-turn detection, reporting, and gating.
agents/​evals/​fixtures/​prep/​en-summary-daily-readings.json	Adds an English daily-readings summary case.
agents/​evals/​fixtures/​prep/​en-summary-ambiguous-dates.json	Adds an English ambiguous-date summary case.
agents/​evals/​fixtures/​prep/​de-summary-haustiere.json	Adds a German pet-count summary case.
agents/​evals/​fixtures/​prep/​de-summary-ausgaben-breit.json	Adds a German wide-table summary case.
agents/​anyplot/​prompts/​root.md	Defines concise replies, meaningful questions, protected notices, and same-turn handling.
agents/​anyplot/​prompts/​adapter.md	Requires concise plot-language notes without code-specific wording.
agents/​anyplot/​pipeline.py	Prevents self-review notes from repeating prior adapter changes.

_{Review effort: Lite}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!