Repository navigation
feat(agent): per-day numbers on /debug, a service pause refusal - #12518
Merged
Merged
Conversation
…ice pause refusal Work in progress taken over from a killed session: the per-day table, the agents service's live budget view and the `service_paused` refusal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
…refusal Takes the pause note's clock to UTC plus the reader's own time on /debug, keeps the limit hits on one line on a phone, counts no chat for a line without a session, drops an unused `reserve`, and documents all of it: the /debug section, the API reference, the Plausible reasons and the concept doc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Active moderate and critical findings remain in the token-limit bookkeeping, service-paused messaging, and changelog fragment, so the changes require correction before approval.
3 open issues
What changed in this PR
This pull request adds per-day agent usage visibility to /debug and distinguishes service-wide budget exhaustion from individual user limits across the agents service, API, telemetry, documentation, and frontend.
Changes:
- Adds a 14-day UTC usage table with chats, plots, costs, limit hits, and queue-wait percentiles.
- Adds live service-budget and per-user-limit counts through the debug and budget endpoints.
- Introduces the
service_pausedrefusal, status reporting, localized reset-time messaging, telemetry support, documentation, and tests.
| File | Description |
|---|---|
tests/unit/api/test_agent_usage.py |
Tests daily usage aggregation, caching, SQL construction, and authorization. |
tests/unit/api/test_agent_router.py |
Tests the live budget proxy and service_paused mapping. |
tests/unit/agents/runtime/test_policy.py |
Tests localized service-pause refusal text. |
tests/unit/agents/runtime/test_day_budget.py |
Tests budgets, pricing, refusals, telemetry, status, and budget responses. |
docs/reference/plausible.md |
Documents the new analytics refusal reason. |
docs/reference/api.md |
Documents status, budget, daily usage, and pause-related API behavior. |
docs/reference/agent-telemetry.md |
Documents daily telemetry columns, sources, costs, and live budget semantics. |
docs/concepts/agent-network.md |
Documents the service-wide budget and related routes. |
changelog.d/debug-daily-usage.md |
Adds the changelog fragment for the feature. |
app/src/sections/debug/format.ts |
Adds currency formatting for debug usage values. |
app/src/sections/debug/AgentUsageSection.tsx |
Integrates live and per-day usage views into the debug section. |
app/src/sections/debug/AgentDailyUsage.tsx |
Implements live budget and daily usage table rendering. |
app/src/sections/agent-chat/messages.ts |
Adds reset-time and service-pause message helpers. |
app/src/sections/agent-chat/messages.test.ts |
Tests pause and reset-time message formatting. |
app/src/sections/agent-chat/ChatThread.tsx |
Displays local reset-time information for paused refusals. |
app/src/sections/agent-chat/ChatThread.test.tsx |
Tests service-pause rendering in chat threads. |
app/src/pages/DebugPage.test.tsx |
Tests daily rows, live budget output, and loading states. |
app/src/pages/AgentChatPage.tsx |
Displays the service-pause notice before message submission. |
app/src/pages/AgentChatPage.test.tsx |
Tests status-driven pause notices and reset handling. |
app/src/lib/agent.ts |
Extends the agent status response type. |
app/src/hooks/useAgentSession.ts |
Tracks service-pause refusals and reset metadata. |
app/src/hooks/useAgentSession.test.ts |
Tests service-pause refusal state and telemetry behavior. |
api/routers/debug.py |
Adds the daily usage debug endpoint. |
api/routers/agent.py |
Adds the admin-gated budget proxy and pause error mapping. |
api/agent_usage.py |
Builds and reads the BigQuery daily usage query and response models. |
agents/stream.py |
Documents the expanded SSE refusal contract. |
agents/main.py |
Adds status and budget routes and applies service-pause handling. |
agents/anyplot/telemetry.py |
Adds service_paused to telemetry codes and mappings. |
agents/anyplot/prompts/refusals.yaml |
Adds localized service-pause refusal messages. |
agents/anyplot/plugins/scope_guard.py |
Applies service-aware daily refusal classification. |
agents/anyplot/plugins/ledger.py |
Adds global budget tracking, snapshots, pricing, and refusal classification. |
agents/anyplot/plugins/budget.py |
Propagates service-pause decisions during model-budget checks. |
Review effort: Lite
`/status` reports the pause for everyone, so the page cannot know that a user still has their own budget; one who does not reads a false claim and then gets `budget`. The server's refusal keeps the sentence, because `day_refusal` sends it only to users with their own budget left. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
Carries the caller's tier into the pause: `day_refusal` takes it, so an admin, who has no own day budget, hears `service_paused` from the route, the scope guard, the Budget plugin and the paste routes. The usage book learns who came in the admin tier from `_stamp`, which every spending route passes, and the live view counts an admin only at the caps that bind every tier. The per-day table's queue wait reads `pipeline_run.queue_wait_ms` from the line itself, so it fills from #12507's deploy on, before the `pipeline_runs` view is applied again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
/debuggets a table of the last 14 UTC days (new chats, plots, cost, fan-out cost, limit hits by code, p95 queue wait), read read-only from the BigQuery telemetry./debuggets a live line for today from the agents service: the service's spend against its day budget and how many users are at their own limits, counts only, for the instance's lifetime.service_paused, apart from a user's own limit. It names the reset at 00:00 UTC, the page adds the user's own time, andGET /v1/statustells the chat before the user types.Plan
N/A: owner request of 2026-11-02 ("What happens when the $10 daily limit is reached? We definitely need a few more numbers in debug: dollars per day, how many new chats."). This takes over the work of a killed session.
Decisions
budgetcounts too. Limit hits countbudgetas well as the codes the brief named, because a user's own spent token budget is a limit hit too.locked_outis a safety lockout, so it stays in the card's refusal list.livelines fill every column,fanoutlines fill only a separatefan-outcost column (they spend the same day budget), andharnesslines are never read.budget. The pause text says their own limit is untouched, which would be false for them.eu's 10 %. The live line prices weighted tokens at the main model's input list price in the service's location, so the default of 100 M tokens reads$11.00, not$10.00./debugshows both.plot.reason = budget. That is theplotevent's closed failure reason, not a refusal, and it reads "the usage limit is reached", which is true of either budget. The root's next call then sendsservice_pausedthroughday_refusal.Flash pre-check
daily_sqlprices and deduplicates as05_prices_cost.sqldoes.budget, which the code does not do (see above). The double-count finding refutes itself in its own text. The client-side reset time is always the next 00:00 UTC, so passing the server's value would change nothing for a correct clock. It found no gap in the consumers of the new code, no exposure beyond counts and times, and no bug in the pause note.Copilot review
Three findings, one applied:
/statuscannot know. The note now says only that the chat is paused and until when. The server's refusal keeps the sentence, becauseday_refusalsends it only to users with their own budget left./pull_requeststep 7 appends them and the changelog check passes.snapshotandday_refusal; see the next section.Merged with #12507
day_refusaltakes the caller's tier. An admin has no own day budget, so the route, the scope guard, the Budget plugin and the paste routes all answer an adminservice_pausedduring a pause, neverbudget._stamp, which every spending route passes (UsageBook.mark_admin).pipeline_run.queue_wait_msfrom the line itself, not from thepipeline_runsview, so it fills from feat(agent): admin lane, final run limits and anonymous limit telemetry #12507's deploy on, whether or not the view is applied again, and reads—on every earlier day. A test pins the field name to the telemetry model.Owner steps
None. No view changes, no setting changes, no writes to BigQuery.
Verification
costview for every day of the window (10 day and source rows, to the rounding).read_dailyover the real REST path decodes 14 rows.Test plan
429.costview in BigQuery./debugand the chat's pause note in the browser, both viewports and both themes (the note's shorter final wording is covered by unit tests)./debugand check that the table and the live line load.https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8