[MarkusNeusinger]

Summary

• The reply now has the plot's texts in all 15 libraries. In the user test of 2026-11-01, replies for Chart.js, bokeh and pygal said the title and labels could not be read back, and a Makie reply offered a title the plot already had. Only 3 of 15 libraries reported drawn texts (matplotlib, seaborn, plotnine), so shows.texts was null for the other 12. The probe now reports them for:

  • pygal, altair and lets-plot through harness.install_text_probes: pygal from the SVG cairosvg.svg2png receives, altair from vl_convert's scenegraph, lets-plot from its bridge's SVG.
  • Chart.js, D3, ECharts, Highcharts, MUI X, plotly and bokeh through render.mjs writeTextProbe, in sandbox mode only, after the screenshot. It reads SVG and HTML texts by each library's classes, D3 ticks by their axis's text-anchor, ECharts' zrender display list by owning view, and the Chart.js and bokeh chart models.
  • ggplot2 through text_probe.R, R's user profile on the R row. It wraps ggsave and, after a save that returned, reads the saved plot's grob tree by layout cell, a patchwork included.
  • Makie through text_probe.jl, a second -L on the Julia row. Its atexit hook reads the current figure's blocks.

  Both of the last two leave texts-<theme>.json, which the harness's exec mode turns into the probe under the same caps (harness.collect_text_probe).

• The root never sees a null texts. fence_pipeline_notes drops the key, and root.md drops the sentence about unread texts. The rules that remain:

  • Texts are named and quoted only from texts. Without texts, the reply describes the change and names, quotes or asks about no text of the plot.
  • texts can leave a drawn text out, so the reply never says the plot lacks a title, label or legend. Only requested_not_found says a requested text is missing.

  This is structural, with no filtering of model text.

• The eval checks are code, scoped per library. The pipeline_result line carries the roles of shows.texts, never a text, plus texts_expected (RuntimeAdapter.text_probe, now True for all 15). The report counts:

  • untexted: shipped runs without drawn texts where the library reports them.
  • unbacked_quotes: passages a reply to a shipped plot quotes that no drawn or requested text holds. They are compared in process and only the count is kept.
  • probe_errors: failed probes by class.

  The matrix prints a note when either of the first two is above zero.

Best-effort probes

• No probe failure, slow page or unexpected chart model fails, changes or noticeably delays a render. Every failure drops the labels and writes {"error": <class>}, which pipeline_render logs as probe_error (a closed vocabulary, grounding.PROBE_ERRORS).
• Python and JS: a failed read, a page that does not answer within 2 s, or an SVG over 8 MB.
• R: the probe runs after the save, so its second build never reaches the PNG (a test pins the PNG byte for byte). It runs in tryCatch on a null PDF device, restored in on.exit, with its warnings muffled, and writes Error.
• Julia: the hook catches everything and prints nothing, so the row's parsed ERROR: line stays the plot's. A block that cannot be read is skipped.
• The probe file is written with O_NOFOLLOW, in Python and in Node.
• The harness converts the file only after a run that counts. It reads it with O_NOFOLLOW|O_NONBLOCK, as a regular file only, under 4 MiB, and checks every entry.
• Tests cover a model that throws, a model that never returns, a broken, oversized, linked or FIFO file, a failed save, a failed probe through the pipeline, and an R or Julia probe that fails while the render stands.

Added render time (local Docker backend, catalogue bar-grouped, light)

Library	origin/main	this branch	added	labels
pygal	0.977 s	0.990 s	+0.013 s	27
altair	1.799 s	1.825 s	+0.026 s	33
lets-plot	3.670 s	3.779 s	+0.109 s	35
plotly	1.926 s	1.940 s	+0.014 s	31
bokeh	1.801 s	1.804 s	+0.003 s	27
Chart.js	1.010 s	1.010 s	+0.000 s	20
D3	0.998 s	1.010 s	+0.012 s	33
ECharts	1.135 s	1.150 s	+0.015 s	19
Highcharts	1.000 s	1.019 s	+0.019 s	39
MUI X	1.293 s	1.298 s	+0.005 s	23
ggplot2	1.276 s	1.580 s	+0.30 s	30
Makie	4.803 s	5.340 s	+0.54 s	20

• The first ten rows are the median of five renders each through the backend.
• ggplot2 and Makie are the median of seven interleaved pairs run directly, because the machine was under load by then. ggplot2 builds its grob a second time. The Makie hook itself takes 0.2 to 0.3 s; the rest is its code compiled at exit (a precompiled hook would need a toolchain rebuild).
• All wall times include the container start.
• Separately, altair's scenegraph costs 0.03 s at 1,000 points and 0.25 s at 20,000, which is less than the PNG export itself.

Decisions (decide-and-document)

• The requested-text check stays with the reviewer for every library except matplotlib, seaborn and plotnine (review_texts, renamed TEXTLESS_LIBRARIES to REVIEWED_TEXT_LIBRARIES). The new probes ground the reply only. Canvas libraries read their texts from the chart model, which a plugin's own drawing is not in, so a code check could claim a drawn text missing and start a false repair. The Chart.js and bokeh probes set labels_truncated: true. A library can switch to the code check once its probe is proven.
• G12 (decimal mark) now needs tick_axes, which only matplotlib's probe writes. Its advice names matplotlib formatters.
• The UNVERIFIED_LINE wording no longer blames the renderer. It now reads "could not be checked against the render".
• JULIA_FLAGS stay what the toolchain image's build smoke runs. The text probe's -L belongs to the Julia row ((JULIA_BINARY, *JULIA_FLAGS, "-L", JULIA_TEXT_PROBE)), because the probe file lives in the renderer image, not the toolchain image. No toolchain rebuild.

Deploy order

• The renderer image carries everything the R and Julia rows name. agents/renderer/Dockerfile copies text_probe.R and text_probe.jl to /opt/anyplot/, and commands.py names them (R_PROFILE_USER, the row's second -L). Both ship in the same image, built by deploy-renderer, so a row never names a file its image lacks.
• The agents service imports commands.py only for the local Docker backend. That backend mounts the checkout's probe files over the image's, so a developer's older local image works too. Its remote backend runs no command.
• Either deploy order is safe. The executor already ships probe-<theme>.json for every language. With new agents and the old renderer, the root just has no texts for those libraries (no null), and untexted would show it.
• CI covers it. "Build renderer image and render through the harness" now asserts the R and Julia renders' probes (ci-image.yml), along with the new Julia argv.
• No wire, cloudbuild.yaml or deploy_smoke.sh change.

Reviews

• Gemini 3.8 Flash, before the PR opened: five findings, two applied (a bounded ECharts owner walk, a guarded probe write) and three rejected:

  • An unhandled rejection on a probe timeout. A test disproves it.
  • except A, B: as a syntax error. PEP 758 allows it on Python 3.14.
  • An ElementTree entity bomb. expat's amplification limit covers it, and there is an 8 MB cap.

• Copilot, two medium findings, both applied:

  • A role missing from texts proves nothing.
  • No DOM walk over what ECharts or bokeh draw.

• Gemini 3.8 Flash on the R and Julia part (1a64081..): four findings, three applied and one rejected:

  • Applied: a Makie Label is the title only when it is the one largest, Axis3 and PolarAxis titles count, and a hidden colorbar label is left out.
  • Rejected: reading R's grob cells from the innermost. That would make every legend's own title cell the plot's title. Patchwork's numbered cells (title-2) now match by prefix instead.

• Second opinion (Fable and Gemini 4 Argon, 9a596ac): sound with changes, nothing blocking. All eight points applied in 3b15e65:

  1. R: the probe runs after ggsave returned, not at its entry. A trace(exit=) does not work there, because ggsave's own on.exit() replaces it, so ggsave is wrapped. The gtable is not captured at ggplot_gtable's exit, because a patchwork assembles its table elsewhere. The device is restored in on.exit, warnings are muffled, and NA labels are dropped.
  2. subtitle is a role of its own.
  3. ci-image.yml rebuilds the image when only a text probe changes.
  4. An inverted Highcharts chart's axes are named by the side they are drawn on.
  5. Only the one rotated D3 text in the outer tenth is the y axis label.
  6. Makie rich text reads as drawn, and Axis3 and PolarAxis honour titlevisible and their axis labels.
  7. ECharts axis names are read by the view's anid mark (ECharts 6.1.0).
  8. The probe is written with O_NOFOLLOW.

Owner follow-up (outside this PR)

• The Cloud Build trigger deploy-renderer (europe-west4) lists agents/anyplot/render/harness.py and convert.py in includedFiles, but not agents/anyplot/render/text_probe.R or text_probe.jl.
• This PR deploys anyway, because it changes agents/renderer/**. A later change to only a probe file would not redeploy the renderer.
• Adding both paths is a GCP trigger write for the owner (agentic/docs/project-guide.md, the trigger import route).

Plan

N/A (brief from the 2026-11-01 user-test triage; owner decision: all 15 libraries in this PR)

Test plan

• uv run pytest tests/unit tests/integration -n 8: 25,867 passed. ruff, ruff format, mypy api core agents and hadolint on the renderer Dockerfile are clean.
• JS harness tests, run locally with the renderer image's node_modules and Playwright 1.64.1, all passing. They include probe tests for Chart.js, D3, Highcharts, ECharts (canvas and SVG), MUI X, plotly and nested-factor bokeh, the tamper and never-returning cases, and the catalogue page byte for byte unchanged without sandbox mode.
• tests/unit/agents/code/test_text_probe_image.py, run against the local renderer image, 6 passed. It covers ggplot2 and Makie roles, a patchwork, Makie labels and colorbar, and a failing R and Julia probe with the render standing. CI skips it without the image.
• Live through the local Docker backend, catalogue bar-grouped for all 12 libraries without a matplotlib probe. Each probe matched its PNG.
• Live eval, claude-haiku-5-5 on Vertex, $0.107 in total:
  • bar-grouped for chartjs, pygal, bokeh, ggplot2 and makie: all five shipped ok, with untexted 0, unbacked_quotes 0 and probe_errors 0.
  • The Chart.js and Makie replies quoted the drawn title and axis labels, for example „Durchschnittspunkte nach Fach und Jahrgangsstufe“.
• CI "Build renderer image and render through the harness" with the R and Julia probe assertions: green on 9a596ac.
• After the second opinion: tests/unit/agents/code/test_text_probe_image.py 9 passed, the JS harness tests 121 passed, the local gates 25,873 passed.

https://claude.ai/code/session_01H7HSkGyea1k2v293XiAMo8

[Copilot]

🟡 Changes recommended

Two moderate active findings remain: the SVG/HTML text walk double-counts ECharts and bokeh labels and uses up the 400-label cap, and plot_texts drops the labels_truncated flag, so a plugin-drawn title can be missing and wrongly reported as absent.

2 open issues

• Warn that missing text entries do not prove the plot lacks a title · New
• Restrict DOM text walking to libraries in ROLES · New

What changed in this PR

The agent's reply could not quote the texts of most plots. Only matplotlib, seaborn and plotnine reported the texts a plot draws, so the root got texts: null for the other libraries and told users it could not read the title back. This PR makes the render probes report drawn texts for pygal, altair, lets-plot, Chart.js, D3, ECharts, Highcharts, MUI X, plotly and bokeh. It also stops the root from receiving a null texts and adds a per-library count of shipped runs without texts to the eval report. ggplot2 and Makie are out of scope and follow in a separate PR.

Changes:

• Text probes for pygal, altair and lets-plot run in the Python render harness. They read the SVG or Vega scenegraph each library produces and classify texts by the library's own roles.
• Text probes for the browser-rendered libraries run in render.mjs after the screenshot, in sandbox mode only. They read SVG and HTML texts by each library's classes, and Chart.js and bokeh texts from their chart models.
• Missing drawn texts no longer reach the root as null. The eval report counts shipped runs without texts per library, and the matrix prints a note when there are any. The G12 decimal-mark gate now runs only when the probe includes tick_axes.

File	Description
tests/​unit/​automation/​test_js_render_sandbox.py	Adds tests for the browser text probe: Chart.js, D3, Highcharts, ECharts, MUI X, tampering, and a probe that never returns.
tests/​unit/​automation/​test_js_render_figure.py	Adds tests for the plotly and bokeh text probes in figure mode.
tests/​unit/​agents/​runtime/​test_runtimes.py	Checks that collect reads the text probe and ignores a probe that is not a JSON object.
tests/​unit/​agents/​runtime/​test_grounding.py	Tests the pygal, lets-plot and Vega probe readers, and that a result without texts gives the root no texts key.
tests/​unit/​agents/​runtime/​test_density_locale_gates.py	Limits the G12 decimal-mark gate to probes that include tick_axes.
tests/​unit/​agents/​runtime/​test_catalogue_flow.py	Covers a text probe that grounds the reply while the reviewer still checks the requested texts.
tests/​unit/​agents/​evals/​test_report.py	Tests the per-library count of shipped runs without drawn texts.
tests/​unit/​agents/​evals/​test_matrix.py	Checks that the recorded text roles are kept and that no text is recorded.
docs/​concepts/​agent-network.md	Documents the new probes, the root's handling of missing texts, and the reviewer split.
changelog.d/​agent-plot-texts.md	Changelog fragment for the text probes and the null fix.
automation/​js-render/​render.mjs	Adds writeTextProbe and the per-library text readers, run after the screenshot in sandbox mode.
agents/​evals/​report.py	Counts shipped runs without drawn texts (untexted) and shows them in the summary.
agents/​evals/​matrix.py	Records the text roles of each run and prints a note when runs lack texts.
agents/​anyplot/​render/​runtimes/​python.py	Reads the probe for every Python library and renames the reviewed-text set.
agents/​anyplot/​render/​runtimes/​javascript.py	Reads the browser probe in collect and updates the review-texts notes.
agents/​anyplot/​render/​png.py	Adds read_probe, which returns the probe JSON object or None.
agents/​anyplot/​render/​harness.py	Adds the pygal, altair and lets-plot probe readers and wraps their PNG exports.
agents/​anyplot/​render/​gates.py	Limits the number-format gate to probes with tick_axes.
agents/​anyplot/​render/​contract.py	Updates the review_texts docstring to match the new probe behaviour.
agents/​anyplot/​prompts/​root.md	Tells the root to describe changes without texts when none are reported, and not to call a text missing when it appears in texts.
agents/​anyplot/​plugins/​tool_safety.py	Drops texts from the fenced shows block when it is null.
agents/​anyplot/​pipeline.py	Reports drawn-text roles per run and checks requested texts against them, except for libraries the reviewer checks.
agents/​anyplot/​grounding.py	Updates the docs and the unverified-texts line for the new probes.

_{Review effort: Lite}

[codecov]

Codecov Report

❌ Patch coverage is 96.63609% with 11 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
agents/anyplot/render/harness.py	95.81%	9 Missing ⚠️
agents/evals/matrix.py	95.45%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!