diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b25484625e..1e0c508f54d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,6 +34,1479 @@ aggregate instead: an italic *Catalog* line at the end of the version section an ## [Unreleased] +## [3.6.0] — 2026-11-01 — All fifteen + +### Added + +- **The prep eval measures the root's replies.** Every reply's words, questions and closing question are counted; + a closing question the turn was not blocked for, or a reply over 100 words, is a reply miss, gated at 10 % + (`--max-reply-misses`). A case can send the page's summary action, and four step-1 summary cases join the set. + (#12506) +- **`--theme dark` for the eval matrix.** Every turn of the run sends the + site theme, as the chat page does, so dark first renders can be measured + before the public release. Every committed baseline is light. (#12505) +- **plotly and bokeh in "Use with my data", for the admin's live tests.** The + Python runtime serves both through the renderer's two-step figure mode: the + plot writes its figure as JSON under the plot guard, and the JavaScript + harness draws it in Chromium at the catalogue's canvas. The admin deploy names + them, so all 15 catalogue libraries are served, and each one's 80 % eval + matrix gates the public release. Across the catalogue 273 of 325 plotly and + 252 of 325 bokeh files are eligible; in the renderer image 254 of the 273 and + 245 of the 252 render on target in both themes as they stand, the rest only + because their code divides by the length of the emptied catalogue title. + (#12490) +- **The validator rules for plotly and bokeh.** Both refuse private names, + anything but `None` where the library expects a URL, and every model that + loads from the network or a file: plotly's geo and map traces and its + `plotly_relayout` family, bokeh's JavaScript, markup, CSS and tile models and + its import by name. bokeh's tooltip templates pass as the catalogue writes + them, text tags with a quoted `style` or `class`, never `{safe}`, because + BokehJS parses a template and each tooltip value as HTML; plotly text holds + no link. (#12490) +- **The renderer checks a plotly or bokeh figure for markup before it draws + it.** A figure that holds what a browser would run, load or link (a bokeh + JavaScript, HTML or CSS model or any model no catalogue plot uses, a tooltip + outside the template rule, a link in plotly text, an image URL) fails as + `figure UnsafeMarkup`, whatever code built it, so the downloaded code never + writes it into the user's HTML. The renderer image's CI asserts the refusal + through the image's own harness. (#12490) +- **The figure save and the exported code.** The normaliser turns the + catalogue's save block into one protected statement that writes the figure as + JSON (plotly's size moved into the figure; bokeh's Selenium screenshot block + removed). The downloaded `plot.py` swaps it back for the library's normal + save, so it runs unchanged on the user's machine with `data.csv`: plotly's + `write_image` and `write_html`, bokeh's `export_png`, `output_file` and + `save`. This is the one place where the export is not the run form byte for + byte (owner decision). (#12490) +- **plotly and bokeh adapter sections.** Measured in the renderer image: the + protected save, the theme tokens on the figure, timestamps converted to UTC + (plotly draws an offset timestamp's clock time as given), plotly's + `separators` for the plot language's decimal and thousands marks, bokeh's + ticks on fixed values with labels written in Python (BokehJS writes English + marks whatever its `language`), `render_mode="svg"` and no WebGL. An image + test renders each idiom from a table with missing cells in both themes. + (#12490) +- **plotly and bokeh eval fixtures.** The ten matrix specs for each (120 cases, + 900 generated in all) and the smoke cases `scatter-basic-plotly-renamed` and + `scatter-basic-bokeh-renamed`; all twenty files are eligible. (#12490) +- **A plotly figure render gates the renderer's deploy.** The candidate smoke + renders a plotly figure with an axis range through both steps of the figure + mode under gVisor and refuses the promotion when it fails twice. (#12490) +- **The pipeline's Claude steps can run on API credits, by workload identity + federation or an API key: the repository variable `CLAUDE_AUTH`.** Unset or + `subscription`, all 13 Claude steps in the 8 workflows that run Claude use + the `CLAUDE_CODE_OAUTH_TOKEN` secret as before. `oidc` exchanges the job's + GitHub OIDC token for a short-lived token of the `ci-regen` workspace (the + four `CLAUDE_WIF_*` variables; the Console rule accepts only runs on + `main`), and `api` uses the `ANTHROPIC_API_KEY` secret. A new first step, + "Resolve Claude credentials", stops the job on any other value or a missing + credential instead of falling back to the subscription and the owner's + interactive quota, and every Claude step reads its outputs, so each passes + exactly one credential and a new mode changes one step. Under `oidc`, the + runs federation refuses (pull-request review events, other refs, + `review-retest.yml`) use the subscription and say so in the job summary. + `docs/workflows/overview.md` has the Console setup, the switch commands, + and how to close the federation path when `oidc` is off. (#12487) +- **Every Claude session's cost in the job summary, and a clean stop when + Claude refuses a session.** A step after each Claude step writes + `total_cost_usd` (Claude Code's estimate at API list price), the model and + the credential mode to the job summary and the log, and never fails the + job. It also reads, from the error fields of a failed session, the API's + spend-limit, spend-cap and credit messages, the subscription's usage-limit + messages, and, with `api` or `oidc`, refused credentials: the retry is + skipped, `impl-generate.yml` parks the pair, `impl-review.yml` and + `impl-repair.yml` pause the PR with the new labels `ai-review-paused` and + `ai-repair-paused` without spending their automatic retry or recording a + score, the watchdog skips paused PRs, and nothing counts as an attempt or + marks `impl::failed`, a first implementation included. Before, a + subscription usage limit used up a pair's three attempts. The weekly regen + review's `collect_week.py --costs` reads the line, now counts the + `daily-regen.yml` pre-flight too, and names the part that ran on API + credits. (#12487) +- **Daily regeneration by implementation instead of by spec: the repository + variable `REGEN_PAIRS_PER_DAY`.** Set to N, the scheduled `daily-regen.yml` + run picks the N oldest (spec, library) pairs across the catalogue + (`automation/scripts/regen_pick_pairs.py`), skips the spec polish, and + dispatches each pair straight to `impl-generate.yml` with + `bulk-generate.yml`'s arguments and pace, since that workflow's single + concurrency group would cancel queued runs. A pair merged or really + attempted (a closed PR, a kept regeneration, a failed generation) within + the attempt window, 30 days unless `attempt_window_days` or + `REGEN_ATTEMPT_WINDOW_DAYS` says otherwise, is not picked again, while an + infrastructure failure or a generation Claude stopped leaves it in the + queue. Paused reviews and repairs are resumed first and count against N, + and the pick stops when it can't list the implementation PRs. Unset, the + schedule regenerates whole specs as before; a manual run opts in with the + new `pairs` input, and the watchdog's cron rescue forwards the variable. (#12487) +- **The renderer draws plotly and bokeh figures in two steps, not served yet.** + The render harness forks the plot before it arms anything: the plot runs + under the plot guard and the exec filter with the Python limits and may write + only its figure as JSON, and the harness, never armed, then hands the figure + to the JavaScript row's browser harness, which draws it as data with the + installed package's own plotly.js or BokehJS in the sandboxed page, with a + stricter policy that runs no script the figure carries. The figure keeps the + size its code gave it, and the scale makes the image exactly 3200x1800 or + 2400x2400, so the catalogue's layouts look as published. Failures in either + step end with the same final error line as a JavaScript render. The renderer + image now carries plotly and bokeh without their kaleido and Selenium export + stacks. The agent chat does not offer plotly or bokeh yet. (#12484) +- **MUI X in "Use with my data", for the admin's live tests.** The JavaScript + runtime serves MUI X (`muix`) next to Chart.js, D3, ECharts and Highcharts: + `plot.tsx`, a React module the renderer's harness bundles through its closed + import plugin and mounts as a component, with the user's table as typed JSON. + The admin deploy names it, so 13 of the 15 catalogue libraries are served, and + its 80 % eval matrix gates the public release. A catalogue file written for + MUI X 7 is served `coupled` with its `deprecated-api` note, never blocked for + its version (owner decision): the adapter rewrites `ChartContainer` to MUI X + 9.16's `ChartsContainer`. The interactive `plot.html` and the in-chat preview + work for MUI X's React bundle as for the other four. (#12483) +- **MUI X idioms in the JavaScript adapter prompt.** `adapter_javascript.md` + teaches what was measured with MUI X 9.16 in the renderer image: the module's + imports and its one default export, MUI X 9's names for MUI X 7's, axis bands + sized for the user's labels (MUI X cuts tick labels that do not fit), overlays + drawn through the chart's own scales instead of pixel positions from `margin`, + a `"utc"` time axis with a formatter (a `"time"` axis follows the browser's + zone in the exported page), `fmt` on every value axis (MUI X writes + `1,250,000`), and a series formatter that guards `null`, which MUI X hands it + when a reader hovers a gap; unguarded, the hover removes the whole chart. A + test runs the idioms through both validator profiles, and an image test + renders them from a table with missing cells in both themes. (#12483) +- **MUI X eval fixtures.** The ten matrix specs for MUI X (60 cases, 780 + generated in all) and the smoke case `pie-basic-muix-renamed`; five of the + ten files are `clean` (written for MUI X 9.16), five `coupled`, one of them + with the `deprecated-api` note. Across the catalogue 311 of 323 MUI X files + are eligible. (#12483) +- **A Chart.js, D3, ECharts or Highcharts plot plays in the result card.** + `.interactive()` on the image shows the plot's `plot.html` in the card, in + the theme on screen, with its tooltips and hover; `.preview()` brings the + image back, and a theme switch shows that theme's page. Nothing loads before + the click: the chat then fetches the very page the download serves, through + the same authenticated route, and holds the session while the server builds + it. The page reaches the card through a relay on the API, so no token or + cookie ever meets the plot code, and the site's content security policy is + unchanged. The frame takes the plot's orientation from the page itself, and + collapsing the card drops the preview's pages. Opening the preview is its + own usage outcome, `opened_preview`, a look that counts as no keep. (#12482) +- **The relay page of the chat's interactive preview, `GET /preview/relay`.** + A static, unauthenticated page of 1 KB on the API, outside `/debug/*`, + served with `Content-Security-Policy: sandbox allow-scripts`, the browser + harness's policy and `frame-ancestors` for the site's origins and local dev + servers, so it runs in an opaque origin without network that no other site + can frame. Its only message is a constant ready signal without content to + its parent; the chat posts the page only on that signal from the relay's own + frame, so a page that stands in at the relay's URL, such as an error page of + the edge, never receives it, and the card says `relay_unavailable` when no + signal comes. The relay takes one page, the first that its parent posts from + an allowed origin, writes it into an `