Repository navigation
ci(ui): port the Test UI per-file timings reporter and its weekly dataset refresh - #29588
Conversation
…ekly dataset refresh The test-ui job's whole-suite leg now passes ui/scripts/test-file-timings.mjs beside the default and github-actions reporters, and uploads each shard's report as test-file-timings-N on the full battery's events (continue-on-error, a missing report is a warning). The new ui-test-timings-refresh.yml selects the newest eligible runs, downloads their reports, merges them into ui/scripts/test-shard-timings.json and opens the PR as the fleet App, dry-run on pull_request, mirroring shard-timings-refresh.yml. The selector's shard job name follows the root job (Test UI (N/M)), the generator's generated_by names the workflow that now exists, and the pins the objectui ci.yml carried come back against the root ci.yml and the refresh. Report-only: no threshold, no new failing check. Claude-Session: https://claude.ai/code/session_01JQwxbXf3xRiPQrLhrgsFzw Co-Authored-By: Claude <noreply@anthropic.com>
…e the refresh's no-check-families check-self-test-wired keys a `./scripts/` path in a workflow to the root scripts/ directory, so the ui-relative reporter spelling minted a key with no file behind it. The refresh workflow is path-filtered and runs no named check, so it carries the dispatch-gates no-check-families declaration. Claude-Session: https://claude.ai/code/session_01JQwxbXf3xRiPQrLhrgsFzw Co-Authored-By: Claude <noreply@anthropic.com>
dispatch-gates' self-test names every live population marker; the new ui-test-timings-refresh.yml declaration is the tenth row. Claude-Session: https://claude.ai/code/session_01JQwxbXf3xRiPQrLhrgsFzw Co-Authored-By: Claude <noreply@anthropic.com>
|
domain:devx#1 · os-warren · session_01JQwxbaBjisxkQo9hht9bFB Contract-review-claim: session_01JQwxbaBjisxkQo9hht9bFB · head 82d9d26 · 2026-11-02T20:20Z |
|
domain:devx#1 · os-warren · session_01JQwxbaBjisxkQo9hht9bFB Contract reviewServed-tier: Inputs read: card #29517 body and all four comments (triage 6583780380, claim 6583993194, surface note 6585439229, os-dev-report 6587495139), PR #29588 body, file list and net diff (5 files, 511+/40-, against merge-base dbca3ef), the check-runs on 82d9d26 (refreshed read-only: 56 completed, every completed one ① Derived judgments
② Semver level
③ Boundary flags
④ Design judgmentci.yml (
ui-test-timings-refresh.yml: holds against the precedent and the ruling.
Security-relevant observation, stated plainly: the write path is reachable on Must-fix: none. Should-fix: A (runner proof of the reporter load), B ( Implemented-by: VERDICT: PASS |
…on re-run, main-only refresh dispatch - test-file-timings.test.ts spawns the vitest CLI with the reporter by absolute path over one fixture test and asserts exit 0 and a schema-tagged report (a PR runs Test UI at the affected speed, so nothing else proves it). - The per-shard timings upload sets overwrite: true, so a re-run shard replaces its report; its comment now says retention is over-provisioned against the selector's window rather than measuring it. - ui-test-timings-refresh.yml refuses a workflow_dispatch on any ref but main before anything else runs. Claude-Session: https://claude.ai/code/session_01JQwxbXf3xRiPQrLhrgsFzw Co-Authored-By: Claude <noreply@anthropic.com>
|
domain:devx#1 · os-warren · session_01JQwxbaBjisxkQo9hht9bFB Contract-review-claim: session_01JQwxbaBjisxkQo9hht9bFB · head 5f0c784 · 2026-11-02T20:57Z |
|
domain:devx#1 · os-warren · session_01JQwxbaBjisxkQo9hht9bFB Contract reviewServed-tier: Second record on this PR; the first (6587756255, on 82d9d26) returned PASS with four should-fixes. This head is 82d9d26 plus one commit (3 files, 60+/4-) taking all four. The whole PR is judged here on 5f0c784, carrying forward what still holds; the delta was read as ① Derived judgmentsCarried forward unchanged from 6587756255, re-read on this tree, all still right:
② Semver levelUnchanged: ③ Boundary flags
④ Design judgmentci.yml (
ui-test-timings-refresh.yml: holds.
ui/scripts/tests/test-file-timings.test.ts (read as the gate of the ci.yml hunk; A's instrument):
Security-relevant observation, stated plainly: unchanged from 6587756255 and strengthened by D. The write path is reachable on Must-fix: none. Should-fix: none. The four should-fixes of 6587756255 are taken and hold against the tree; the core-lane half of D is carded separately by the seat. Implemented-by: VERDICT: PASS |
|
domain:devx#1 · os-warren · session_01JQwxbaBjisxkQo9hht9bFB 维护者速读
席位核对ACCEPT ·
|
Requested by Warren · project thread
Fixes #29517
Clause-②: no
Metadata-protocol: none
维护者速读
ci.yml的 Test UI 全量腿加挂逐文件耗时 reporter 并在全量电池事件上传每片报告;新增每周一的ui-test-timings-refresh.yml,把最新合格运行的报告合并成ui/scripts/test-shard-timings.json并以 fleet App 开 PR。continue-on-error,缺报告只是 warning,不会让分片转红;回滚即 revert 本 PR。.github/workflows/ci.yml(test-ui 的Run the ui tests与新的上传步)、.github/workflows/ui-test-timings-refresh.yml、ui/scripts/test-file-timings.mjs(shardJobName改为Test UI (N/M))。cd ui && pnpm exec vitest run scripts/__tests__/test-file-timings.test.ts,预期 15 passed。.github/workflows/**,Tier H),请你审批并合并;合并后首个全量电池跑完,在 Actions 里手动触发一次 UI Test Timings Refresh 即可拿到首个刷新 PR。What changes
.github/workflows/ci.yml, jobtest-ui:Run the ui tests (shard N/16): on the whole-suite branch (MODE=full:push,schedule,workflow_dispatch, and a pull request that failed open) it now also passes--reporter="${GITHUB_WORKSPACE}/ui/scripts/test-file-timings.mjs"besidedefaultandgithub-actions, and the step setsOBJECTUI_TEST_TIMINGS_OUTtoRUNNER_TEMP/test-file-timings/shard-N.json. The affected speed passes no reporter.Publish this shard's per-file test timings:actions/upload-artifact@v7, nametest-file-timings-N, only when the event is neitherpull_requestnormerge_group(the file's own deny-list for the full battery),continue-on-error: true,if-no-files-found: warn, retention 7 days onscheduleand 2 otherwise..github/workflows/ui-test-timings-refresh.yml(new), jobRegenerate the per-file test timings: Mondays 05:47 UTC,workflow_dispatch, andpull_requeston its own file and the three scripts it loads. It runstest-file-timings.mjs select --shards 16 --limit 6, downloads each run'stest-file-timings-*artifacts withgh run download, drops a run whose download fails or comes back with fewer than 16 reports, runsmergeintoui/scripts/test-shard-timings.json, and when the bytes changed pushesclaude/ui-test-timings-refresh-RUNand opens the PR withskip-changesetas the fleet App. Onpull_requestevery outward act is a dry run. With no eligible run (the state until a full-battery run uploads reports) it reads NOT MEASURED, warns, and stays green. It mirrorsshard-timings-refresh.yml(Test Core, chore(ci): refresh the Test Core shard-timings dataset #28978) in shape, without that lane's ceilings, coverage accumulation and bin pins, which the ui dataset does not have.ui/scripts/test-file-timings.mjs:shardJobNamenow spells the root job,Test UI (N/M)(it spelled objectui'sTest (shard N/M), so the selector could never find a shard here).generated_bynames the workflow that exists. Header and one comment updated to the root paths.ui/scripts/__tests__/test-file-timings.test.ts: the pins its retirement note promised back, now against the rootci.yml(parsed) and the refresh: the shard job name, the reporter on the whole suite and not on the affected speed, the upload name and path and its report-only shape, the refresh's--shardsagainst the matrix width, its download pattern, and that the workflowgenerated_bynames exists.scripts/pm/dispatch-gates.self-test.mjs: the live population-marker census names every declaration by file and line, and its comment asks the author of a new one to add the row. The refresh'sno-check-familiesdeclaration is the tenth row (count 9 to 10). Outside the claim's file surface, and the only file outside it.Why
ui/scripts/test-shard-timings.jsonis what the Test UI sequencer splits on. Onmain399df5bbitsmeasured.atis2026-10-19T…, itsgenerated_bynames.github/workflows/test-shard-timings-refresh.yml, which does not exist in this repository, and theci.ymlcomment aboveRun the ui testssaid the reporter and refresh were not ported. The writer, selector and generator were all present inui/scripts/test-file-timings.mjs; only the CI wiring was missing, and the selector's job name still spelled objectui's.Tests
cd ui && pnpm exec vitest run scripts/__tests__/test-file-timings.test.ts scripts/__tests__/select-affected-tests-wiring.test.ts: 2 files, 25 tests passed (the wiring test runs the edited step as written).GITHUB_WORKSPACE=$PWD OBJECTUI_TEST_TIMINGS_OUT=… pnpm --dir ui run test --shard=1/1 --reporter=default --reporter=github-actions --reporter="$PWD/ui/scripts/test-file-timings.mjs" scripts/__tests__/test-file-timings.test.tsexited 0 and wrote{"schema":"objectui.test-file-timings/1","shard":"1/1",…,"files":[{"file":"scripts/__tests__/test-file-timings.test.ts","project":"unit",…}]}.merge --runs-file … --out COPY --summary …wrote a valid dataset (the reader's owndatasetWeightscheck runs insidebuildDataset) carrying the newgenerated_by.Acceptance notes
ui/scripts/test-shard-timings.jsonstill carries the oldgenerated_bystring. It is generator output and is never hand-edited; the first refresh PR rewrites it from the corrected generator, together with a measurement date after this card. That refresh PR is what closes the card's "Done when"../scripts/test-file-timings.mjsrelative toui/(objectui's spelling):check-self-test-wiredkeys a./scripts/path in a workflow to the rootscripts/, so the relative spelling minted a key with no file behind it and redded its--self-test(measured on the first gate pass).dispatch-gates: no-check-familiesdeclaration: it is path-filtered and runs a generator, not a named check; the scripts it loads are pinned by Test UI'stest-file-timings.test.ts.pushrun that starts Test UI runs the whole ui suite (mode=full), so it is a measurement. The selector already reads both.pull_requestleg on this PR will read NOT MEASURED (no run has uploaded reports yet) and stay green.ui/scripts/dependabot-merge-gate.mjslistsRegenerate the per-file test timingsamong its optional contexts; the new job carries that name. That file is otherwise not touched.Verification
All readings at head
82d9d267(branch off399df5bb, thenorigin/mainmerged after PR #29556 landed, since that PR also editsscripts/pm/dispatch-gates.self-test.mjs).dispatch-gates: 69 derived, 69 exit 0
node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack: 69 commands, each run with its exit code captured before any pipe.pnpm --dir ui exec vitest run scripts/exited 1 on the first pass after the merge (5 cases incheck-doc-json-fences.test.ts,TypeError: fn is not a functioninside zod, a file this diff does not touch); after re-running the seven-package@objectstackbuild the ui jobs run (turbo replay, 1 s under the lock) it exited 0:Test Files 170 passed | 2 skipped (172),Tests 4976 passed | 2 skipped (4978). Read as stale build state after the merge.pnpm check:pm-dispatch-gatesexit 0:dispatch-gates self-test (fast tier): 1945 cases pass, includingthe live tree carries the ten declarations measured for this census.dispatch-gates --ran: "69 derived famil(ies) accounted for — 69 run, 0 NOT-MEASURED", exit 0.b117998e) shaped the final diff:check-self-test-wired --self-test(the./scripts/reporter spelling) and the dispatch-gates coverage guard (the refresh's missingno-check-familiesdeclaration). Both are described under Acceptance notes.outside the derived set
pnpm --dir ui run type-check:scripts(what Type Check UI runs forui/scripts): exit 0, 0 errors.pnpm --dir ui exec eslint scripts/test-file-timings.mjs scripts/__tests__/test-file-timings.test.ts: exit 0.yamlpackage the ui pins use.Human Minimum
packages/specholds no runtime logic. (not touched)pnpm lintpasses. (eslint on the two ui files: exit 0; repo-wide lint is CI's)main, one pull request.skip-changeset)packages/spectouched: n/a.content/docs/references/,packages/*/CHANGELOG.mdorcontent/docs/releases/.Generated by Claude Code