Problem
/overview shows the latest run per model×hardware (30 combos), but there is no scheduled benchmark run — data only lands when a PR touches a recipe. Cells go stale unevenly, and cross-hardware numbers come from different dates/batches (parent: #2304).
Plan
Weekly cron sweep, 1 best config per combo, same batch → cross-hardware comparable:
- New
schedule: workflow reusing e2e-tests.yml + ingest-results dispatch (app side needs zero changes — changelog-metadata falls back to the run name)
- Lowest effective priority:
schedule events get no event bump in ci-priority.yaml, so PR/main sweeps preempt
- Saturday 06:00 UTC, off-peak
- Partial-failure runs still ingest (app skips failed rows)
Cost: pilot ~18 node-hours/week; full 30-combo scope ~8–40 node-hours/week ≈ 0.3–2% of current CI load (~2,000–2,500 job-hours/wk measured).
Rollout
Accepted trade-off
/overview is latest-run-wins: a scheduled run replaces the displayed number. Accepted for the pilot (option A); revisit source tagging if a bad scheduled run ever regresses a cell.
问题
/overview 每个 model×hardware 格子只显示最新一次 run(共 30 个 combo),但目前没有任何定时 benchmark——数据只在 PR 动到 recipe 时产生。格子新鲜度参差,且跨硬件数字来自不同日期/批次(父 issue:#2304)。
方案
每周 cron sweep,每个 combo 跑 1 个 best config,同批次 → 跨硬件可比:
- 新增
schedule: workflow,复用 e2e-tests.yml + ingest-results dispatch(app 侧零改动——changelog-metadata 缺失时回退到 run name)
- 实际最低优先级:
schedule 事件在 ci-priority.yaml 中无 event 加分,PR/main sweep 天然抢占
- 周六 06:00 UTC,避开高峰
- 部分失败的 run 仍然 ingest(app 跳过失败行)
成本: pilot 约 18 node-hours/周;全量 30-combo 约 8–40 node-hours/周 ≈ 当前 CI 负载(实测 ~2,000–2,500 job-hours/周)的 0.3–2%。
推进
已接受的取舍
/overview 是 latest-run-wins:定时 run 会替换展示数字。pilot 阶段接受(方案 A);若定时 run 拉低格子数字再考虑 source tag。
Problem
/overviewshows the latest run per model×hardware (30 combos), but there is no scheduled benchmark run — data only lands when a PR touches a recipe. Cells go stale unevenly, and cross-hardware numbers come from different dates/batches (parent: #2304).Plan
Weekly cron sweep, 1 best config per combo, same batch → cross-hardware comparable:
schedule:workflow reusinge2e-tests.yml+ingest-resultsdispatch (app side needs zero changes —changelog-metadatafalls back to the run name)scheduleevents get no event bump inci-priority.yaml, so PR/main sweeps preemptCost: pilot ~18 node-hours/week; full 30-combo scope ~8–40 node-hours/week ≈ 0.3–2% of current CI load (~2,000–2,500 job-hours/wk measured).
Rollout
Accepted trade-off
/overviewis latest-run-wins: a scheduled run replaces the displayed number. Accepted for the pilot (option A); revisit source tagging if a bad scheduled run ever regresses a cell.问题
/overview每个 model×hardware 格子只显示最新一次 run(共 30 个 combo),但目前没有任何定时 benchmark——数据只在 PR 动到 recipe 时产生。格子新鲜度参差,且跨硬件数字来自不同日期/批次(父 issue:#2304)。方案
每周 cron sweep,每个 combo 跑 1 个 best config,同批次 → 跨硬件可比:
schedule:workflow,复用e2e-tests.yml+ingest-resultsdispatch(app 侧零改动——changelog-metadata缺失时回退到 run name)schedule事件在ci-priority.yaml中无 event 加分,PR/main sweep 天然抢占成本: pilot 约 18 node-hours/周;全量 30-combo 约 8–40 node-hours/周 ≈ 当前 CI 负载(实测 ~2,000–2,500 job-hours/周)的 0.3–2%。
推进
已接受的取舍
/overview是 latest-run-wins:定时 run 会替换展示数字。pilot 阶段接受(方案 A);若定时 run 拉低格子数字再考虑 source tag。