Skip to content

Commit 6817bf1

Browse files
ci: prune the build-core, typecheck and pack-smoke Turbo seeds before saving them (#27067)
Fixes #26957 Part of #25452 (step 2 of its directive, rewritten by ruling 6458731814 to "prune, not drop"; step 3, the pool reading after at least 5 merges, stays open there) Clause-②: no Metadata-protocol: none ## 维护者速读 - 改了什么:把剩下四处 per-sha Turbo 种子(Build Core、Type Check · workspace、两个 pack-smoke 作业)的保存步骤接到现成的 `prune-turbo-cache.mjs` 后面,种子只留本次运行用到的任务条目;不删种子、不加脚本、不加门禁、不抬池子上限。 - 为什么改:裁决 6458731814 选 A「修剪,不删」;#26070 → A 后这些种子是 PR 构建唯一的缓存,但整份保存会把旧种子一路累积进去,挤爆 10 GB 缓存池。 - 风险与回滚:只动 `.github/workflows/` 的四个文件;最坏情况是某次 push 不保存种子(无运行摘要即 `save=false`),下一个 PR 退回上一份种子;回滚 = revert 本 PR。 - 先看:`.github/workflows/lint.yml`(typecheck 两条 turbo 命令加 `--summarize`,否则修剪会丢掉 typecheck 条目);`.github/workflows/publish-smoke.yml`(`actions/cache` 拆成 restore + save,保存门槛与原 post-step 一致);`.github/workflows/ci.yml` Build Core 尾部。 - 两分钟验证:`node scripts/ci/prune-turbo-cache.mjs --self-test`,预期最后一行 `prune-turbo-cache self-test: OK (25 case(s) across 5 batteries).` - 需要你做的:作为 GOVERNED_APPROVERS 审批(或亲手合并)这个 Tier H draft PR。 ## What changes Ruling 6458731814, verbatim from #26957: "Letter **A, prune, do not drop**: the Build Core, Type Check · workspace and pack-smoke per-sha seeds stay, and each seed-save step is wired behind `scripts/ci/prune-turbo-cache.mjs --site …` exactly as Test Core, Build UI Deps, Type Check UI and Lint UI are today, so a seed holds only the task entries its own run used." and "⛔ The pool limit is not raised here" stands. Every save site of the three seed families, derived from the YAML at `origin/main` `cb8988ee14` (each `actions/cache*` step whose key resolves to `turbo-build-core-`, `turbo-typecheck-` or `turbo-pack-smoke-`): | Site (job) | File | Saves when (unchanged) | Turbo run had `--summarize`? | Change | |---|---|---|---|---| | Build Core (`build-core`) | `ci.yml` | `always() && push` | yes (`pnpm build --summarize`, summary left in `.turbo/runs/`) | prune step `--site 'Build Core'`; save ANDs `save == 'true'` | | Type Check · workspace (`typecheck-workspace`), the one designated typecheck writer | `lint.yml` | `always() && push` | build yes; `turbo run typecheck` NO | `--summarize` added to both `turbo run typecheck` lines; prune step `--site 'Type Check · workspace'`; save ANDs `save == 'true'` | | Packed-tarball smoke, release candidate (`pack-smoke`) | `publish-smoke.yml` | every run it runs on (`workflow_run` after Release, `workflow_dispatch`), job green, exact key missed (the `actions/cache` post-step) | NO (`pnpm run build`) | `--summarize` added; `actions/cache` split into `actions/cache/restore` + `actions/cache/save` behind the prune, same gate: `success() && cache-hit != 'true' && save == 'true'` | | Packed-tarball smoke, opt-in (`pack-smoke`) | `pack-smoke-optin.yml` | `pull_request` with `needs:pack-smoke`, job green, exact key missed | NO | same as the row above | Restore-only sites, untouched: `ci.yml` `ui-published-artifacts`; `lint.yml` `typecheck-source-gates`, `typecheck-debt`, `typecheck-consumers`; `test-nightly-tiers.yml` `tiers` (L186 is a restore, not a save, so no wiring is owed there). The pack-smoke seed is keyed `${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }}`, which is why a grep for the literal `turbo-pack-smoke` finds nothing. Wiring notes: - `--summarize` does not enter a task hash (ci.yml Build Core's ASSERTED paragraph: 83 of 83 identical with and without, turbo 2.11.5), and none of the four TURBO_* remote-cache lines is added anywhere, so the PINS in that comment still hold. - The pack-smoke save keeps `continue-on-error: true` and keeps the post-step's semantics: a failed `actions/cache` post-step never failed the job, and an explicit save step without the flag could have. - The two pack-smoke prune steps also carry `cache-hit != 'true'`, so an exact-key hit, which the post-step never saved, does not touch the restored cache either. - No `--site` list is pinned anywhere: `git grep -n "prune-turbo-cache" -- scripts packages .github` hits only the script itself and `ci.yml` (its CI-filter path and its `--self-test` in Build Core), and the script's self-test uses the fixture site name only. ## Tests - `node scripts/ci/prune-turbo-cache.mjs --self-test` (script unchanged): `prune-turbo-cache self-test: OK (25 case(s) across 5 batteries).`, exit 0. - End-to-end on a real turbo run, private `--cache-dir` so the shared cache was untouched: a planted stale entry `deadbeef` plus `pnpm exec turbo run typecheck --filter=@objectstack/sdui-parser --summarize`, then the exact prune command of the `lint.yml` site. Output: `kept 1 task entry named by 1 run summary` / `dropped 3 file(s) the run did not use`, `save=true`; what is left is the typecheck task's three files only. That shows the `--summarize` this PR adds is what keeps the typecheck entries. - `pnpm run build --summarize --dry=json` echoes `turbo run build --filter=!@objectstack/docs --filter=!./ui*/apps/site --summarize --dry=json`, so pnpm forwards the flag to turbo in the two pack-smoke jobs. - YAML parsed with PyYAML; every changed step rendered (`if:`, key, command): <details><summary>Rendered steps</summary> ``` ci.yml build-core | Prune the Turbo cache to this run's tasks (main only) | id=prune-turbo-cache | if="always() && github.event_name == 'push'" | --site 'Build Core' ci.yml build-core | Save Turbo cache (main only) | if="always() && github.event_name == 'push' && steps.prune-turbo-cache.outputs.save == 'true'" | key=${{ runner.os }}-turbo-${{ github.job }}-${{ github.ref_name }}-${{ github.sha }} lint.yml typecheck-workspace | Type check workspace packages | 'pnpm exec turbo run typecheck "${UNIVERSE[@]}" --summarize', 'pnpm exec turbo run typecheck "${FILTERS[@]}" --summarize' lint.yml typecheck-workspace | Prune the Turbo cache to this run's tasks (main only) | id=prune-turbo-cache | if="always() && github.event_name == 'push'" | --site 'Type Check · workspace' lint.yml typecheck-workspace | Save Turbo cache (main only) | if="always() && github.event_name == 'push' && steps.prune-turbo-cache.outputs.save == 'true'" | key=${{ runner.os }}-turbo-typecheck-${{ github.ref_name }}-${{ github.sha }} publish-smoke.yml pack-smoke | Setup turbo cache | uses=actions/cache/restore@v6 | id=turbo-cache | key=${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }} publish-smoke.yml pack-smoke | Build | pnpm run build --summarize publish-smoke.yml pack-smoke | Prune the Turbo cache to this run's tasks | id=prune-turbo-cache | if="success() && steps.turbo-cache.outputs.cache-hit != 'true'" | --site 'Packed-tarball smoke (release candidate)' publish-smoke.yml pack-smoke | Save turbo cache | if="success() && steps.turbo-cache.outputs.cache-hit != 'true' && steps.prune-turbo-cache.outputs.save == 'true'" | key=${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }} pack-smoke-optin.yml pack-smoke | Setup turbo cache | uses=actions/cache/restore@v6 | id=turbo-cache | key=${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }} pack-smoke-optin.yml pack-smoke | Build | pnpm run build --summarize pack-smoke-optin.yml pack-smoke | Prune the Turbo cache to this run's tasks | id=prune-turbo-cache | if="success() && steps.turbo-cache.outputs.cache-hit != 'true'" | --site 'Packed-tarball smoke (opt-in)' pack-smoke-optin.yml pack-smoke | Save turbo cache | if="success() && steps.turbo-cache.outputs.cache-hit != 'true' && steps.prune-turbo-cache.outputs.save == 'true'" | key=${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }} ``` </details> ## Acceptance notes - Post-prune seed size: **NOT MEASURED: seed size, reason: only a push run on `main` saves these seeds, so the first reading is the first push run after this lands.** Read the `prune-turbo-cache: Build Core: …` and `prune-turbo-cache: Type Check · workspace: …` lines (`kept N task entries …` / `dropped N file(s) the run did not use: X MB -> Y MB on disk.`) in that run's Build Core and Type Check · workspace logs. The pack-smoke lines (`prune-turbo-cache: Packed-tarball smoke (…): …`) appear on the next Release-triggered Publish Smoke run, or on a PR carrying `needs:pack-smoke`. The MB-scale figure on the card is an extrapolation from Test Core, not a measurement. - Pool reading (#25452 step 3) is not this PR's: it needs at least 5 merges and `actions/cache/*` read access. - Seen, not changed: both pack-smoke restores keep their bare `${{ runner.os }}-turbo-` restore-key fallback, which can restore any Turbo seed family. Wiring only; noted, not filed. ## Verification Derived gates (`node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack`), run on `56f9bbce` with each exit code captured before any pipe; reconciled on `b782ee2d` (the merge of `origin/main` `3230bdbf`, which touches no workflow; the derived list is byte-identical on both): <details><summary>dispatch-gates --ran</summary> ``` Run reconciliation — 56 derived, 52 run, 4 NOT-MEASURED, 0 UNRUN. NOT-MEASURED · DERIVED (4) — your record carries exit 3 for these - pnpm check:dts-closure [recorded exit 3 — PREREQUISITE NOT MET] - pnpm check:dual-build-cjs-loads [recorded exit 3 — PREREQUISITE NOT MET] - pnpm check:lean-entry-closure [recorded exit 3 — PREREQUISITE NOT MET] - pnpm check:sourcemap-no-sources-content [recorded exit 3 — PREREQUISITE NOT MET] ✓ dispatch-gates --ran: 56 derived famil(ies) accounted for — 52 run, 4 NOT-MEASURED (4 DERIVED from a recorded exit 3). ``` </details> The four NOT-MEASURED gates read a built `dist/`; this diff changes nothing they read. CI runs them in Build Core. `pnpm check:pm-dispatch-gates` exited 0 (`1935 cases pass` in the fast tier). Every other derived family exited 0, among them `check:self-test-wired`, `check:self-test-workflow-commands`, `check:ci-filter-parity`, `check:workflow-status-functions`, `check:workflow-step-name-quoting`, `check:step-collectors`, `check:shard-attestation` and `check:required-contexts`. Governed surface: `.github/workflows/**`, Tier H. This lands only through a GOVERNED_APPROVERS approval or the maintainer's hand. --- _Generated by [Claude Code](https://claude.ai/code/session_01QSeKYooUn7D13pPguPf727)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 5193634 commit 6817bf1

4 files changed

Lines changed: 92 additions & 8 deletions

File tree

‎.github/workflows/ci.yml‎

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2997,9 +2997,22 @@ jobs:
29972997
packages/spec/json-schema/
29982998
retention-days: 30
29992999

3000+
# Keeps only the entries the build's run summary (`--summarize`, left in
3001+
# `.turbo/runs/` by the assertion above) names, as Test Core's prune step
3002+
# does (#25452): the restore step takes the newest seed by prefix and
3003+
# turbo never evicts its local cache, so a seed saved whole is every seed
3004+
# before it plus this run. No summary -- the build never finished --
3005+
# means no save, and the previous seed stays the newest.
3006+
- name: Prune the Turbo cache to this run's tasks (main only)
3007+
id: prune-turbo-cache
3008+
if: always() && github.event_name == 'push'
3009+
run: |
3010+
node scripts/ci/prune-turbo-cache.mjs --site 'Build Core' \
3011+
--cache .turbo/cache --run-summaries .turbo/runs
3012+
30003013
# Seed the shared Turbo cache from main only (see the restore step above).
30013014
- name: Save Turbo cache (main only)
3002-
if: always() && github.event_name == 'push'
3015+
if: always() && github.event_name == 'push' && steps.prune-turbo-cache.outputs.save == 'true'
30033016
uses: actions/cache/save@v6
30043017
with:
30053018
path: .turbo/cache

‎.github/workflows/lint.yml‎

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -5684,7 +5684,7 @@ jobs:
56845684
fi
56855685
if [ -n "$WHOLE" ]; then
56865686
echo "Type-checking the whole workspace -- $WHOLE."
5687-
pnpm exec turbo run typecheck "${UNIVERSE[@]}"
5687+
pnpm exec turbo run typecheck "${UNIVERSE[@]}" --summarize
56885688
exit 0
56895689
fi
56905690
pnpm exec turbo ls "${UNIVERSE[@]}" --output=json > "$RUNNER_TEMP/typecheck-universe.json"
@@ -5700,7 +5700,7 @@ jobs:
57005700
printf ' %s\n' "${SELECTED[@]}"
57015701
FILTERS=()
57025702
for P in "${SELECTED[@]}"; do FILTERS+=("--filter=$P"); done
5703-
pnpm exec turbo run typecheck "${FILTERS[@]}"
5703+
pnpm exec turbo run typecheck "${FILTERS[@]}" --summarize
57045704
57055705
# Seed the shared Turbo cache from main only (see the restore step above).
57065706
#
@@ -5721,8 +5721,21 @@ jobs:
57215721
# merge_group is deliberately NOT added to this condition — see the
57225722
# "why queue-branch saves are not the win they look like" note on the
57235723
# `typecheck` aggregator at the bottom of this file.
5724-
- name: Save Turbo cache (main only)
5724+
#
5725+
# The seed keeps only the entries this lane's two turbo runs used, as
5726+
# ci.yml Test Core's prune step does (#25452): the build's and the
5727+
# typecheck's `--summarize` summaries, both left in `.turbo/runs/`. A
5728+
# bookkeeping step, not a gate: no `continue-on-error`, and it exits 0 on
5729+
# every run-time path.
5730+
- name: Prune the Turbo cache to this run's tasks (main only)
5731+
id: prune-turbo-cache
57255732
if: always() && github.event_name == 'push'
5733+
run: |
5734+
node scripts/ci/prune-turbo-cache.mjs --site 'Type Check · workspace' \
5735+
--cache .turbo/cache --run-summaries .turbo/runs
5736+
5737+
- name: Save Turbo cache (main only)
5738+
if: always() && github.event_name == 'push' && steps.prune-turbo-cache.outputs.save == 'true'
57265739
uses: actions/cache/save@v6
57275740
with:
57285741
path: .turbo/cache

‎.github/workflows/pack-smoke-optin.yml‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,8 +121,11 @@ jobs:
121121
- name: Setup workspace
122122
uses: ./.github/actions/setup-workspace
123123

124+
# Restore here, save at the end of the job: the save sits behind the
125+
# prune step below, which a combined `actions/cache` post-step cannot.
124126
- name: Setup turbo cache
125-
uses: actions/cache@v6
127+
id: turbo-cache
128+
uses: actions/cache/restore@v6
126129
with:
127130
path: .turbo/cache
128131
key: ${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }}
@@ -132,12 +135,38 @@ jobs:
132135
133136
# The driver's own prerequisite, asserted by the script itself: pack mode
134137
# fails fast unless packages/cli/dist and the create-objectstack bin exist.
138+
# `--summarize` writes the run summary the prune step below keeps.
135139
- name: Build
136-
run: pnpm run build
140+
run: pnpm run build --summarize
137141

138142
# SMOKE_MODE defaults to `pack`, so this is the same invocation
139143
# publish-smoke.yml's pack-smoke job makes. ⛔ Do not add flags or env here
140144
# to make a red go away: the assertions are #14000's deliverable and a
141145
# refusal this script reports is a refusal a user would get.
142146
- name: Publish smoke (packed tarballs)
143147
run: bash scripts/publish-smoke.sh
148+
149+
# Keeps only the entries the build's run summary (`.turbo/runs/`) names,
150+
# as ci.yml Test Core's prune step does (#25452): the restore step takes
151+
# the newest seed by prefix and turbo never evicts its local cache, so a
152+
# seed saved whole is every seed before it plus this run. The save keeps
153+
# the gate the combined `actions/cache` post-step had: a green job whose
154+
# exact key missed, and a save failure never fails the job.
155+
- name: Prune the Turbo cache to this run's tasks
156+
id: prune-turbo-cache
157+
if: success() && steps.turbo-cache.outputs.cache-hit != 'true'
158+
run: |
159+
node scripts/ci/prune-turbo-cache.mjs --site 'Packed-tarball smoke (opt-in)' \
160+
--cache .turbo/cache --run-summaries .turbo/runs
161+
162+
- name: Save turbo cache
163+
if: >-
164+
success()
165+
&& steps.turbo-cache.outputs.cache-hit != 'true'
166+
&& steps.prune-turbo-cache.outputs.save == 'true'
167+
uses: actions/cache/save@v6
168+
timeout-minutes: 5
169+
continue-on-error: true
170+
with:
171+
path: .turbo/cache
172+
key: ${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }}

‎.github/workflows/publish-smoke.yml‎

Lines changed: 31 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -201,17 +201,21 @@ jobs:
201201
- name: Setup workspace
202202
uses: ./.github/actions/setup-workspace
203203

204+
# Restore here, save at the end of the job: the save sits behind the
205+
# prune step below, which a combined `actions/cache` post-step cannot.
204206
- name: Setup turbo cache
205-
uses: actions/cache@v6
207+
id: turbo-cache
208+
uses: actions/cache/restore@v6
206209
with:
207210
path: .turbo/cache
208211
key: ${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }}
209212
restore-keys: |
210213
${{ runner.os }}-turbo-${{ github.job }}-
211214
${{ runner.os }}-turbo-
212215
216+
# `--summarize` writes the run summary the prune step below keeps.
213217
- name: Build
214-
run: pnpm run build
218+
run: pnpm run build --summarize
215219

216220
- name: Publish smoke (packed tarballs)
217221
run: bash scripts/publish-smoke.sh
@@ -237,6 +241,31 @@ jobs:
237241
-f description="$desc" \
238242
-f target_url="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
239243
244+
# Keeps only the entries the build's run summary (`.turbo/runs/`) names,
245+
# as ci.yml Test Core's prune step does (#25452): the restore step takes
246+
# the newest seed by prefix and turbo never evicts its local cache, so a
247+
# seed saved whole is every seed before it plus this run. The save keeps
248+
# the gate the combined `actions/cache` post-step had: a green job whose
249+
# exact key missed, and a save failure never fails the job.
250+
- name: Prune the Turbo cache to this run's tasks
251+
id: prune-turbo-cache
252+
if: success() && steps.turbo-cache.outputs.cache-hit != 'true'
253+
run: |
254+
node scripts/ci/prune-turbo-cache.mjs --site 'Packed-tarball smoke (release candidate)' \
255+
--cache .turbo/cache --run-summaries .turbo/runs
256+
257+
- name: Save turbo cache
258+
if: >-
259+
success()
260+
&& steps.turbo-cache.outputs.cache-hit != 'true'
261+
&& steps.prune-turbo-cache.outputs.save == 'true'
262+
uses: actions/cache/save@v6
263+
timeout-minutes: 5
264+
continue-on-error: true
265+
with:
266+
path: .turbo/cache
267+
key: ${{ runner.os }}-turbo-${{ github.job }}-${{ github.sha }}
268+
240269
registry-canary:
241270
name: Registry canary (published latest)
242271
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'

0 commit comments

Comments
 (0)