Skip to content

chore(ci): cut PR wall-clock ~9.5min → ~5-6min — dedupe dogfood, shard it, cache lint's build - #3622

Merged
os-zhuang merged 2 commits into
mainfrom
claude/ci-performance-optimization-ms8ll6
Jul 27, 2026
Merged

chore(ci): cut PR wall-clock ~9.5min → ~5-6min — dedupe dogfood, shard it, cache lint's build#3622
os-zhuang merged 2 commits into
mainfrom
claude/ci-performance-optimization-ms8ll6

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Why

Job-level timing of recent PR runs shows where the time actually goes:

Job Wall time Long pole
CI / Test Core 9m21s turbo run test --affected — 8m35s
CI / Dogfood Regression Gate 8m41s dogfood suite — 7m34s
Lint / TypeScript Type Check 6m09s turbo run build — 4m38s (no cache)
CodeQL ~4m analysis (parallel, acceptable)

Two structural problems dominate: the dogfood suite ran twice per core PR (once in its dedicated job, once again inside Test Core's affected set, in parallel on two runners), and lint's typecheck job is the only build-running job with no turbo cache, so it rebuilt the entire workspace from scratch on every run — the identical build takes 51s in ci.yml's cached Build Core job.

What

  1. Exclude @objectstack/dogfood from Test Core (both the PR --affected run and the push run). The dedicated Dogfood job — gated on the same core filter — remains the sole, unconditional runner of the suite. Verified locally: turbo unions inclusive filters and then subtracts ! negations, so --affected --filter=!@objectstack/dogfood is exactly "affected minus dogfood" (140 → 137 tasks, nothing else dropped).

  2. Shard the Dogfood job 2-way (vitest run --shard=N/2, ~60 independent test files, deterministic file-level partition). The shard arg passes through turbo and is part of the turbo task hash, so each shard caches independently (verified: distinct hashes). The objectstack verify CLI step is shard-independent and runs on shard 1 only. Turbo cache keys are shard-scoped so the two matrix jobs don't race on one cache entry.

  3. Add the missing turbo cache step to lint's typecheck job — same key scheme as ci.yml, so the fallback prefix also hits caches saved by main pushes.

  4. Add concurrency cancel-in-progress to lint.yml and codeql.yml (ci.yml already has it). Both trigger on every PR sync; superseded runs were burning runners and delaying the queue.

Expected effect

  • CI workflow: ~9.5min → ~5–6min (Test Core sheds the duplicated 7.5-min suite; Dogfood halves to ~4–5min per shard).
  • Lint workflow: ~6min → ~2–2.5min on warm cache.
  • Meaningful runner-minute savings per PR sync from de-duplication + cancellation.

Verification

  • turbo run test --affected --filter=!@objectstack/dogfood --dry=json with a spec change: dogfood excluded, all other affected tasks intact.
  • vitest run --shard=1/30 in packages/qa/dogfood: 3 files / 13 tests, green — sharding partitions and runs correctly.
  • Full-chain smoke of the exact CI command shape: turbo run test --filter=@objectstack/dogfood -- --shard=1/30 → 61 tasks successful, exit 0.
  • --dry=json task-hash comparison between --shard=1/2 and --shard=2/2: hashes differ, so per-shard turbo caching is safe.
  • All three YAML files parse.

⚠️ Action needed at merge time

If branch protection lists "Dogfood Regression Gate" as a required status check, it must be updated to the two sharded check names (Dogfood Regression Gate (1/2) / Dogfood Regression Gate (2/2)).

🤖 Generated with Claude Code

https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt

Generated by Claude Code

@claude
chore(ci): cut PR wall-clock ~9.5min → ~5-6min — dedupe dogfood, shar…
61585a3 
…d it, cache lint's build

Three data-driven fixes from job-level timing of recent PR runs:

1. Test Core re-ran the whole ~7.5-minute dogfood suite that the dedicated
   Dogfood job was already running in parallel (both gate on the same `core`
   filter). Exclude @objectstack/dogfood from both the PR --affected run and
   the push run; the Dogfood job remains the sole (and unconditional) runner
   of the suite. Verified locally: turbo unions inclusive filters and then
   subtracts `!` negations, so `--affected --filter=!@objectstack/dogfood`
   is exactly "affected minus dogfood" (140 → 137 tasks, nothing else lost).

2. The Dogfood job itself was the workflow's longest pole (7m34s for the
   ~60-file suite on one 4-vCPU runner). Shard it 2-way with vitest's
   deterministic file-level --shard, passed through turbo (pass-through args
   are part of the turbo task hash, so each shard caches independently —
   verified: distinct hashes per shard). The `objectstack verify` CLI step is
   shard-independent and runs on shard 1 only. NOTE: if branch protection
   requires "Dogfood Regression Gate", the required check must be renamed to
   the two sharded names.

3. lint.yml's "TypeScript Type Check" job runs a full workspace build but was
   the only build-running job with NO turbo cache step — it rebuilt everything
   from scratch every run (4m38s observed, vs 51s for the same build in
   ci.yml's cached Build Core). Add the same .turbo/cache step.

Also: lint.yml and codeql.yml had no concurrency group, so superseded runs of
both (they trigger on every PR sync) kept burning runners and delaying the
queue. Add the same cancel-in-progress policy ci.yml already documents.

Full-chain smoke: `turbo run test --filter=@objectstack/dogfood -- --shard=1/30`
runs green end-to-end (turbo → pnpm → vitest, 61 tasks OK).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt
@vercel

vercel Bot commented Jul 27, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Jul 27, 2026 10:05am

Request Review

@github-actions github-actions Bot added documentation Improvements or additions to documentation tooling labels Jul 27, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review July 27, 2026 10:31
@os-zhuang
os-zhuang merged commit 1676de0 into main Jul 27, 2026
16 checks passed
@os-zhuang
os-zhuang deleted the claude/ci-performance-optimization-ms8ll6 branch July 27, 2026 10:32
@os-zhuang os-zhuang mentioned this pull request Jul 27, 2026
Merged
os-zhuang added a commit that referenced this pull request Jul 27, 2026
@os-zhuang @claude
fix(ci): restore the stable "Dogfood Regression Gate" required-check …
e3eb93f 
…name (#3622 follow-up) (#3644)

INCIDENT: every open PR in the repo is BLOCKED — mergeable, all checks
green, merge button dead.

Root cause: #3622 sharded the dogfood job 2-way, so its checks now
publish as "Dogfood Regression Gate (1/2)" / "(2/2)". Branch protection
still requires the bare context "Dogfood Regression Gate", which no job
emits anymore — the requirement can never be satisfied. #3622's own
comment called for updating branch protection; that step never happened.

Fix keeps the contract in CODE rather than in protection settings: a
tiny aggregator job carrying the stable name, gated on the matrix's
aggregate result. A future shard-count change therefore cannot deadlock
the repo again. `if: always()` + result inspection so a legitimately
skipped matrix (filter says no core paths changed) still satisfies the
gate.

Verified: workflow YAML parses; job graph has dogfood-gate needs:
dogfood with name "Dogfood Regression Gate".

Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 27, 2026
Merged
Closed
os-zhuang added a commit that referenced this pull request Jul 27, 2026
@os-zhuang @claude
chore(ci): cache Release's build + console dist, stop PR-side cache c…
b04b458 
…hurn, move CodeQL/coverage off the hot path (#3666)

Data from today's runs (post-#3622 steady state): every main push paid a
~9.5-minute Release job of which Build (4m21s, no turbo cache — the same
omission lint.yml had) and the vendored Console SPA build (3m11s, a pure
function of the .objectui-sha pin that rarely changes) were 80%. Meanwhile
PR-side cache saves (~5 turbo entries per push) churned the 10 GB Actions
cache pool and evicted the main seeds — observed as sudden cold spikes
(Build Core 51s → 4m30s).

- release.yml: add the standard turbo cache (Build ~4½min → expected <1min),
  and cache packages/console/dist keyed on hashFiles(.objectui-sha,
  build-console.sh) — skip build-console.sh on hit, and verify the dist
  stamp against the pin either way via check:console-sha (its exit-1-on-
  drift contract covers a stale restore).
- ci.yml + lint.yml: turbo caches become restore-only on PRs
  (actions/cache/restore); explicit main-push-only actions/cache/save steps
  (always(), matching the old post-step semantics) keep seeding. PRs read
  main's entries via the existing prefix restore-keys.
- codeql.yml: drop the pull_request trigger (user decision) — ~4½ runner-
  minutes per PR sync; every merge is analyzed on main within minutes at
  the current merge cadence, plus the weekly deep pass. Note left about the
  Analyze check no longer reporting on PRs.
- coverage → nightly (user decision): new coverage-nightly.yml (05:00 UTC +
  workflow_dispatch) owns the instrumented spec suite; ci.yml's push run now
  includes spec's plain test task instead (uninstrumented — net faster).
  Verified locally that spec's suite runs coverage without any build (spec
  has no workspace deps) and the v8 provider is present.

All five workflow files parse; changeset is empty (releases nothing).


Claude-Session: https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt

Co-authored-by: Claude <noreply@anthropic.com>
os-zhuang pushed a commit that referenced this pull request Jul 27, 2026
@claude
fix(ci): dogfood gate passes cancelled runs through — a cancelled mat…
e2efcb6 
…rix is not a failed one

Closes #3668. With cancel-in-progress on, every consecutive push cancelled
the in-flight dogfood matrix (the longest job in the workflow) and the
gate's catch-all branch turned that into a red X on the superseded SHA —
two observed on #3660 alone. False reds train everyone to ignore the one
check that must never be ignored.

Safety premise verified experimentally before landing (per the issue's own
ask): run 30271824408 executed a fail-fast matrix where shard 1 really
failed and fail-fast cancelled shard 2 mid-run — the aggregate
needs.<job>.result reads 'failure', not 'cancelled'. Failure dominates, so
an aggregate of 'cancelled' can only come from the whole run being stopped
externally (supersession or manual cancel) and passing it masks no real
regression. The manual-cancel case going green is the issue's accepted
trade-off; the rejected alternative (skipping the gate via !cancelled())
would republish the #3622 required-context deadlock.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt
@os-zhuang os-zhuang mentioned this pull request Jul 27, 2026
Merged
os-zhuang added a commit that referenced this pull request Jul 27, 2026
@os-zhuang @claude
fix(ci): dogfood gate passes cancelled runs through — a cancelled mat…
c2ecfda 
…rix is not a failed one (#3671)

Closes #3668. With cancel-in-progress on, every consecutive push cancelled
the in-flight dogfood matrix (the longest job in the workflow) and the
gate's catch-all branch turned that into a red X on the superseded SHA —
two observed on #3660 alone. False reds train everyone to ignore the one
check that must never be ignored.

Safety premise verified experimentally before landing (per the issue's own
ask): run 30271824408 executed a fail-fast matrix where shard 1 really
failed and fail-fast cancelled shard 2 mid-run — the aggregate
needs.<job>.result reads 'failure', not 'cancelled'. Failure dominates, so
an aggregate of 'cancelled' can only come from the whole run being stopped
externally (supersession or manual cancel) and passing it masks no real
regression. The manual-cancel case going green is the issue's accepted
trade-off; the rejected alternative (skipping the gate via !cancelled())
would republish the #3622 required-context deadlock.


Claude-Session: https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt

Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Jul 31, 2026
Merged
Merged
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Aug 2, 2026
@os-zhuang @claude
ci: Test Core 按包两路分片,verify-CLI 拆为独立并行 job(PR 关键路径 ~13min → ~7min) (o…
d7759f3 
…bjectstack-ai#4505)

* ci: Test Core 按包两路分片,verify-CLI 拆为独立并行 job

CI 提速:PR 关键路径从 ~13 分钟降到 ~7 分钟。

- Test Core 改为 2 路矩阵,按包分片:scripts/partition-test-shards.mjs 以
  测试文件数为权重做确定性 LPT 均衡(当前全量 573/572)。不用 dogfood 的
  vitest --shard 透传,因为 vitest 4 对测试文件数少于分片数的包直接报错,
  加 --passWithNoTests 后该包在所有分片上都不执行(静默丢覆盖);今天有
  3 个单测试文件的包。required check 名称契约由新增的 test-gate 聚合 job
  以裸 "Test Core" 上下文承载(objectstack-ai#3622 教训,同 dogfood-gate)。
- `objectstack verify` CLI 步骤从 dogfood shard 1 尾部(串行 +4.5 分钟)
  拆为并行的 dogfood-verify job,结果并入既有 Dogfood Regression Gate,
  分支保护无需变更。
- temporal-conformance 的 Turbo 缓存回退键从 turbo-test-* 改指
  turbo-build-core-*:test 命名空间分片化后,单个分片不再是其构建闭包
  的超集,而 Build Core 构建全部包。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U8Ms7unkKxTrmNCX2r1dfs

* ci: dogfood-verify 需构建示例 app 的完整依赖闭包

verify 步骤挂在 dogfood shard 1 时,workspace 已被测试步骤全量构建,
--filter=@objectstack/cli 就够;独立成 job 后,打包的 objectstack.config
在运行时还要导入示例 app 自身的依赖(如 app-showcase 的
@objectstack/connector-mcp),其 dist 无人构建 → verify 时
ERR_MODULE_NOT_FOUND。改用 pkg... 闭包过滤同时构建 CLI 与两个示例 app
的依赖闭包,依赖变动时自维护。已本地实测 app-showcase verify 通过。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01U8Ms7unkKxTrmNCX2r1dfs

---------

Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Aug 3, 2026
Closed
Merged
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Aug 4, 2026
@xuyushun441-sys @claude
ci: merge queue 吞吐——Test Core/Dogfood 各 3 分片 + 队列失败自动分诊评论 (objectstac…
c8ea853 
…k-ai#4863)

* ci: merge queue 吞吐——Test Core/Dogfood 各 3 分片 + 队列失败自动分诊评论 (objectstack-ai#4859)

2026-08-03 上午实测:队列构建关键路径 ~11 min(Test Core 慢分片 9m50s),
且 20 个队列构建 7 个失败,失败被盲目重排、每次连坐重建后面所有构建——
失败放大才是队列慢的主因,构建时长是次因。

- Test Core 2→3 分片:慢分片 ~10min → 预期 ~5-6min。required 上下文由
  test-gate 聚合层承载("Test Core"),分片数可变正是它存在的目的(objectstack-ai#3622)。
- Dogfood 2→3 分片:Test Core 砍掉后的新长杆(7m38s → 预期 ~5min)。顺带
  给其 turbo restore 补 job 级 catch-all 恢复键(Test Core 已有同款),
  新分片号在 main 播种前不至于全冷。
- 新增 merge-queue-triage.yml:merge_group 的 CI 失败 → 自动在 PR 上评论
  失败 job/测试行 + 本 PR 24h 重排失败史 + 队列失败面板 + 分诊清单,
  给人和 PM agent「先诊断再重排」的机器可读信号。cancelled(连坐撤销)
  不触发。

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YNH5jsbfeRTZPwBmBbkwMH

* chore: 空 changeset —— 本 PR 仅改 workflow，不发布任何包

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YNH5jsbfeRTZPwBmBbkwMH

---------

Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Aug 6, 2026
Closed
Open
Closed
Merged
Merged
@os-project-manager os-project-manager mentioned this pull request Aug 8, 2026
Closed
This was referenced Aug 8, 2026
Merged
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

ci/cd documentation Improvements or additions to documentation size/s tooling

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@os-zhuang @claude