ci: PR-time Test Core (6/6) reds on shard-timing-drift at 1.60–1.62x on three PRs that change low-graph packages (core, metadata-*) — runtime, service-analytics, rest and client run 1.4–2.4x their refreshed weights #22966

Filing gate ①, class (a): a reproducible CI defect that reds pull requests whatever their diff. reach: three named producers in 25 minutes, listed below. Every one is a PR CI run, not a queue build.

Filed by domain:engine seat 2 (#20966) · os-steve · session_01ADzJtzYTLUfgrRZHxkagkX · 2026-10-12T05:52Z. ⛔ Not a claim. Lane and grade are triage's. #22917 (closed) recorded this step's previous signature, on shard 3/6 in the queue. Its landing record (6117493951) says "A red on the same signature after this point files fresh, per the card convention", so this card is the fresh one.

What is measured

The step is Test Core (6/6) → "Check this shard's timing drift" (node scripts/partition-test-shards.mjs --check-drift, #16465: a warning past 1.3x, red past 1.5x). The dataset is scripts/test-shard-timings.json as 1bd612bd68 (#22923) refreshed it on 2026-10-12. Every run below accounts for all of its declared tests (check-test-completeness: OK). The only red is the timing comparison.

• The first row was recorded on ci: merge-queue Test Core (3/6) reds on shard-timing-drift — @objectstack/cli measured 931.4s vs 579.6s predicted (1.61x, red past 1.5x); scripts/test-shard-timings.json was last refreshed 2026-10-08 #22917 by domain:cli seat 2 (6118439015). It also read runtime's vitest line as import 2157.27s against tests 239.30s.
• None of the three PRs touches a package in the overshoot lists. PR feat(verify): the delete-behaviour derived proof family, runnable as os verify --delete-behaviors (#15953 slice 2) #22953 touches a packages/core test file. PR feat(core): pure row-to-definition mappers for the ADR-0131 upgrade ceremony (C7b build-ahead, PR 1 of 2) #22959 adds two pure functions to packages/core. PR fix(metadata-protocol): two locales of one email template survive a restart — sys_metadata overlay rows keyed by bundle member (#22877, stage 1) #22954 changes packages/metadata-core, metadata-protocol and platform-objects. All three sit low in the dependency graph, so shard 6 executes its whole package set (runtime, rest, service-analytics, client, plugin-approvals and others) instead of replaying most of it from cache.
• CONTROL: main's completed CI runs since the refresh are green: push 38208625537 on da9c2bf2f2, the hourly schedule run 38206660804 on 51b43a8dc4, and pushes 38204522474 and 38203196264. PR fix(objectql)!: a predicate rule's read through a master-detail FK asks the caller's read door for the header first — an unreadable header answers as a nonexistent one (#22892) #22956 and PR fix(objectql)!: a multi-valued reference element is held to the single value's stored form (#22908) #22947 (packages/objectql) passed all six shards on their heads.

Why (as read, not measured)

The overshoot is the same packages at nearly the same ratio on three unrelated diffs, and runtime's import time dwarfs its test time. That reads as contention: on a PR whose affected set executes every package on shard 6 at once, each package runs much slower than the per-package weights the dataset records. A refresh will not close that gap if the window it measures has fewer packages executing concurrently than these PR runs. That is a hypothesis, not a measurement.

Not measured

• Whether shards 1 to 5 sit near 1.5x on these same PRs.
• The concurrency of the 12 hourly runs behind the 2026-10-12 dataset.
• Whether one re-run of the same head goes green. This seat has not re-run anything.

Cost

Every PR that changes a package low in the graph (core, metadata-*, spec) reds Test Core, a required context, at PR time. Today that is PRs #22953, #22954 and #22959 in three lanes. It is not a queue-only cost: these PRs cannot be armed at all while it holds.

Direction (not a ruling)

The step's own text forbids hand-editing the dataset and raising the 1.5x bound. The remedy is triage's to route and, where #16465's bound is concerned, the maintainer's. Neighbour: #22949 is the same morning's runner-load reading on the other timing gate (the per-package ceilings, #16468). If triage folds the two, this card's rows move there.

Dedupe

• MCP search_issues on this repo, closed included, for shard timing drift Test Core red measured vs predicted refresh dataset: the only on-signature hit is ci: merge-queue Test Core (3/6) reds on shard-timing-drift — @objectstack/cli measured 931.4s vs 579.6s predicted (1.61x, red past 1.5x); scripts/test-shard-timings.json was last refreshed 2026-10-08 #22917, which is closed.
• REST list of every card created since 04:30Z: none on this step; ci: merge-queue Test Core reds on the new suite-duration ceiling — @objectstack/cloud-connection measured 177.5 s and 200.3 s vs its 163.89 s ceiling, in two groups of PRs that do not touch it (live since 1bd612bd6) #22949 is on the other step.

Dedupe words: shard-timing-drift; check-drift; Test Core (6/6) drift; measured vs predicted; partition-test-shards; test-shard-timings.json refresh; runtime import time.

Generated by Claude Code