Skip to content

ci(coverage): run the coverage job on a sixteen vCPU runner - #4664

Merged
thymikee merged 1 commit into
test/host-kit-proc-fixturefrom
ci/coverage-16vcpu
Nov 18, 2026
Merged

thymikee merged 1 commit into
test/host-kit-proc-fixturefrom
ci/coverage-16vcpu

Conversation

@thymikee

@thymikee thymikee commented Nov 17, 2026 •

Copy link
Copy Markdown
Member

Summary

The Coverage job ran on blacksmith-8vcpu-ubuntu-2404. Its suite is CPU-bound — instrumented execution plus transform — and Vitest takes its worker count from the machine, so seven workers left the rest of the run queueing. Moving the job to blacksmith-16vcpu-ubuntu-2404 doubles the workers.

One runs-on line in .github/workflows/ci.yml, plus a comment. No gate, threshold or verdict changes: the same unit-ci, coverage-model, coverage-crap-model and tmpdir-leaks gates run in the same order, with the changed-line and CRAP steps still reusing the one lcov.

Depends on #4662, which has to merge first. Three host-kit files let the runner answer for a fixture pid through /proc/<pid>/stat; those pids were unused on an 8 vCPU runner and used on a 16 vCPU one, so the job would go red at exactly this head.

Validation

Measured in CI at identical gates, Run coverage step and whole job from the Actions API.

Runner Run coverage Job Job min x rate Cost/PR
8 vCPU — 35 green runs 298 s p50, 214–389 338 s p50 5.63 x $0.016 $0.090
16 vCPU — 4 runs at this head 146, 146, 174, 176 s 172, 179, 203, 209 s 3.1 avg x $0.032 $0.099
32 vCPU — run 44176250780 98 s 111 s 1.85 x $0.064 $0.118

16 vCPU samples: run 44175823688 job 1, then attempts 1–3 of 44186117469. The spread is 146–176 s against 214–389 s on 8 vCPU, so the worst 16 vCPU sample still beats the 8 vCPU p90. Every gate step was green in all four. 16 vCPU halves the wall time for about 10% more money; 32 vCPU was green only in the probe lane and costs 31% more, so 16 is the pick.

Head 02cc3803c4. node --experimental-strip-types scripts/gate/check.ts — manifest ok. Unresolved risk: the coordinate.contract.test.ts 5 s timeout seen once on #4660 was not reproduced in these four runs, on 8 or 16 vCPU.

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 6.38 MB 6.38 MB 0 B
Package (unpacked) 6.38 MB 6.38 MB 0 B
Package (download) 1.94 MB 1.94 MB -8 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 40.2 ms 44.0 ms +3.8 ms
CLI --help 81.8 ms 92.4 ms +10.6 ms

@thymikee

Copy link
Copy Markdown
Member Author

The change at b004cb9 is ready: it only switches the Coverage job to the 16 vCPU runner label, and that job passed on it at this head (run 44175823688). All other checks except one pass, and that one is still queued, not failed. There are no conflicts. Before merge, #4662 needs to land first and the queued check needs to finish. Not blocking, take or leave: the three-line comment at ci.yml#L293 repeats the PR body's measurements, which will go stale, so one line like "16 vCPU: the suite is CPU-bound; halves wall time at about equal billed minutes (#4664)" would do; and one green run (146 s) does not show 16 vCPU variance against the 8 vCPU range of 214-389 s, so re-running Coverage two or three more times on this head would show whether a larger runner exposes tests that assume small pids or fixed /proc reads, as #4662 did for three host-kit files. Runner rates and quota limits for 16 vCPU are not visible from the repo, so the cost side is a maintainer call.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Nov 17, 2026
The job's suite is CPU-bound: instrumented execution and transform, with Vitest deriving its worker
count from the machine. On eight vCPU the seven workers left the rest of the run queueing. The same
gate on a sixteen vCPU runner finished the run step in 146 s, against 298 s at the median of green
runs on eight, for close to the same runner cost.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@apex-sentinel-staging

Copy link
Copy Markdown

Apex Sentinel · ✅ Approve · review of 02cc380

No open findings.

Reviewed with this repository's own Sentinel configuration (655ede2796e5).

💬 Talking to Sentinel

In a finding's thread, reply with:

Command What it does
@apex-sentinel-staging note <context> Adds context; Sentinel re-checks the cited code
@apex-sentinel-staging pivot <why> Says the finding is wrong; Sentinel re-checks the cited code
@apex-sentinel-staging like Marks the finding as useful
@apex-sentinel-staging dislike <why> Marks it as not useful, so it is not raised again

On this conversation, comment @apex-sentinel-staging review to run a full review once again.

@thymikee

Copy link
Copy Markdown
Member Author

Applied both notes.

  1. The runs-on comment is one line now (ci.yml:293); the reasoning is in the commit message and the body instead. Head 02cc3803c4.
  2. Three more Coverage runs at a head differing only by that comment, re-running the job in place:
Attempt Run coverage Job
44175823688 job 1 (b004cb9d1c) 146 s 179 s
44186117469 attempt 1 146 s 172 s
44186117469 attempt 2 176 s 209 s
44186117469 attempt 3 174 s 203 s

Range 146–176 s for the step, 172–209 s for the job. The slowest sample is still under the 8 vCPU p90 of 360 s, and under its max of 389 s. All four green, with Run coverage, tmpdir-leaks, the changed-line gate and both CRAP steps green in each. The body's table carries the spread.

Not seen: the coordinate.contract.test.ts 5 s timeout. It is absent from all four 16 vCPU logs and from the 8 vCPU runs measured before, so it is not a cost of this change on the evidence I have. Left unfixed here, as agreed.

@thymikee
thymikee added this pull request to stack #4666 November 18, 2026 06:44
@thymikee
thymikee merged commit c8e0e10 into main Nov 18, 2026
30 checks passed
@thymikee
thymikee deleted the ci/coverage-16vcpu branch November 18, 2026 06:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant