Skip to content

P12: Windows e2e shard 2/10 (every outcome class alone) takes 10 min #118

Open

Description

@KyleAMathews

Summary

The Windows e2e job Windows gap report end-to-end oracle (fixed seed, 2/10) runs one unit, fixed seed 20261105: every outcome class. On PR #115 it took 10m05s (run 43783774902, job 133026717354; the unit alone took 562 s). On main it took 9m13s (run 43693786590, job 132749781500). That is over, or at, the P12 budget of 10 minutes per job.

#115 does not cause it. Its only change is an oracle comparison that is linear in the number of gap lines, and the unit takes as long on main. The job also fails on main and on #115 on the reach gate: "did not enter a hang cut off for idleness". That is the Windows idle gap recorded in the e2e oracle's Limits, and #99 owns it.

Why the shard can't be split

Shard 2/10 already holds this one unit (scripts/pr-shards.mjs, oracle-gaps-fixed-seed-windows, shard 2). Splitting the job means splitting the constructed history itself, as #73 did when it moved the warm and abort stops into every outcome class, the warm and abort stops. Or the witness module's win32 cost must come down.

Options (for the maintainer)

  1. Split every outcome class again, for example the hang and slow witnesses apart from the rest. The framework checks reach over the union of the halves, as it does for the current split.
  2. Record a win32 exception to P12 for this unit while the Windows e2e jobs are non-blocking (C entry (c)).
  3. Reduce the win32 cost of the hang witnesses: the idle sampler's window and the backstop.

Related: #106, the same P12 problem for the witness module's determinism unit on Linux.

Authority

P12 (register): every CI job stays under 10 minutes, and the cost comes from real cuts first and sharding second.

Activity

  1. KyleAMathews commented on Nov 16, 2026

    @KyleAMathews
    ContributorAuthor

    Adding Windows 4/10 to this issue (from the merge gate of PR #57, its F5 finding).

    Under P12 this shard is inside the 10-minute job limit, but with only about 10 s of margin. Rebalancing it is a follow-up for this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    windowsWindows-specific behaviour

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions