Skip to content

ci(e2e): the nightly also runs the stress suite on chromium in two shards - #572

Merged
arrzdev merged 1 commit into
mainfrom
ci/nightly-stress-chromium
Nov 17, 2026
Merged

arrzdev merged 1 commit into
mainfrom
ci/nightly-stress-chromium

Conversation

@arrzdev

@arrzdev arrzdev commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

The stress suite ran only in the Sunday e2e-full matrix, so a stress regression (the dev SSR socket reset, the WebKit reload failure) showed up a week late at best. The nightly ran only the five PR suites.

What changes in .github/workflows/e2e.yml:

  • New job e2e-stress: test:e2e:stress --project=chromium, in two Playwright shards (--shard=n/2), on the nightly cron beside e2e. The suite is one worker by design and takes ~30 min in one job; per-file timings from the 2026-11-08 Sunday run put the split at ~16 / ~14 min, so the nightly's wall clock stays that of e2e (~26 min). WebKit stays weekly.
  • Same "skip if main has not moved" step and the same "open an issue for a red nightly" step as e2e. A red shard fails the scheduled run, which is what the existing failure monitoring already reads; nothing new to wire.
  • workflow_dispatch gains a scope input: full (default, unchanged: e2e-full) or nightly (the e2e and e2e-stress jobs on the dispatched ref), so the nightly set can be run on a branch.
  • Docs that describe the nightly (docs/guides/e2e.md, docs/guides/testing.md, docs/roadmap/open-questions.md O11a, CLAUDE.md) say so.

On Sundays chromium stress runs twice (03:00 nightly and 04:00 full). Left as is: the minutes are free on a public repo and the extra sample is useful.

Reproduce: gh run list --workflow e2e.yml --event schedule on main: no scheduled run before this PR has a stress job except the Sunday one.

Verify:

  • gh workflow run e2e.yml --ref ci/nightly-stress-chromium -f scope=nightly → https://github.com/arrzdev/adaptv/actions/runs/44024358836
    • e2e-stress (chromium, 1/2): 116 tests, 115 passed (15.0m in Playwright), job 15.6 min, green
    • e2e-stress (chromium, 2/2): 116 tests, 116 passed (15.2m), job 16.3 min, green
    • e2e: 26.5 min, so the run's wall clock is 26.6 min (it was ~26 min before; the stress shards finish ~10 min earlier)
    • e2e-full: skipped, as it should be for scope=nightly
    • e2e was red on one test, unrelated to this change (workflow-only diff): plain › wheel-column.spec.ts:259 "a fast mouse fling coasts past where the pointer let go", the settle predicate timed out with 29 px left (≤ 1 expected). The same spec passed in the main suite of the same job. Tracked separately as a flake.
  • YAML parses (python3 -c "import yaml; yaml.safe_load(open('.github/workflows/e2e.yml'))").

Tested on: GitHub Actions ubuntu-latest (dispatch above).
Not tested: the schedule trigger itself (it fires only on main); the skip and issue steps are copied verbatim from e2e.

@arrzdev
arrzdev marked this pull request as ready for review November 17, 2026 05:55
@arrzdev

arrzdev commented Nov 17, 2026

Copy link
Copy Markdown
Owner Author

Reviewed: the workflow change is correct and the dispatch run (44024358836) shows both stress shards green, with e2e-full skipped for scope=nightly. The e2e red in that run is the wheel-column.spec.ts:259 fling flake, and this PR doesn't touch it.

Checked:

  • if: guards. The skip step runs only on schedule. On a dispatch its output is empty, so != 'false' lets the tests run, which is what we want.
  • "Last green scheduled run": the run now succeeds only when both e2e and e2e-stress are green, so a red stress shard makes the next night run again. A green Sunday e2e-full run still counts, and it covers stress.
  • Plain workflow_dispatch keeps the old behaviour (scope defaults to full).

Nits, not blocking. Leaving them for a follow-up:

  • concurrency.group doesn't include inputs.scope, so a scope=nightly dispatch and a scope=full dispatch on the same ref cancel each other. This was already true for any two dispatches.
  • When both shards fail, one night opens two issues.
  • The edited lines in CLAUDE.md and docs/guides/e2e.md go past the ~100-column wrap.

Squash-merging.

@arrzdev
arrzdev merged commit f4ff6d9 into main Nov 17, 2026
8 of 9 checks passed
@arrzdev
arrzdev deleted the ci/nightly-stress-chromium branch November 17, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant