feat(firefox): roll to r1549 by microsoft-playwright-automation[bot] · Pull Request #42767 · microsoft/playwright

microsoft-playwright-automation · 2026-09-18T01:20:41Z

Browser build: https://github.com/microsoft/playwright-browsers/commit/45e7c1cb885b3bf6dfb5ef30865dea88b6048956. Build has full platform coverage - check that all tests pass there.

github-actions · 2026-09-18T02:03:13Z

Test results for "tests 1"

2 failed
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://playwright.download.prss.microsoft.com/dbazure/download/playwright) @package-installations-macos-latest

10 flaky

⚠️ [installation tests] › playwright-packages-install-behavior.spec.ts:99 › @playwright/test should work `@package-installations-ubuntu-latest`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/video.spec.ts:113 › screencast › should capture static page `@chromium-ubuntu-22.04-node22`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-event-request.spec.ts:181 › should return response body when Cross-Origin-Opener-Policy is set `@firefox-ubuntu-22.04-node20`
⚠️ [playwright-test] › ui-mode-trace.spec.ts:715 › should indicate current test status `@windows-latest-node22`

51992 passed, 1247 skipped

Merge workflow run.

github-actions · 2026-09-18T02:06:39Z

🟢 Both failures are a pre-existing macOS CDN outage, not this roll

Hi, I'm the Playwright bot and I triaged the CI failures here.

The two failing tests are both installation tests › playwright-cdn.spec.ts:43 › playwright cdn failover should work on package-installations-macos-latest, and they die while downloading Chrome for Testing, not Firefox. The same test is failing right now on main and on unrelated PRs, so this roll is clear.

Details

Overall: 2 failed, 10 flaky, 51992 passed. Neither failure touches Firefox, and nothing in the diff (browsers.json, juggler, Firefox patches) reaches the CDN download path for Chromium.

Pre-existing flake / infra

[installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (both CDN variants — cdn.playwright.dev and playwright.download.prss.microsoft.com) @package-installations-macos-latest

This is a macOS-only CDN failure, and it is happening independently of this PR. In the aggregated CI results the test fails only on package-installations-macos-latest (60 of 1923 runs); it is green on all ubuntu and windows installation bots (0 of ~6400 runs). The failures cluster into an ongoing incident that started around 2026-09-17 15:00 UTC and is currently taking down roughly two thirds of macOS runs:

hour (UTC) runs failed

09-18 01:00 3 2

09-18 00:00 3 2

09-17 23:00 27 18

09-17 22:00 3 2

09-17 21:00 6 4

09-17 19:00 12 8

09-17 17:00 9 6

09-17 07:00 3 0

Same test, same error, on SHAs this PR cannot be responsible for — main at d16ccf0c, da3a50aa, 464318b1, plus feat(reporter): and printOnlyFailures and --reporter-only-failures #42766 (cb72afe4) and fix(docs): document credentials, opfs, and indexeddb in storageState #42761 (652e036e), all within the same hour as this run.

The error is a truncated download of a Chromium build:
```
Command: npx playwright install
EXIT CODE: 1
Downloading Chrome for Testing 154.0.8037.0 (playwright chromium v1246) from
https://cdn.playwright.dev/builds/cft/154.0.8037.0/mac-arm64/chrome-mac-arm64.zip
... 70% of 182.5 MiB
```
It stalls partway through the zip. This roll changes only the Firefox revision, so it can't affect a chrome-mac-arm64.zip fetch.

Not triaged

The 10 flaky results all passed on retry. The two firefox-library › browsercontext-cookies-third-party.spec.ts flakes are the only ones in Firefox territory — worth a glance if they turn solid red on a rerun, but they self-rescued here.

_{Triaged by the Playwright bot - agent run}

github-actions · 2026-09-18T02:07:32Z

Test results for "MCP"

8629 passed, 1446 skipped

Merge workflow run.

github-actions · 2026-09-18T02:46:10Z

Test results for "tests 2"

39 failed
❌ [chromium-library] › library/coverage.spec.ts:34 › should collect istanbul coverage into the trace @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:55 › should collect coverage per trace chunk @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:81 › should report maps once and counters incrementally @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:109 › should accumulate counters across pulls and keep never hit files @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:135 › should collect coverage of a page closed by in-page script @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:157 › should not double count a stash picked up twice @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:179 › should discard stashes of other sessions @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:199 › should collect coverage of an origin left without a page @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:229 › should pull counters as the actions go @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:246 › should not collect coverage without the option @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:259 › should count only the hits after start @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:276 › should stop collecting when tracing stops @tracing-chromium
❌ [chromium-library] › library/coverage.spec.ts:293 › should surface a failure to stash the coverage @tracing-chromium
❌ [firefox-library] › library/coverage.spec.ts:34 › should collect istanbul coverage into the trace @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:55 › should collect coverage per trace chunk @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:81 › should report maps once and counters incrementally @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:109 › should accumulate counters across pulls and keep never hit files @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:135 › should collect coverage of a page closed by in-page script @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:157 › should not double count a stash picked up twice @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:179 › should discard stashes of other sessions @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:199 › should collect coverage of an origin left without a page @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:229 › should pull counters as the actions go @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:246 › should not collect coverage without the option @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:259 › should count only the hits after start @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:276 › should stop collecting when tracing stops @tracing-firefox
❌ [firefox-library] › library/coverage.spec.ts:293 › should surface a failure to stash the coverage @tracing-firefox
❌ [webkit-library] › library/coverage.spec.ts:34 › should collect istanbul coverage into the trace @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:55 › should collect coverage per trace chunk @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:81 › should report maps once and counters incrementally @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:109 › should accumulate counters across pulls and keep never hit files @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:135 › should collect coverage of a page closed by in-page script @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:157 › should not double count a stash picked up twice @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:179 › should discard stashes of other sessions @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:199 › should collect coverage of an origin left without a page @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:229 › should pull counters as the actions go @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:246 › should not collect coverage without the option @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:259 › should count only the hits after start @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:276 › should stop collecting when tracing stops @tracing-webkit
❌ [webkit-library] › library/coverage.spec.ts:293 › should surface a failure to stash the coverage @tracing-webkit

36 flaky

⚠️ [chromium-page] › page/page-screenshot.spec.ts:504 › page screenshot › should take fullPage screenshots during navigation `@tracing-chromium`
⚠️ [chromium-library] › library/defaultbrowsercontext-1.spec.ts:28 › context.cookies() should work Radoslav Kirilov (@smoke) `@msedge-dev-windows-latest`
⚠️ [chromium-library] › library/browsercontext-add-cookies.spec.ts:426 › should allow unnamed cookies `@chrome-ubuntu-22.04`
⚠️ [chromium-library] › library/browsercontext-user-agent.spec.ts:110 › should work for navigator.userAgentData and sec-ch-ua headers `@msedge-windows-latest`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-windows-latest`
⚠️ [chromium-page] › page/workers.spec.ts:63 › should have timestamp on worker console messages `@chromium-windows-latest`
⚠️ [chromium-library] › library/browsercontext-page-event.spec.ts:173 › should work with Ctrl-clicking `@chromium-macos-15-large`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:373 › should produce network events, routing, and annotations for Service Worker `@chromium-macos-14-xlarge`
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:179 › should take screenshot `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@chromium-macos-15-xlarge`
⚠️ [chromium-library] › library/selector-generator.spec.ts:180 › selector generator › should not escape text with >> `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:272 › selector generator › should use readable id `@chrome-macos-latest`
⚠️ [chromium-library] › library/selector-generator.spec.ts:569 › selector generator › should not accept invalid role for candidate consideration `@chrome-macos-latest`
⚠️ [chromium-library] › library/selectors-register.spec.ts:29 › should work `@chrome-macos-latest`
⚠️ [chromium-library] › library/video.spec.ts:152 › screencast › should continue recording main page after popup closes `@chrome-macos-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@tracing-firefox`
⚠️ [firefox-page] › page/page-goto.spec.ts:90 › should work with Cross-Origin-Opener-Policy `@tracing-firefox`
⚠️ [firefox-page] › page/page-wait-for-function.spec.ts:43 › should poll on interval `@tracing-firefox`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-large`
⚠️ [firefox-library] › library/inspector/cli-codegen-aria.spec.ts:87 › should update aria snapshot highlight `@firefox-macos-15-large`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-macos-15-large`
⚠️ [firefox-page] › page/page-request-continue.spec.ts:288 › should work with Cross-Origin-Opener-Policy `@firefox-macos-15-large`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-windows-latest`
⚠️ [firefox-library] › library/inspector/cli-codegen-python.spec.ts:36 › should print the correct context options for custom settings `@firefox-windows-latest`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/browsercontext-reuse.spec.ts:412 › reuse launch › should update viewport and media `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/trace-viewer.spec.ts:1915 › should not leak recorders `@firefox-macos-15-xlarge`
⚠️ [firefox-library] › library/video.spec.ts:294 › screencast › should capture navigation `@firefox-macos-15-xlarge`
⚠️ [firefox-page] › page/page-add-init-script-callback.spec.ts:50 › should pass arguments to the callback `@firefox-macos-15-xlarge`
⚠️ [webkit-library] › library/signals.spec.ts:25 › should close the browser when the node process closes `@webkit-windows-latest`
⚠️ [webkit-library] › library/browsertype-connect.spec.ts:813 › launchServer › should upload a folder `@webkit-macos-15-xlarge`
⚠️ [webkit-library] › library/tracing.spec.ts:533 › should produce screencast frames crop `@webkit-macos-15-xlarge`

111915 passed, 4744 skipped

Merge workflow run.

github-actions · 2026-09-18T02:49:29Z

🟢 All 39 failures are already red on `main` — this roll is clear

Hi, I'm the Playwright bot and I triaged the CI failures here.

Every failure in "tests 2" is library/coverage.spec.ts on the @tracing-* bots, failing with Error: tracing.start: Tracing has been already started — identically on chromium, firefox and webkit. The same 39 failures land on main (d16ccf0c, da3a50aa, 464318b1, 63919674, …) and on the other in-flight rolls, so they pre-date this Firefox roll.

Details

Overall: 39 failed, 36 flaky, 111915 passed. A Firefox roll cannot explain chromium and webkit failing the same 13 tests, and the diff here is browsers.json, UPSTREAM_CONFIG.sh, juggler FrameTree.js, the bootstrap patch and device descriptors — none of which reach tracing setup.

Pre-existing flake / infra

library/coverage.spec.ts (13 tests × chromium/firefox/webkit) @tracing-chromium @tracing-firefox @tracing-webkit — the spec starts its own trace, which conflicts with the @tracing-* bots that already turn tracing on, so all 13 tests fail with tracing.start: Tracing has been already started. In the aggregated CI results this is an exact, deterministic 39-failure signature that appears on every tests 2 run since at least 2026-09-16 22:00 UTC, on main at bf9d04f2, 24b69326, 8bb2c097, ad9c8593, 63919674, cd80f507, d7ce229c, 3cd10253, da3a50aa, 464318b1, d16ccf0c — and on the sibling rolls feat(webkit): roll to r2365 #42762 (webkit fbd2504c) and feat(chromium): roll to r1246 #42757 (chromium 60621bc9). It is not flaky, it is broken on main: the spec needs a @tracing-* gate, or to skip when the context already has a trace running.
library/browsercontext-cookies-third-party.spec.ts:257 / :470 (firefox, on tracing-firefox, firefox-macos-15-large, firefox-macos-15-xlarge, firefox-windows-latest) — these two flaked here but passed on retry. Long-standing flake, not the roll: on main alone a retry rescues them in 1201 of 2576 runs, with 31 outright failures. The rate looks the same on this branch.

Not triaged

The remaining 34 flaky results all passed on retry and spread across chromium, firefox and webkit with no shared signature.

_{Triaged by the Playwright bot - agent run}

feat(firefox): roll to r1549

3f1485b

microsoft-playwright-automation Bot added the CQ1 label Sep 18, 2026

Devin Rousso (dcrousso) approved these changes Sep 18, 2026

View reviewed changes

Devin Rousso (dcrousso) merged commit ebf3a87 into main Sep 18, 2026
71 of 75 checks passed

Devin Rousso (dcrousso) deleted the roll-into-pw-firefox/1549 branch September 18, 2026 04:39

This was referenced Sep 18, 2026

test(coverage): skip coverage tests on tracing bots #42778

Open

chore: der utils export #42781

Open

feat(reporter): and printOnlyFailures and --reporter-only-failures #42766

Open

Navigation Menu

feat(firefox): roll to r1549 - #42767

feat(firefox): roll to r1549#42767
Devin Rousso (dcrousso) merged 1 commit into
mainfrom
roll-into-pw-firefox/1549

microsoft-playwright-automation Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

microsoft-playwright-automation Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Test results for "tests 1"

Uh oh!

github-actions Bot commented Sep 18, 2026

🟢 Both failures are a pre-existing macOS CDN outage, not this roll

Uh oh!

github-actions Bot commented Sep 18, 2026

Test results for "MCP"

Uh oh!

github-actions Bot commented Sep 18, 2026

Test results for "tests 2"

Uh oh!

github-actions Bot commented Sep 18, 2026

🟢 All 39 failures are already red on main — this roll is clear

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

🟢 All 39 failures are already red on `main` — this roll is clear