feat(trace-viewer): show screencast frames during playback by pavelfeldman · Pull Request #42751 · microsoft/playwright

Pavel Feldman (pavelfeldman) · 2026-09-16T22:03:33Z

Summary

Show the screencast in the snapshot area while playing and while dragging the scrubber, so that animations are visible.
Snap back to the action snapshot when the playback stops, the scrubber is released, an action is selected or hovered, or a locator is picked.
Traces with a video play the video itself, the thumbnails only index the film strip lanes.

Playback and dragging the scrubber show the screencast at the current time. Stopping, releasing the scrubber, hovering an action or picking a locator snap back to the action snapshot. Traces with a video play the video itself, the thumbnails only index the film strip lanes.

Yury Semikhatsky (yury-s) · 2026-09-18T00:49:17Z

      target={ariaModeTargets[snapshotTab]}
      point={snapshotTab === 'action' ? action?.point : undefined}
      box={snapshotTab === 'action' ? action?.box : undefined}
+      screencastFrame={screencastFrame}


what if video drifts away from the action and the image is different from the snapshot? may be confusing for the user that stopped on an action

github-actions · 2026-09-18T01:20:12Z

Test results for "tests 1"

2 failed
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) @package-installations-macos-latest
❌ [installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://playwright.download.prss.microsoft.com/dbazure/download/playwright) @package-installations-macos-latest

11 flaky

⚠️ [chromium-library] › library/video.spec.ts:690 › screencast › should capture full viewport `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:762 › screencast › should work with video+trace `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:521 › screencast › should capture static page in persistent context Radoslav Kirilov (@smoke) `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/screencast.spec.ts:28 › screencast.start delivers frames via onFrame callback `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:725 › screencast › should capture full viewport on hidpi `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:257 › third party 'Partitioned;' cookies `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/browsercontext-cookies-third-party.spec.ts:470 › top level 'Partitioned;' cookie and same origin iframe `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/trace-viewer-scrub.spec.ts:226 › should reveal snapshot when hovering an action or picking a locator during playback `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-event-request.spec.ts:181 › should return response body when Cross-Origin-Opener-Policy is set `@firefox-ubuntu-22.04-node20`

52012 passed, 1250 skipped

Merge workflow run.

github-actions · 2026-09-18T01:23:14Z

Hi, I'm the Playwright bot and I took a look at the CI failures here.

🟢 CI is clear — both failures are a macOS installation-bot outage

playwright-cdn.spec.ts:43 › playwright cdn failover should work has been failing on
package-installations-macos-latest for every run since ~2026-09-17 21:00 UTC — including plain
main (shas d16ccf0c, da3a50aa, 464318b1, 3cd10253) and PRs #42758, #42764, #42765,
#42761, #42766. This PR only touches trace-viewer UI code, which never reaches npx playwright install.

Details

Overall: 2 failed, 11 flaky in the latest report. Neither failure is attributable to this PR — the
diff is confined to packages/trace-viewer/src/ui/* (playbackControl.tsx, snapshotTab.tsx,
videoFrame.tsx, videoThumbnails.ts, filmStrip.tsx, ariaModeView.tsx, workbench.tsx, plus CSS)
and a new tests/library/trace-viewer-scrub.spec.ts.

Pre-existing flake / infra

[installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work (https://cdn.playwright.dev/dbazure/download/playwright) and the https://playwright.download.prss.microsoft.com/... variant @package-installations-macos-latest — not a flake, an ongoing break on that bot. Both titles are 29 failed / 611 passed over 640 runs, and every failure is macOS-only (0/626 on Windows, 0/629 on Linux). All four attempts fail in every failing run, so retries never rescue it. Run history for the first title (oldest first, one square per run attempt):

🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟩🟥🟩🟥🟥🟥🟥🟥🟥🟩🟩🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥🟥

The green-to-red flip lands at run 35238682839, before this PR's latest push, and it's solidly red on main since. The error is identical every time: npx playwright install exits 1 on mac-arm64 after the Chrome for Testing 154.0.8037.0 (playwright chromium v1246) zip finishes downloading — so it's the post-download step on the macOS bot, not the CDN failover logic under test. Worth a separate look, but it isn't this PR.
The 11 flaky tests all passed on retry. The video/screencast ones (library/video.spec.ts, library/screencast.spec.ts) are recorder-side, not trace-viewer rendering, so the diff doesn't reach them; the rest (third-party cookies, page-emulate-media, page-event-request, beforeunload) sit in areas the diff doesn't touch.

Worth a glance, not a failure

[firefox-library] › library/trace-viewer-scrub.spec.ts:226 › should reveal snapshot when hovering an action or picking a locator during playback flaked on firefox-ubuntu-22.04-node20 and passed on retry. That's this PR's own new test, and it has no final failures across its runs so far — but it's new, so there isn't much history to lean on yet. Might be worth a look at the hover/playback race before this lands.

_{Triaged by the Playwright bot - agent run}

github-actions · 2026-09-18T01:29:45Z

Test results for "MCP"

5 failed
❌ [chromium] › mcp/network.spec.ts:210 › browser_network_requests lists a failed request once @mcp-macos-latest-chromium
❌ [firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-core.spec.ts:140 › uncheck @mcp-windows-latest-firefox
❌ [firefox] › mcp/cli-drag.spec.ts:19 › drag between elements @mcp-windows-latest-firefox
❌ [msedge] › mcp/http.spec.ts:528 › http transport shared context: one idle timer across clients @mcp-windows-latest-msedge

8624 passed, 1446 skipped

Merge workflow run.

github-actions · 2026-09-18T01:33:19Z

Hi, I'm the Playwright bot and I took a look at the CI failures here.

🟡 Six of the seven failures are provable noise — one I can't clear

The two playwright-cdn failures are a macOS installation-bot outage, and four of the five MCP
failures are bimodal tests with a failure history on main. The odd one out is
[msedge] › mcp/http.spec.ts:528 › http transport shared context: one idle timer across clients,
which has only ever failed on this PR — I can't prove it's a flake, though I also can't find a path
from a trace-viewer UI diff to the MCP HTTP transport.

Details

Overall: 2 failed in "tests 1", 5 failed in "MCP". The diff is confined to
packages/trace-viewer/src/ui/* (playbackControl.tsx, snapshotTab.tsx, videoFrame.tsx,
videoThumbnails.ts, filmStrip.tsx, ariaModeView.tsx, workbench.tsx, plus CSS) and the new
tests/library/trace-viewer-scrub.spec.ts. None of the failing tests load trace-viewer UI code.

Pre-existing flake / infra

[installation tests] › playwright-cdn.spec.ts:43 › playwright cdn failover should work — both
URL variants. Not a flake, an ongoing break on one bot: 58 failures in 1920 runs on
package-installations-macos-latest, and 0 failures in ~5500 runs across the Ubuntu and
Windows installation bots. Every failure is from ~2026-09-17 21:00 UTC onwards, including plain
main (shas d16ccf0c, da3a50aa, 464318b1, 3cd10253, d7ce229c, cd80f507) and PRs
feat(reporter): and printOnlyFailures and --reporter-only-failures #42766, fix(docs): document credentials, opfs, and indexeddb in storageState #42761, feat(trace-viewer): show film strip lanes for all videos, grouped by page #42765, devops(bidi): fix publishing when a single csv report is uploaded #42764, feat(screencast): style action decorations, keep cursor visible #42758. The error is npx playwright install exiting 1 while
downloading Chrome for Testing from cdn.playwright.dev.
[firefox] › mcp/annotate.spec.ts:446 › should switch screencast to -s session on show --annotate
@mcp-windows-latest-firefox — the most bimodal test in this batch: 74 failed / 619 passed over
693 runs, with failures on main (f4244842, ad86e96f, 52a0e720, d1ead3ec, dc0f8522,
and ~25 more) and on unrelated PRs fix(trace-viewer): reject unsafe snapshot tag names #42712, feat(webkit): roll to r2362 #42736, feat(mcp): add --file-paths=absolute for absolute paths in tool results #42673, feat(mcp): offer page-registered WebMCP tools as MCP tools #42671.
[firefox] › mcp/cli-drag.spec.ts:19 › drag between elements @mcp-windows-latest-firefox —
11 failed / 684 passed over 695 runs; failures on main (654654fc, 8db8731c, b856094e,
e4a6eeb6) and PRs fix(docs): document credentials, opfs, and indexeddb in storageState #42761, fix(reporter): fail perfetto writes when the output stream errors #42715, test: fix click tests on android #42726, feat(mcp): add media emulation #42668.
[firefox] › mcp/cli-core.spec.ts:140 › uncheck @mcp-windows-latest-firefox — 12 failed /
683 passed over 695 runs; failures on main (b99b75de, 5812882a) and PRs test: response.body() for content-encoding:identity #42537, fix(video): embed recording start time in WebM metadata #42478,
fix(cli): stop daemon when socket disappears #42454, fix(mcp): drop the backend after browser_close with a shared browser #42365, feat(tracing): allow configuring the screenshots size #42283.
[chromium] › mcp/network.spec.ts:210 › browser_network_requests lists a failed request once
@mcp-macos-latest-chromium — rare, but it has failed off this PR: 2 failed / 711 passed over
713 runs, the other failure on fix(recorder): split showRecorder and startRecording, deliver events per client #42627 (0d0d9978).

Uncertain

[msedge] › mcp/http.spec.ts:528 › http transport shared context: one idle timer across clients
@mcp-windows-latest-msedge — 1 failure in 108 runs on this bot, and it's this run. The same
test on the firefox project failed once on feat(screencast): style action decorations, keep cursor visible #42758, but that's a different project with a
different error, so it doesn't clear this one under my own bar. What it looks like is a timing
race rather than a logic break — the assertion picked up two extra events, close browser: 1 and
disconnect from shared browser: 1, i.e. the idle timer fired while the events were still being
collected. Zero failures on the other 11 MCP bots (~1200 runs). A re-run is the cheapest way to
settle it.

Also worth knowing: the PR's own new test
[firefox-library] › library/trace-viewer-scrub.spec.ts:226 › should reveal snapshot when hovering an action or picking a locator during playback
was flaky on firefox-ubuntu-22.04-node20 — it passed on retry, so it isn't a failure, but it's a
brand-new test with no history and it's the one piece of this report the diff does reach.

_{Triaged by the Playwright bot - agent run}

This comment has been minimized.

Sign in to view

Pavel Feldman (pavelfeldman) force-pushed the playback-screencast branch 2 times, most recently from 467a8ea to 7290921 Compare September 18, 2026 00:13

Pavel Feldman (pavelfeldman) marked this pull request as ready for review September 18, 2026 00:13

This comment has been minimized.

Sign in to view

Pavel Feldman (pavelfeldman) force-pushed the playback-screencast branch from 7290921 to 7e4567b Compare September 18, 2026 00:23

This comment has been minimized.

Sign in to view

Pavel Feldman (pavelfeldman) force-pushed the playback-screencast branch from 7e4567b to c7d2d66 Compare September 18, 2026 00:38

This comment has been minimized.

Sign in to view

Yury Semikhatsky (yury-s) approved these changes Sep 18, 2026

View reviewed changes

github-actions Bot mentioned this pull request Sep 18, 2026

test(coverage): skip coverage tests on tracing bots #42778

Open

Pavel Feldman (pavelfeldman) merged commit 0ee695b into microsoft:main Sep 18, 2026
41 of 45 checks passed

This was referenced Sep 18, 2026

chore: der utils export #42781

Open

feat(reporter): and printOnlyFailures and --reporter-only-failures #42766

Open

Navigation Menu

feat(trace-viewer): show screencast frames during playback - #42751

feat(trace-viewer): show screencast frames during playback#42751
Pavel Feldman (pavelfeldman) merged 1 commit into
microsoft:mainfrom
pavelfeldman:playback-screencast

Pavel Feldman (pavelfeldman) commented Sep 16, 2026 •

edited

Loading

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Yury Semikhatsky (yury-s) Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Pavel Feldman (pavelfeldman) commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Yury Semikhatsky (yury-s) Sep 18, 2026

Choose a reason for hiding this comment

Uh oh!

github-actions Bot commented Sep 18, 2026

Test results for "tests 1"

Uh oh!

github-actions Bot commented Sep 18, 2026

🟢 CI is clear — both failures are a macOS installation-bot outage

Uh oh!

github-actions Bot commented Sep 18, 2026

Test results for "MCP"

Uh oh!

github-actions Bot commented Sep 18, 2026

🟡 Six of the seven failures are provable noise — one I can't clear

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Pavel Feldman (pavelfeldman) commented Sep 16, 2026 •

edited

Loading