feat: retain-on-failure-and-retries trace option

[Yuvalmorami]

Closes #34588

Summary

Adds a new retain-on-failure-and-retries trace mode that records traces for every test attempt (initial run + retries) and retains all of them when any attempt fails. This makes it possible to compare failing and passing runs side-by-side when debugging flaky tests.

Behavior

• Test passes on first try (no retries) → trace discarded (same as retain-on-failure)
• Test fails then passes on retry (flaky) → both traces retained
• Test fails on all attempts → all traces retained

Changes

• packages/playwright/src/worker/testTracing.ts — Always capture trace; abandon only if test passed on first try (retry=0)
• packages/playwright/src/program.ts — Add to CLI modes list
• utils/generate_types/overrides-test.d.ts / packages/playwright/types/test.d.ts — Add to TraceMode union
• docs/src/test-api/class-testoptions.md / docs/src/test-cli-js.md — Document the new mode
• tests/playwright-test/playwright.trace.spec.ts — Tests for flaky, passing, and always-failing scenarios
• tests/playwright-test/playwright.artifacts.spec.ts — Artifact layout test with retries

[Yuvalmorami]

@microsoft-github-policy-service agree

[yury-s]

remove this test, this case is already covered by test(trace:${mode} should not create trace zip artifact if page test passed above

[github-actions]

Test results for "tests 1"

2 failed
❌ [playwright-test] › playwright.unhandled.spec.ts:35 › should produce unhandledRejection when page.route raises @macos-latest-node20
❌ [playwright-test] › test-step.spec.ts:1245 › should report api step failure @ubuntu-latest-node24

2 flaky

⚠️ [chromium-library] › library/popup.spec.ts:258 › should not throw when click closes popup `@chromium-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/inspector/cli-codegen-1.spec.ts:1082 › cli codegen › should not throw csp directive violation errors `@firefox-ubuntu-22.04-node20`

38605 passed, 843 skipped

---

Merge workflow run.

[github-actions]

Test results for "MCP"

7 failed
❌ [chromium] › mcp/profile-lock.spec.ts:28 › isProfileLocked detects a real browser holding the profile @mcp-ubuntu-latest
❌ [chrome] › mcp/autowait.spec.ts:19 › racy navigation destroys context @mcp-windows-latest
❌ [chromium] › mcp/profile-lock.spec.ts:28 › isProfileLocked detects a real browser holding the profile @mcp-windows-latest
❌ [chrome] › mcp/sse.spec.ts:171 › sse transport browser lifecycle (persistent) @mcp-macos-15
❌ [chromium] › mcp/profile-lock.spec.ts:28 › isProfileLocked detects a real browser holding the profile @mcp-macos-15
❌ [firefox] › mcp/cli-session.spec.ts:50 › close non-running session @mcp-macos-15
❌ [webkit] › mcp/sse.spec.ts:171 › sse transport browser lifecycle (persistent) @mcp-macos-15

4970 passed, 171 skipped

---

Merge workflow run.