feat(junit): add includeRetries option to emit each retry as separate testcase

[vazidmansuri005]

Description

Adds a new includeRetries option to the JUnit reporter that emits each retry attempt as a separate <testcase> entry, enabling flaky test detection tools to see full retry history.

Fixes #29446

Changes

• New includeRetries option on JUnit reporter (default: false)
• Environment variable PLAYWRIGHT_JUNIT_INCLUDE_RETRIES for CI configuration
• When enabled, each retry gets its own <testcase> with (retry #N) name suffix
• Failed retries are classified as <failure> or <error> individually
• Test/failure/error counts reflect all attempts (not just the final outcome)
• Default behavior (without the option) is completely unchanged

Usage

// playwright.config.ts
export default defineConfig({
  reporter: [['junit', { includeRetries: true }]],
});

Or via environment variable:

PLAYWRIGHT_JUNIT_INCLUDE_RETRIES=1 npx playwright test --reporter=junit

Example output (test passes on retry #2)

<testsuites tests="3" failures="2" errors="0">
  <testsuite name="a.test.js" tests="3" failures="2">
    <testcase name="one" classname="a.test.js" time="0.005">
      <failure message="..." type="expect.toBe">...</failure>
    </testcase>
    <testcase name="one (retry #1)" classname="a.test.js" time="0.004">
      <failure message="..." type="expect.toBe">...</failure>
    </testcase>
    <testcase name="one (retry #2)" classname="a.test.js" time="0.003">
    </testcase>
  </testsuite>
</testsuites>

Edge cases verified

Scenario	Behavior	Status
Flaky test (fails, fails, passes)	3 testcases: one (failure), one (retry #1) (failure), one (retry #2) (pass)	Tested
All retries fail	N+1 testcases, all with <failure>	Tested
No retries configured	Single testcase, no suffix	Tested
includeRetries: false (default)	Unchanged behavior — 1 testcase	Tested
Skipped test with includeRetries	Falls through to default path correctly	Code verified
includeProjectInTestName + includeRetries	Name: [project1] one (retry #1)	Code verified
Env var PLAYWRIGHT_JUNIT_INCLUDE_RETRIES	Works via getAsBooleanFromENV same pattern as other options	Code verified
Merged report (blob)	All 3 tests pass in "merged" variant too	Tested

Tests

Added 3 new tests covering:

• Flaky test with retries → verifies separate testcases with correct names
• Default behavior unchanged → single testcase when option not set
• All-failing retries → verifies failure counts per attempt

[vazidmansuri005]

@microsoft-github-policy-service agree

[vazidmansuri005]

@yury-s @dgozman — Would appreciate your review on this. This adds an includeRetries option to the JUnit reporter (requested in #29446) so that each retry attempt is emitted as a separate <testcase>, enabling flaky test detection tools to see the full retry history. Default behavior is unchanged.