feat(test-runner): add repeat config in test

[vctqs1]

Motivation

Playwright currently provides:

• --retries=N to rerun failed tests
• --repeat-each=N to rerun every test multiple times

However, there is no built-in mechanism to repeatedly execute only a subset of tests.

This becomes useful when investigating flaky tests or validating the stability of a newly added test. In these scenarios, running the entire suite repeatedly can be expensive and time-consuming, while retries only execute after failures occur.

Example

test('should repeat 20 times', {
  repeat: 20,
}, async ({ page }) => {
  // ...
});

The test would execute 20 consecutive times regardless of pass/fail status.

Use Cases

• Investigating flaky tests.
• Stress-testing newly added tests.
• Verifying test stability before enabling CI gates.
• Reproducing intermittent failures locally.
• Running targeted stability checks without repeating the entire suite.

Why existing features are insufficient

--retries

• Only reruns failed tests.
• Does not help proactively validate stability.

--repeat-each

• Repeats every test in the suite.
• Can significantly increase execution time when only a few tests need additional validation.

This proposal provides a targeted alternative focused on individual tests.

[vctqs1]

@microsoft-github-policy-service agree

[vctqs1]