Add --scan-dryrun-results for Maven subset to honor pom.xml test filtering - #1363
Add --scan-dryrun-results for Maven subset to honor pom.xml test filtering#1363ItIsUday wants to merge 1 commit into
Conversation
…ering The Maven subset command scans all *Test.java files and sends them to the backend, ignoring Surefire filtering (excludedGroups, excludes, profiles). Excluded tests then have no matching report at record time, producing "missing test report" warnings and skewing subset accuracy. --scan-dryrun-results parses target/surefire-reports/TEST-*.xml instead of scanning source files. Run `mvn test -Djunit.platform.execution.dryRun.enabled=true` first: Surefire applies all its filtering before the JUnit 5 dry run, so only the classes Maven would actually run get a report. JUnit 5 only. Hard-fails when no reports are found rather than silently sending an empty subset.
Konboi
left a comment
Konboi
left a comment
There was a problem hiding this comment.
My understanding is that this is basically the same as this PR, but it emphasizes that the scope is limited to JUnit 5. Is that correct?
Also, since the target tenant is using the Launchable commands (V1), please create the branch from v1 rather than main. After merging the changes into v1, please merge v1 into main.
|
The main difference compared to that PR is the use of Sure I'll update the PR to point to v1 |
|
v1 counterpart is #1368. These land independently — not merging v1 → main, since main has diverged a lot with v1 |
|
Putting this in draft. Per the v1 → v2 forward-porting guide, the sanctioned path is to land the v1 change (#1368) first, then forward-port it into main via |
|
Superseded by #1370, which forward-ports the change from v1 via |
2 participants
Why
The Maven
subsetcommand scans every*Test.javaunder the source roots and sends them all to the backend. But Surefire filters tests at runtime viapom.xml—<excludedGroups>/<includedGroups>(JUnit 5@Tag),<excludes>/<includes>(class globs), and profiles. The subset never sees that filtering, so it includes tests Maven will never run. Atrecordtime those tests have no matching report, which produces "missing test report" warnings and skews subset accuracy.This is the customer-reported case behind LCHIB-693:
@Tag("IntegrationTest")/@Tag("SmokeTest")tests excluded via<excludedGroups>IntegrationTest,SmokeTest</excludedGroups>still get sent to the subset.What
New
--scan-dryrun-resultsflag onsubset ... maven. Instead of scanning source files, it parses**/target/surefire-reports/TEST-*.xmland sends the fully-qualified class name from each report's rootnameattribute.The workflow is to run a JUnit 5 dry run first:
Limitations
@Categoryand TestNG groups are not covered.