Add --scan-dryrun-results for Maven subset to honor pom.xml test filtering - #1368
Merged
Conversation
…ering The Maven subset command scans all *Test.java files and sends them to the backend, ignoring Surefire filtering (excludedGroups, excludes, profiles). Excluded tests then have no matching report at record time, producing "missing test report" warnings and skewing subset accuracy. --scan-dryrun-results parses target/surefire-reports/TEST-*.xml instead of scanning source files. Run `mvn test -Djunit.platform.execution.dryRun.enabled=true` first: Surefire applies all its filtering before the JUnit 5 dry run, so only the classes Maven would actually run get a report. JUnit 5 only. Hard-fails when no reports are found rather than silently sending an empty subset.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Maven
subsetcommand scans every*Test.javaunder the source roots and sends them all to the backend. But Surefire filters tests at runtime viapom.xml—<excludedGroups>/<includedGroups>(JUnit 5@Tag),<excludes>/<includes>(class globs), and profiles. The subset never sees that filtering, so it includes tests Maven will never run. Atrecordtime those tests have no matching report, which produces "missing test report" warnings and skews subset accuracy.This is the customer-reported case behind LCHIB-693:
@Tag("IntegrationTest")/@Tag("SmokeTest")tests excluded via<excludedGroups>IntegrationTest,SmokeTest</excludedGroups>still get sent to the subset.What
New
--scan-dryrun-resultsflag onsubset ... maven. Instead of scanning source files, it parses**/target/surefire-reports/TEST-*.xmland sends the fully-qualified class name from each report's rootnameattribute.The workflow is to run a JUnit 5 dry run first:
Limitations
@Categoryand TestNG groups are not covered.