HADOOP-19978. Upgrade maven-surefire-plugin to 3.6. - #8720
Open
joseluisll wants to merge 1 commit into
Open
Conversation
Bump maven-surefire-plugin.version from 3.5.3 to 3.6.0. 3.6.0 ships
SUREFIRE-1639, which teaches the JUnit Platform provider to read the
"listener" property that eight poms already carry: it loads each named
class and registers it as a TestExecutionListener. The 3.5.3 provider
has no listener handling at all, so the property is inert there. 3.6.0
is the floor: 3.6.0-M1 carries only the JUnit 4 RunListener adapter, not
the TestExecutionListener path.
The listener stays dormant until HADOOP-19964 lands. TimedOutTestsListener
lost its RunListener base in HADOOP-19415, so today it implements neither
interface, and 3.6.0 skips a listener class it cannot adapt.
Move -Djava.net.preferIPv4Stack=true out of surefire's
systemPropertyVariables and into maven-surefire-plugin.argLine.
systemPropertyVariables are applied by the forked booter once the JVM is
already running, so they never reach the fork's input arguments. 3.6.0
initialises the fork's networking before it applies them, so the flag is
read too late and the fork falls back to IPv6; 3.5.3 applies them before
any networking class loads, which is why the same configuration works
there. As an argLine entry the flag is a real JVM startup argument and
cannot be late. The 22 argLine overrides in the tree all interpolate
${maven-surefire-plugin.argLine}, and maven-failsafe-plugin shares the
property, so it propagates everywhere. hadoop-registry carried its own
copy of the same ineffective systemPropertyVariable; drop it, as its
argLine override already interpolates the property.
Restore -DskipTests over the integration tests. SUREFIRE-823 removed the
${skipTests} binding from maven-failsafe-plugin's own skipTests
parameter, so on 3.6.0 -DskipTests skips only surefire and failsafe runs
anyway: hadoop-client-integration-tests executed five integration tests
including a MiniCluster, and hadoop-azure forked its whole ABFS suite,
under a plain "mvn install -DskipTests". Bind the parameter back in
hadoop-project so -DskipTests again means no tests at all. skipITs is
untouched, so -DskipITs still skips only the integration tests and the
auth-keys.xml gating in hadoop-aws, hadoop-gcp and hadoop-bos keeps
working.
Add hadoop-common's test-jar in test scope to hadoop-yarn-services-core
and hadoop-yarn-applications-catalog-webapp. Both inherit the listener
property from hadoop-yarn but resolve only hadoop-common's main jar, by
any path including hadoop-minicluster. Surefire swallows the resulting
ClassNotFoundException and skips the listener without a warning, so
without this the two modules would silently lose it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
joseluisll
force-pushed
the
HADOOP-19978
branch
from
September 5, 2026 01:40
d21e0e8 to
c86ba55
Compare
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
Upgrades
maven-surefire-pluginfrom 3.5.3 to 3.6.0, plus the three changes the new version needs.3.6.0 is the first release that registers surefire's
listenerproperty as a JUnit PlatformTestExecutionListener(SUREFIRE-1639). Eight poms already carry that property forTimedOutTestsListener, and on 3.5.3 it does nothing at all. This bump is what lets HADOOP-19964's listener bind, so a timed-out test prints a thread dump again. Until 19964 lands the listener stays dormant — this change is what unblocks it.java.net.preferIPv4Stackmoves intomaven-surefire-plugin.argLine. As asystemPropertyVariableit is applied only after the forked JVM has started. 3.6.0 initialises the fork's networking before applying those properties, so the flag arrives too late and the fork falls back to IPv6, turning 5 of 7 CI test jobs red; 3.5.3 applies them before any networking class loads, which is why the same configuration works there. As an argLine entry the flag is a real JVM startup argument and cannot be late. All the<argLine>overrides in the tree interpolate the property and maven-failsafe-plugin shares it, so it propagates everywhere.hadoop-registrycarried a second copy of the same ineffective setting; removed.-DskipTestsis restored over the integration tests. SUREFIRE-823 removed the${skipTests}binding from maven-failsafe-plugin's ownskipTestsparameter, so on 3.6.0-DskipTestsskips surefire only and failsafe runs regardless. Measured on a plainmvn install -DskipTests:hadoop-client-integration-testsexecuted five integration tests including a MiniCluster, andhadoop-azureforked its entire ABFS suite. This binds the parameter back inhadoop-project, so-DskipTestsagain means no tests at all.skipITsis untouched, so-DskipITsstill skips only integration tests and theauth-keys.xmlgating inhadoop-aws,hadoop-gcpandhadoop-boskeeps working.hadoop-common's test-jar added tohadoop-yarn-services-coreandhadoop-yarn-applications-catalog-webapp. Both inherit the listener property fromhadoop-yarnbut resolve onlyhadoop-common's main jar, so the listener class cannot be loaded there and surefire silently drops it.How was this patch tested?
WSL2 Ubuntu 24.04, OpenJDK 17, Maven 3.9.16. Full reactor build clean.
Full suites on the modules this touches, all green:
hadoop-registry(164 tests),hadoop-yarn-services-core(145),hadoop-yarn-applications-catalog-webapp(5).hadoop-commonfull suite run against unpatched trunk at the same commit as a control: every failure reproduces on the baseline, and the two that did not pass on rerun. No new failures.The tests that fail when 3.6.0 is bumped without the argLine fix all pass:
TestDNS,TestNameNodeRpcServer,TestBlockRecovery2,TestWebHdfsTokens,TestHSAdminServer.Skip semantics verified on
hadoop-azure,hadoop-awsandhadoop-client-integration-tests: with the fix,-DskipTestsskips both surefire and failsafe in all three; without it, the first and third ran integration tests.Notes on 3.6.0
3.6.0 removes the legacy
surefire-junit3/junit4/junit47/testngproviders. The tree has no JUnit 4 or TestNG tests and pins no provider, so this does not apply. Checked and unaffected: stack-trace handling (reimplemented upstream, but output is identical withtrimStackTrace=false),@Nestedselection,@Suiteflake classification,runOrder, and rerun handling under-Dsurefire.rerunFailingTestsCount=2.Three reporting changes, none needing infrastructure work:
<testsuite>gains aflakesattribute.@AfterAll/@AfterClassis reported as<class>.executionError; on 3.5.3 it produced an empty testcase name.hadoop-common's skip count by 40 (two OpenSSL codec classes).Yetus is unaffected: its
junit.shtakes failed-test names from the report filename and greps<failure|<errorat file level, so it reads none of these.dorny/test-reporterinreport_cloud_aws.ymldoes publish skipped counts, which will rise.For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?AI Tooling
If an AI tool was used:
Contains content generated by Claude Code.