HADOOP-19528. Update Surefire plugin to 3.5.3

[stoty]

Description of PR

Hadoop currently uses the old 3.0.0-M2 surefire version, which swallows all errors in the test class setup @BeforeAll methods.
This leads to some tests not being run and not being reported as failing/erroring.
Update to the latest version, which fixes this issue.

Note that this may well cause new test failures, which were previously unreported, but that's not a problem with this patch.

How was this patch tested?

Ran it on JDK24, where some previously silently failing test properly errored out.

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 21s		trunk passed
+1 💚	compile	0m 22s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 22s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	0m 24s		trunk passed
+1 💚	javadoc	0m 26s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	81m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 13s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	0m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 17s		the patch passed
+1 💚	javadoc	0m 15s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
-1 ❌	shadedclient	2m 31s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 14s		hadoop-project in the patch passed.
+1 💚	asflicense	0m 28s		The patch does not generate ASF License warnings.
		87m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/1/artifact/out/Dockerfile
GITHUB PR	#7574
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux 9c5f5fcc3e4c 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 651b504
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/1/testReport/
Max. process+thread count	525 (vs. ulimit of 5500)
modules	C: hadoop-project U: hadoop-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 52s		trunk passed
+1 💚	compile	0m 21s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 21s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	0m 25s		trunk passed
+1 💚	javadoc	0m 26s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	81m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 13s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	0m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 16s		the patch passed
+1 💚	javadoc	0m 15s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
-1 ❌	shadedclient	2m 31s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 13s		hadoop-project in the patch passed.
+1 💚	asflicense	0m 29s		The patch does not generate ASF License warnings.
		87m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/2/artifact/out/Dockerfile
GITHUB PR	#7574
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint
uname	Linux 6a50f92a40a8 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 651b504
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/2/testReport/
Max. process+thread count	535 (vs. ulimit of 5500)
modules	C: hadoop-project U: hadoop-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/3/console in case of problems.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	41m 8s		trunk passed
+1 💚	compile	0m 23s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 21s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	0m 26s		trunk passed
+1 💚	javadoc	0m 27s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	38m 56s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 13s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	0m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 17s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	javadoc	0m 15s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
-1 ❌	shadedclient	7m 7s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 16s		hadoop-project in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		93m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/3/artifact/out/Dockerfile
GITHUB PR	#7574
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint
uname	Linux 83a39637713d 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 1b24347
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/3/testReport/
Max. process+thread count	609 (vs. ulimit of 5500)
modules	C: hadoop-project U: hadoop-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/3/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/4/console in case of problems.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	42m 7s		trunk passed
+1 💚	compile	0m 22s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 21s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	mvnsite	0m 26s		trunk passed
+1 💚	javadoc	0m 27s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 22s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	39m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 14s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 13s		the patch passed
+1 💚	compile	0m 13s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	javac	0m 13s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 17s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	javadoc	0m 15s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
+1 💚	shadedclient	39m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 17s		hadoop-project in the patch passed.
+1 💚	asflicense	0m 36s		The patch does not generate ASF License warnings.
		126m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/4/artifact/out/Dockerfile
GITHUB PR	#7574
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient xmllint
uname	Linux 153a01e57bb9 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 3145aa8
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu120.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/4/testReport/
Max. process+thread count	534 (vs. ulimit of 5500)
modules	C: hadoop-project U: hadoop-project
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7574/4/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[slfan1989]

@stoty Thank you very much for your contribution! However, I believe that upgrading the versions of the Surefire plugin and JUnit may not be possible for the time being. I had planned to upgrade the Surefire plugin twice, but after each upgrade, some unit tests failed to execute, so the plugin upgrade has been temporarily put on hold. As for JUnit, it is still in a mixed state (with JUnit 4 and JUnit 5 being used together) within Hadoop. I hope to consider upgrading to a major version once the transition to JUnit 5 is complete.

[stoty]

Thanks for looking into this, @slfan1989 .

I'm still looking into some related failures, but so far the failing tests are not failing because 3.5.3 is causing problems, but because the tests fail (in some cases time out) silently with the 3.0.0M4 , so it's not that 3.5.3 causes the tests failures, but rather 3.0.0-M4 does not correctly report them.

I'm going to open/link tickets for these issues as I identify them.