HADOOP-19236 Incorporate VolcanoEngine Cloud TOS File System Implementation. (Part 2: Add unit tests) #7545

wojiaodoubao · 2025-03-27T13:01:11Z

Description of PR

HADOOP-19236 Incorporate VolcanoEngine Cloud TOS File System Implementation.

The full patch is too big to run ci, so it is split into 2 patches. This is the second one, containing the unit tests.

Please see #7194 for more details.
HADOOP-19236

How was this patch tested?

Unit tests and hadoop cluster.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-19236. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2025-03-27T15:06:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 57 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 53s		trunk passed
+1 💚	compile	0m 30s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 27s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 28s		trunk passed
+1 💚	mvnsite	0m 38s		trunk passed
+1 💚	javadoc	0m 36s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 29s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 52s		trunk passed
+1 💚	shadedclient	37m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 23s		the patch passed
+1 💚	compile	0m 23s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 23s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 16s		the patch passed
+1 💚	mvnsite	0m 25s		the patch passed
+1 💚	javadoc	0m 22s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 22s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 55s		the patch passed
+1 💚	shadedclient	37m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 26s	/patch-unit-hadoop-cloud-storage-project_hadoop-tos.txt	hadoop-tos in the patch passed.
-1 ❌	asflicense	0m 38s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		124m 16s

Reason	Tests
Failed junit tests	hadoop.fs.tosfs.TestTosFileSystem
	hadoop.fs.tosfs.object.tos.TestTOSRetryPolicy
	hadoop.fs.tosfs.commit.TestCommitter
	hadoop.fs.tosfs.commit.mapred.TestCommitter
	hadoop.fs.tosfs.TestRawFileSystem
	hadoop.fs.tosfs.object.TestDirectoryStorage

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/1/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux c7fba9ced968 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e392fad`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/1/testReport/
Max. process+thread count	545 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-03-28T04:38:38Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 57 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 42s		trunk passed
+1 💚	compile	0m 29s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 28s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 30s		trunk passed
+1 💚	mvnsite	0m 33s		trunk passed
+1 💚	javadoc	0m 37s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 30s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 53s		trunk passed
+1 💚	shadedclient	33m 54s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 23s		the patch passed
+1 💚	compile	0m 22s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 22s		the patch passed
+1 💚	compile	0m 20s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 25s		the patch passed
+1 💚	javadoc	0m 23s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 22s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 50s		the patch passed
+1 💚	shadedclient	33m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 23s	/patch-unit-hadoop-cloud-storage-project_hadoop-tos.txt	hadoop-tos in the patch passed.
+1 💚	asflicense	0m 36s		The patch does not generate ASF License warnings.
		116m 1s

Reason	Tests
Failed junit tests	hadoop.fs.tosfs.commit.mapred.TestCommitter
	hadoop.fs.tosfs.TestRawFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/2/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 73e65800ac29 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b26beb4`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/2/testReport/
Max. process+thread count	700 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-03-28T08:36:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 57 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 57s		trunk passed
+1 💚	compile	0m 28s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	compile	0m 28s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	checkstyle	0m 27s		trunk passed
+1 💚	mvnsite	0m 33s		trunk passed
+1 💚	javadoc	0m 36s		trunk passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 30s		trunk passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 52s		trunk passed
+1 💚	shadedclient	34m 16s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s		the patch passed
+1 💚	compile	0m 22s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javac	0m 22s		the patch passed
+1 💚	compile	0m 22s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	javac	0m 22s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 24s		the patch passed
+1 💚	javadoc	0m 23s		the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04
+1 💚	javadoc	0m 21s		the patch passed with JDK Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
+1 💚	spotbugs	0m 51s		the patch passed
+1 💚	shadedclient	34m 13s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 21s		hadoop-tos in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		116m 16s

Subsystem	Report/Notes
Docker	ClientAPI=1.48 ServerAPI=1.48 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/3/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 4fa19b664053 5.15.0-130-generic #140-Ubuntu SMP Wed Dec 18 17:59:53 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `63fc8e8`
Default Java	Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06~~us1-0ubuntu1~~20.04-b06
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/3/testReport/
Max. process+thread count	553 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

wojiaodoubao · 2025-03-28T11:09:43Z

Hi @slfan1989 @Hexiaoqiao , this is the second part of HADOOP-19236, including all the unit tests. Could you help taking a look, thanks very much !

slfan1989 · 2025-03-31T02:01:47Z

@wojiaodoubao Thank you for inviting me to participate in this pr! I am currently working on upgrading hadoop-common from JUnit4 to JUnit5. It is expected that two or three related PRs will need to be merged before this PR can be merged.
I will complete some preparation work this week.

wojiaodoubao · 2025-03-31T02:12:15Z

I am currently working on upgrading hadoop-common from JUnit4 to JUnit5.

Nice to hear this, great work @slfan1989 !!! I'll wait, and update hadoop-tos to junit5 after hadoop-common is updated.

hadoop-yetus · 2025-08-24T08:39:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 16s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 59 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	38m 36s		trunk passed
+1 💚	compile	0m 30s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 31s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 30s		trunk passed
+1 💚	mvnsite	0m 34s		trunk passed
+1 💚	javadoc	0m 38s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 32s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 53s		trunk passed
+1 💚	shadedclient	35m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s		the patch passed
+1 💚	compile	0m 23s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 23s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 21s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 24s		the patch passed
+1 💚	javadoc	0m 24s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 22s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 52s		the patch passed
+1 💚	shadedclient	35m 17s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1m 28s	/patch-unit-hadoop-cloud-storage-project_hadoop-tos.txt	hadoop-tos in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		135m 2s

Reason	Tests
Failed junit tests	hadoop.fs.tosfs.object.tos.TestDelegationClientBuilder
	hadoop.fs.tosfs.TestTosChecksum

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/4/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux f35c5cebdb88 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ca2bc0f`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/4/testReport/
Max. process+thread count	763 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2025-08-24T12:15:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 59 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	38m 27s		trunk passed
+1 💚	compile	0m 31s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 28s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 23s		trunk passed
+1 💚	mvnsite	0m 28s		trunk passed
+1 💚	javadoc	0m 30s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 25s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 47s		trunk passed
+1 💚	shadedclient	36m 22s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 23s		the patch passed
+1 💚	compile	0m 21s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 21s		the patch passed
+1 💚	compile	0m 20s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 25s		the patch passed
+1 💚	javadoc	0m 23s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 22s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 51s		the patch passed
+1 💚	shadedclient	35m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 29s		hadoop-tos in the patch passed.
+1 💚	asflicense	0m 37s		The patch does not generate ASF License warnings.
		122m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/5/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 4fd95d76871b 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `ac15867`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/5/testReport/
Max. process+thread count	711 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

wojiaodoubao · 2025-08-24T13:08:18Z

Hi @slfan1989 , all tests have been refactored with junit5. Could you help take a look when you have time, thanks very much!

...orage-project/hadoop-tos/src/test/java/org/apache/hadoop/fs/tosfs/contract/TestUnbuffer.java

...project/hadoop-tos/src/test/java/org/apache/hadoop/fs/tosfs/object/TestDirectoryStorage.java

...rc/test/java/org/apache/hadoop/fs/tosfs/object/tos/auth/TestAbstractCredentialsProvider.java

slfan1989 · 2025-09-01T05:18:33Z

Hi @slfan1989 , all tests have been refactored with junit5. Could you help take a look when you have time, thanks very much!

@wojiaodoubao It looks good overall, just a few minor suggestions.

hadoop-yetus · 2025-09-04T04:55:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+0 🆗	xmllint	0m 1s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 59 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 18s		trunk passed
+1 💚	compile	0m 28s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 29s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 29s		trunk passed
+1 💚	mvnsite	0m 34s		trunk passed
+1 💚	javadoc	0m 38s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 32s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 53s		trunk passed
+1 💚	shadedclient	35m 34s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s		the patch passed
+1 💚	compile	0m 23s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 23s		the patch passed
+1 💚	compile	0m 22s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 22s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 24s		the patch passed
+1 💚	javadoc	0m 24s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 23s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 53s		the patch passed
+1 💚	shadedclient	35m 18s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 25s		hadoop-tos in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		123m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/6/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux f10462198a20 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `991cced`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/6/testReport/
Max. process+thread count	725 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

wojiaodoubao · 2025-09-04T05:58:40Z

Hi @slfan1989 , the pr is ready for review now, please help when you have time, thanks very much!

wojiaodoubao · 2025-09-11T14:44:29Z

hi @slfan1989 , could you help review when you have time, thanks very much!

slfan1989 · 2025-09-16T04:46:10Z

hi @slfan1989 , could you help review when you have time, thanks very much!

@wojiaodoubao We've removed the JUnit 4 dependency from the trunk branch. Could you please rebase this PR? If it compiles successfully, we’ll proceed with the merge.

…ntation (Part 2: Add unit tests.). Contributed by: ZhengHu, SunXin, XianyinXin, Rascal Wu, FangBo, Yuanzhihuan.

wojiaodoubao · 2025-09-19T05:29:14Z

hi @slfan1989 , could you help review when you have time, thanks very much!

@wojiaodoubao We've removed the JUnit 4 dependency from the trunk branch. Could you please rebase this PR? If it compiles successfully, we’ll proceed with the merge.

Thanks @slfan1989 , thanks your comments! I rebased and tested it on my local env, it ran well. Let me push it and pending ci.

hadoop-yetus · 2025-09-19T07:41:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 2s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 59 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	50m 38s		trunk passed
+1 💚	compile	0m 28s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 28s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	checkstyle	0m 29s		trunk passed
+1 💚	mvnsite	0m 34s		trunk passed
+1 💚	javadoc	0m 37s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 30s		trunk passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 54s		trunk passed
+1 💚	shadedclient	35m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 24s		the patch passed
+1 💚	compile	0m 23s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 23s		the patch passed
+1 💚	compile	0m 23s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	javac	0m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 25s		the patch passed
+1 💚	javadoc	0m 24s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 22s		the patch passed with JDK Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
+1 💚	spotbugs	0m 51s		the patch passed
+1 💚	shadedclient	35m 1s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 26s		hadoop-tos in the patch passed.
+1 💚	asflicense	0m 39s		The patch does not generate ASF License warnings.
		133m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/7/artifact/out/Dockerfile
GITHUB PR	#7545
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell detsecrets xmllint spotbugs checkstyle
uname	Linux 062370b4c994 5.15.0-143-generic #153-Ubuntu SMP Fri Jun 13 19:10:45 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `faa9ad6`
Default Java	Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-ga~~us1-0ubuntu1~~20.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/7/testReport/
Max. process+thread count	692 (vs. ulimit of 5500)
modules	C: hadoop-cloud-storage-project/hadoop-tos U: hadoop-cloud-storage-project/hadoop-tos
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7545/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

slfan1989 · 2025-09-23T14:57:12Z

@wojiaodoubao Thanks for the contribution! Merged into trunk.

github-actions bot added the trunk label Mar 27, 2025

wojiaodoubao force-pushed the HADOOP-19236-02 branch from e392fad to b26beb4 Compare March 28, 2025 02:41

wojiaodoubao force-pushed the HADOOP-19236-02 branch from b26beb4 to 63fc8e8 Compare March 28, 2025 06:38

wojiaodoubao force-pushed the HADOOP-19236-02 branch from 63fc8e8 to ca2bc0f Compare August 24, 2025 06:23

github-actions bot added the build label Aug 24, 2025

wojiaodoubao force-pushed the HADOOP-19236-02 branch from ca2bc0f to ac15867 Compare August 24, 2025 10:11

wojiaodoubao mentioned this pull request Aug 27, 2025

HADOOP-19236. Integration of Volcano Engine TOS in Hadoop. #7194

Open

4 tasks

slfan1989 reviewed Sep 1, 2025

View reviewed changes

...orage-project/hadoop-tos/src/test/java/org/apache/hadoop/fs/tosfs/contract/TestUnbuffer.java Outdated Show resolved Hide resolved

slfan1989 reviewed Sep 1, 2025

View reviewed changes

...project/hadoop-tos/src/test/java/org/apache/hadoop/fs/tosfs/object/TestDirectoryStorage.java Outdated Show resolved Hide resolved

slfan1989 reviewed Sep 1, 2025

View reviewed changes

...rc/test/java/org/apache/hadoop/fs/tosfs/object/tos/auth/TestAbstractCredentialsProvider.java Outdated Show resolved Hide resolved

slfan1989 approved these changes Sep 16, 2025

View reviewed changes

wojiaodoubao and others added 3 commits September 18, 2025 20:12

HADOOP-19236. Incorporate VolcanoEngine Cloud TOS File System Impleme…

fee829a

…ntation (Part 2: Add unit tests.). Contributed by: ZhengHu, SunXin, XianyinXin, Rascal Wu, FangBo, Yuanzhihuan.

update to junit5

3cbd74c

follow comments

faa9ad6

wojiaodoubao force-pushed the HADOOP-19236-02 branch from 991cced to faa9ad6 Compare September 19, 2025 05:27

wojiaodoubao self-assigned this Sep 19, 2025

slfan1989 merged commit af8375c into trunk Sep 23, 2025
4 checks passed

HADOOP-19236 Incorporate VolcanoEngine Cloud TOS File System Implementation. (Part 2: Add unit tests) #7545

HADOOP-19236 Incorporate VolcanoEngine Cloud TOS File System Implementation. (Part 2: Add unit tests) #7545

Uh oh!

Conversation

wojiaodoubao commented Mar 27, 2025 • edited by slfan1989 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Mar 27, 2025

Uh oh!

hadoop-yetus commented Mar 28, 2025

Uh oh!

hadoop-yetus commented Mar 28, 2025

Uh oh!

wojiaodoubao commented Mar 28, 2025

Uh oh!

slfan1989 commented Mar 31, 2025

Uh oh!

wojiaodoubao commented Mar 31, 2025

Uh oh!

hadoop-yetus commented Aug 24, 2025

Uh oh!

hadoop-yetus commented Aug 24, 2025

Uh oh!

wojiaodoubao commented Aug 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slfan1989 commented Sep 1, 2025

Uh oh!

hadoop-yetus commented Sep 4, 2025

Uh oh!

wojiaodoubao commented Sep 4, 2025

Uh oh!

wojiaodoubao commented Sep 11, 2025

Uh oh!

slfan1989 commented Sep 16, 2025

Uh oh!

wojiaodoubao commented Sep 19, 2025

Uh oh!

hadoop-yetus commented Sep 19, 2025

Uh oh!

Uh oh!

slfan1989 commented Sep 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

wojiaodoubao commented Mar 27, 2025 •

edited by slfan1989

Loading