HADOOP-19666. Add hardware-accelerated CRC32 support for riscv64 using the v,zbc,zvbc extension

[leiwen2025]

This PR implements vector-accelerated CRC32 using the RISC-V V, Zbc and Zvbc
instruction sets, with full functional verification and performance testing completed.

The implementation uses the vclmul.v and vclmulh.v (carry-less multiply) instructions for data folding and computes the final checksum via Barrett reduction.

Key Features:

1. Runtime Hardware Detection
   The PR uses kernel hardware probing and cpuinfo parsing to dynamically detect hardware support for CRC32 acceleration (via v, zbc, and zvbc extensions) at runtime.

2. Performance Improvement
   Hardware-accelerated CRC32 achieves a performance boost of over 3X compared to the software implementation.

Description of PR

How was this patch tested?

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	13m 5s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	26m 30s		trunk passed
+1 💚	compile	7m 30s		trunk passed
-1 ❌	mvnsite	1m 29s	/branch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in trunk failed.
+1 💚	shadedclient	56m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 39s		the patch passed
+1 💚	compile	7m 3s		the patch passed
+1 💚	cc	7m 3s		the patch passed
+1 💚	golang	7m 3s		the patch passed
+1 💚	javac	7m 3s		the patch passed
-1 ❌	blanks	0m 0s	/blanks-eol.txt	The patch has 20 line(s) that end in blanks. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
-1 ❌	blanks	0m 0s	/blanks-tabs.txt	The patch 2 line(s) with tabs.
-1 ❌	mvnsite	1m 29s	/patch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
+1 💚	shadedclient	22m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 44s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 20s		The patch does not generate ASF License warnings.
		123m 6s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7912/1/artifact/out/Dockerfile
GITHUB PR	#7912
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname	Linux 3c797fab6900 5.15.0-142-generic #152-Ubuntu SMP Mon May 19 10:54:31 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 67c832e
Default Java	Red Hat, Inc.-1.8.0_312-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7912/1/testReport/
Max. process+thread count	1262 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7912/1/console
versions	git=2.27.0 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+1 💚	mvninstall	26m 26s		trunk passed
+1 💚	compile	7m 37s		trunk passed
-1 ❌	mvnsite	1m 30s	/branch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in trunk failed.
+1 💚	shadedclient	57m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 44s		the patch passed
+1 💚	compile	6m 57s		the patch passed
+1 💚	cc	6m 57s		the patch passed
+1 💚	golang	6m 57s		the patch passed
+1 💚	javac	6m 57s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-1 ❌	mvnsite	1m 29s	/patch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
+1 💚	shadedclient	23m 35s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 38s		hadoop-common in the patch passed.
+1 💚	asflicense	1m 20s		The patch does not generate ASF License warnings.
		111m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7912/2/artifact/out/Dockerfile
GITHUB PR	#7912
Optional Tests	dupname asflicense compile cc mvnsite javac unit codespell detsecrets golang
uname	Linux 9af82b84cde2 5.15.0-142-generic #152-Ubuntu SMP Mon May 19 10:54:31 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / be2c9c5
Default Java	Red Hat, Inc.-1.8.0_312-b07
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7912/2/testReport/
Max. process+thread count	3150 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-7912/2/console
versions	git=2.27.0 maven=3.6.3
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[leiwen2025]

@steveloughran @ayushtkn Hi, this PR has been open for a while. Could you please take a look when you have time? Thanks!

[steveloughran]

It's been a long time since I did C code, I'll have to stare at this a while.

In #7903 that creation of the file bulk_crc32_riscv.c should be what the new code goes into; work with @PeterPtroc to get something you are both happy with and tested.

Having two people who are set up to build and test this on riscv hardware is exactly what we need to get this done