HBASE-28970 Get asyncfs working with custom SASL mechanisms #6507

stoty · 2024-12-02T21:27:30Z

No description provided.

stoty · 2024-12-02T21:27:57Z

This should be good, but I'm waiting for the Hadoop side fixes before landing this one.

Apache-HBase · 2024-12-03T09:22:48Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 41s		Docker mode activated.
-0 ⚠️	yetus	0m 4s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	2m 34s		master passed
+1 💚	compile	0m 15s		master passed
+1 💚	javadoc	0m 13s		master passed
+1 💚	shadedjars	4m 52s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 38s		the patch passed
+1 💚	compile	0m 17s		the patch passed
+1 💚	javac	0m 17s		the patch passed
+1 💚	javadoc	0m 13s		the patch passed
+1 💚	shadedjars	4m 49s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	1m 21s		hbase-asyncfs in the patch passed.
		18m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6507/3/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6507
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux e2c3549551f0 5.4.0-195-generic #215-Ubuntu SMP Fri Aug 2 18:28:05 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `56a7d83`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6507/3/testReport/
Max. process+thread count	652 (vs. ulimit of 30000)
modules	C: hbase-asyncfs U: hbase-asyncfs
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6507/3/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2024-12-03T09:30:53Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	2m 51s		master passed
+1 💚	compile	0m 19s		master passed
+1 💚	checkstyle	0m 7s		master passed
+1 💚	spotbugs	0m 21s		master passed
+1 💚	spotless	0m 41s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 50s		the patch passed
+1 💚	compile	0m 19s		the patch passed
+1 💚	javac	0m 19s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 8s		the patch passed
+1 💚	spotbugs	0m 29s		the patch passed
+1 💚	hadoopcheck	10m 40s		Patch does not cause any errors with Hadoop 3.3.6 3.4.0.
+1 💚	spotless	0m 41s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 8s		The patch does not generate ASF License warnings.
		26m 45s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6507/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6507
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux dffd481fbb01 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `56a7d83`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	83 (vs. ulimit of 30000)
modules	C: hbase-asyncfs U: hbase-asyncfs
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6507/3/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

stoty · 2024-12-09T14:44:40Z

The standard HBase SASL code already works like this.

anmolnar

Quick question.

...src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java

Apache9

Do you have any real use cases?

IIRC we have done the same fix in hbase's own rpc implementation, but I can not recall the details...

stoty · 2025-01-09T09:24:13Z

I'm not sure about the HBase implementation, but I plan to review it for this issue soon.

As for the real world use case, this has come up when running Hadoop+HBase with the SCRAM mechanism lifted from Kafka.
(The patches necessary for working with SCRAM are only present in Hadoop HEAD)

stoty

Forgot to submit my pending responses to Andor

...src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java

Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 9036d88)

)

HBASE-28970 Get asyncfs working with custom SASL mechanisms

8af0310

stoty marked this pull request as draft December 2, 2024 21:27

This comment has been minimized.

Sign in to view

stoty added 2 commits December 3, 2024 09:26

apply spotless

2969515

minimal optimization

56a7d83

This comment has been minimized.

Sign in to view

stoty marked this pull request as ready for review December 9, 2024 14:38

stoty requested review from Apache9 and jojochuang December 9, 2024 14:41

anmolnar reviewed Dec 9, 2024

View reviewed changes

...src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java Show resolved Hide resolved

Apache9 approved these changes Dec 16, 2024

View reviewed changes

stoty commented Jan 9, 2025

View reviewed changes

...src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java Show resolved Hide resolved

...src/main/java/org/apache/hadoop/hbase/io/asyncfs/FanOutOneBlockAsyncDFSOutputSaslHelper.java Show resolved Hide resolved

stoty merged commit 9036d88 into apache:master Jan 9, 2025
1 check passed

stoty added a commit that referenced this pull request Jan 9, 2025

HBASE-28970 Get asyncfs working with custom SASL mechanisms (#6507)

13c5beb

Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 9036d88)

stoty added a commit to stoty/hbase that referenced this pull request Jan 9, 2025

HBASE-28970 Get asyncfs working with custom SASL mechanisms (apache#6507

5a8f23a

)

ragarkar pushed a commit to ragarkar/hbase that referenced this pull request Jan 13, 2025

HBASE-28970 Get asyncfs working with custom SASL mechanisms (apache#6507

5a21f16

)

HBASE-28970 Get asyncfs working with custom SASL mechanisms #6507

HBASE-28970 Get asyncfs working with custom SASL mechanisms #6507

Uh oh!

Conversation

stoty commented Dec 2, 2024

Uh oh!

stoty commented Dec 2, 2024

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Dec 3, 2024

Uh oh!

Apache-HBase commented Dec 3, 2024

Uh oh!

stoty commented Dec 9, 2024

Uh oh!

anmolnar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

stoty commented Jan 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stoty left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

stoty commented Jan 9, 2025 •

edited

Loading