HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed #4611

Apache9 · 2022-07-10T14:51:07Z

No description provided.

Apache-HBase · 2022-07-10T15:34:38Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 10s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 40s	master passed
+1 💚	compile	2m 55s	master passed
+1 💚	checkstyle	0m 41s	master passed
+1 💚	spotless	1m 1s	branch has no errors when running spotless:check.
+1 💚	spotbugs	1m 48s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 4s	the patch passed
+1 💚	compile	2m 59s	the patch passed
+1 💚	javac	2m 59s	the patch passed
+1 💚	checkstyle	0m 38s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	12m 46s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotless	0m 43s	patch has no errors when running spotless:check.
+1 💚	spotbugs	1m 22s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		39m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4611
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux afc5c6bd765c 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2197b38`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	69 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-07-10T18:31:55Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 39s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 26s	master passed
+1 💚	compile	0m 34s	master passed
+1 💚	shadedjars	3m 54s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 23s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 6s	the patch passed
+1 💚	compile	0m 34s	the patch passed
+1 💚	javac	0m 34s	the patch passed
+1 💚	shadedjars	3m 55s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 21s	the patch passed
		_ Other Tests _
-1 ❌	unit	200m 27s	hbase-server in the patch failed.
		216m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4611
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 24bdc2f2d59a 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2197b38`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/testReport/
Max. process+thread count	2715 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-07-10T19:04:47Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 32s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 33s	master passed
+1 💚	compile	0m 56s	master passed
+1 💚	shadedjars	4m 39s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 31s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 59s	the patch passed
+1 💚	compile	0m 48s	the patch passed
+1 💚	javac	0m 48s	the patch passed
+1 💚	shadedjars	4m 24s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	the patch passed
		_ Other Tests _
-1 ❌	unit	227m 56s	hbase-server in the patch failed.
		249m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4611
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1ccc3f566849 5.4.0-1043-aws #45~18.04.1-Ubuntu SMP Fri Apr 9 23:32:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2197b38`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/testReport/
Max. process+thread count	2619 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…netty channel is closed

Apache9 · 2022-07-11T04:05:21Z

Find a simple way to fix the problem first.

So let's merge this first to unblock the new releases first.

Will work on HBASE-27185 to make the code more stable and clean soon.

@apurtell @virajjasani PTAL.

Thanks.

Apache-HBase · 2022-07-11T04:56:41Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 23s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 9s	master passed
+1 💚	compile	3m 16s	master passed
+1 💚	checkstyle	0m 45s	master passed
+1 💚	spotless	0m 59s	branch has no errors when running spotless:check.
+1 💚	spotbugs	2m 0s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 37s	the patch passed
+1 💚	compile	3m 22s	the patch passed
+1 💚	javac	3m 22s	the patch passed
+1 💚	checkstyle	0m 40s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	16m 13s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotless	1m 4s	patch has no errors when running spotless:check.
+1 💚	spotbugs	2m 4s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 12s	The patch does not generate ASF License warnings.
		46m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#4611
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti spotless checkstyle compile
uname	Linux ee826745b6a0 5.4.0-1025-aws #25~18.04.1-Ubuntu SMP Fri Sep 11 12:03:04 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2197b38`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	65 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-07-11T07:47:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 38s	Docker mode activated.
-0 ⚠️	yetus	0m 2s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 10s	master passed
+1 💚	compile	0m 33s	master passed
+1 💚	shadedjars	3m 54s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 22s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 7s	the patch passed
+1 💚	compile	0m 33s	the patch passed
+1 💚	javac	0m 33s	the patch passed
+1 💚	shadedjars	3m 56s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 20s	the patch passed
		_ Other Tests _
+1 💚	unit	201m 55s	hbase-server in the patch passed.
		217m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#4611
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 3ebf6a07749e 5.4.0-1071-aws #76~18.04.1-Ubuntu SMP Mon Mar 28 17:49:57 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2197b38`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/testReport/
Max. process+thread count	2546 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2022-07-11T07:53:20Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 16s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	2m 48s	master passed
+1 💚	compile	0m 46s	master passed
+1 💚	shadedjars	3m 44s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 29s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	2m 39s	the patch passed
+1 💚	compile	0m 46s	the patch passed
+1 💚	javac	0m 46s	the patch passed
+1 💚	shadedjars	3m 47s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 26s	the patch passed
		_ Other Tests _
+1 💚	unit	205m 7s	hbase-server in the patch passed.
		223m 9s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#4611
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 792b1c36c47e 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `2197b38`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/testReport/
Max. process+thread count	2556 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-4611/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

virajjasani · 2022-07-11T20:03:43Z

hbase-server/src/main/java/org/apache/hadoop/hbase/ipc/NettyServerRpcConnection.java

-    disposeSasl();
    channel.close();
-    callCleanupIfNeeded();


Hmm not calling this is also a problem, but that's fine we can track that later.

We register close hook, so close the channel will trigger the close hook and call these two methods.

No I didn't mean call to disposeSasl() and callCleanupIfNeeded(). I meant this whole NettyServerRpcConnection#close not getting called is a problem.

Having the channel close hook to dispose sasl is the right thing to do but I think we can simplify this a bit and maybe remove close() from NettyServerRpcConnection or do some better refactoring so that first time readers don't get confused and add more closable stuffs in NettyServerRpcConnection#close.

virajjasani · 2022-07-11T20:06:16Z

Thanks for the quick turnaround on this @Apache9 !

…netty channel is closed (#4611) Signed-off-by: Viraj Jasani <[email protected]> (cherry picked from commit 5bc8670)

…netty channel is closed (apache#4611) Signed-off-by: Viraj Jasani <[email protected]> (cherry picked from commit 5bc8670) (cherry picked from commit d054c93) Change-Id: I61f8eb8f01bde5ec8576e58922bab2bdbd6e7079

Apache9 force-pushed the HBASE-27189 branch from 91efd3f to 48462ec Compare July 10, 2022 14:53

HBASE-27189 NettyServerRpcConnection is not properly closed when the …

2192d45

…netty channel is closed

Apache9 force-pushed the HBASE-27189 branch from 48462ec to 2192d45 Compare July 11, 2022 03:58

Apache9 changed the title ~~HBASE-27189 UT to reproduce the problem~~ HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed Jul 11, 2022

Apache9 requested review from apurtell and virajjasani July 11, 2022 04:03

virajjasani approved these changes Jul 11, 2022

View reviewed changes

Apache9 merged commit 5bc8670 into apache:master Jul 12, 2022

Apache9 added a commit that referenced this pull request Jul 12, 2022

HBASE-27189 NettyServerRpcConnection is not properly closed when the …

d054c93

…netty channel is closed (#4611) Signed-off-by: Viraj Jasani <[email protected]> (cherry picked from commit 5bc8670)

Apache9 added a commit that referenced this pull request Jul 12, 2022

HBASE-27189 NettyServerRpcConnection is not properly closed when the …

c39025f

…netty channel is closed (#4611) Signed-off-by: Viraj Jasani <[email protected]> (cherry picked from commit 5bc8670)

Apache9 added a commit that referenced this pull request Jul 12, 2022

HBASE-27189 NettyServerRpcConnection is not properly closed when the …

68f539e

…netty channel is closed (#4611) Signed-off-by: Viraj Jasani <[email protected]> (cherry picked from commit 5bc8670)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed #4611

HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed #4611

Uh oh!

Apache9 commented Jul 10, 2022

Uh oh!

Apache-HBase commented Jul 10, 2022

Uh oh!

Apache-HBase commented Jul 10, 2022

Uh oh!

Apache-HBase commented Jul 10, 2022

Uh oh!

Apache9 commented Jul 11, 2022

Uh oh!

Apache-HBase commented Jul 11, 2022

Uh oh!

Apache-HBase commented Jul 11, 2022

Uh oh!

Apache-HBase commented Jul 11, 2022

Uh oh!

virajjasani Jul 11, 2022

Uh oh!

Apache9 Jul 12, 2022

Uh oh!

virajjasani Jul 12, 2022

Uh oh!

virajjasani commented Jul 11, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed #4611

HBASE-27189 NettyServerRpcConnection is not properly closed when the netty channel is closed #4611

Uh oh!

Conversation

Apache9 commented Jul 10, 2022

Uh oh!

Apache-HBase commented Jul 10, 2022

Uh oh!

Apache-HBase commented Jul 10, 2022

Uh oh!

Apache-HBase commented Jul 10, 2022

Uh oh!

Apache9 commented Jul 11, 2022

Uh oh!

Apache-HBase commented Jul 11, 2022

Uh oh!

Apache-HBase commented Jul 11, 2022

Uh oh!

Apache-HBase commented Jul 11, 2022

Uh oh!

virajjasani Jul 11, 2022

Choose a reason for hiding this comment

Uh oh!

Apache9 Jul 12, 2022

Choose a reason for hiding this comment

Uh oh!

virajjasani Jul 12, 2022

Choose a reason for hiding this comment

Uh oh!

virajjasani commented Jul 11, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants