HBASE-26479: Print too slow/big scan's operation_id in region server log #3870

liangxs · 2021-11-22T12:40:28Z

Tracing is very important in large-scale distributed systems.

The attribute _operation.attributes.id of a scan request can be used as trace id between hbase client and region server.

We should print operation id in region server's log if the scan request is too slow or too big.

It will be very helpful for finding problematic requests.

Apache-HBase · 2021-11-22T13:30:49Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 10s	master passed
+1 💚	compile	3m 17s	master passed
+1 💚	checkstyle	1m 4s	master passed
+1 💚	spotbugs	2m 5s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 52s	the patch passed
+1 💚	compile	3m 13s	the patch passed
+1 💚	javac	3m 13s	the patch passed
-0 ⚠️	checkstyle	1m 3s	hbase-server: The patch generated 1 new + 32 unchanged - 0 fixed = 33 total (was 32)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 36s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	2m 18s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		49m 29s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3870
JIRA Issue	HBASE-26479
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 8a8dcfd9ca18 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `04635f4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache9

Mind adding a simple UT for this?

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java

Apache-HBase · 2021-11-22T15:44:59Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 25s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 14s	master passed
+1 💚	compile	1m 2s	master passed
+1 💚	shadedjars	8m 18s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 53s	the patch passed
+1 💚	compile	1m 1s	the patch passed
+1 💚	javac	1m 1s	the patch passed
+1 💚	shadedjars	8m 17s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	153m 6s	hbase-server in the patch passed.
		183m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3870
JIRA Issue	HBASE-26479
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 067b5a2dbe59 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `04635f4`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/testReport/
Max. process+thread count	4821 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-23T04:19:03Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 7s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	8m 57s	master passed
+1 💚	compile	3m 30s	master passed
+1 💚	checkstyle	1m 48s	master passed
+1 💚	spotbugs	2m 19s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 14s	the patch passed
+1 💚	compile	3m 25s	the patch passed
+1 💚	javac	3m 25s	the patch passed
+1 💚	checkstyle	1m 11s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	22m 23s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	2m 23s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		60m 33s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3870
JIRA Issue	HBASE-26479
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 457fb5cafbd6 4.15.0-153-generic #160-Ubuntu SMP Thu Jul 29 06:54:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b870b6d`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-23T07:57:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 2s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	10m 3s	master passed
+1 💚	compile	1m 5s	master passed
+1 💚	shadedjars	9m 9s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 45s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 17s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	9m 0s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	the patch passed
		_ Other Tests _
+1 💚	unit	239m 35s	hbase-server in the patch passed.
		278m 30s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3870
JIRA Issue	HBASE-26479
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux b5d7c739b6ba 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b870b6d`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/testReport/
Max. process+thread count	2398 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-23T11:11:30Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 26s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 12s	master passed
+1 💚	compile	3m 11s	master passed
+1 💚	checkstyle	1m 5s	master passed
+1 💚	spotbugs	2m 6s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 49s	the patch passed
+1 💚	compile	3m 11s	the patch passed
+1 💚	javac	3m 11s	the patch passed
+1 💚	checkstyle	1m 3s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	19m 27s	Patch does not cause any errors with Hadoop 3.1.2 3.2.2 3.3.1.
+1 💚	spotbugs	2m 17s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		49m 8s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3870
JIRA Issue	HBASE-26479
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 7d3e7d1ed60d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b870b6d`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-11-23T14:37:20Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 35s	master passed
+1 💚	compile	1m 7s	master passed
+1 💚	shadedjars	9m 30s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 45s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 19s	the patch passed
+1 💚	compile	1m 6s	the patch passed
+1 💚	javac	1m 6s	the patch passed
+1 💚	shadedjars	9m 6s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	220m 41s	hbase-server in the patch passed.
		254m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3870
JIRA Issue	HBASE-26479
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux a607f20de5a7 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `b870b6d`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/testReport/
Max. process+thread count	3540 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3870/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

…og (#3870) Signed-off-by: Duo Zhang <[email protected]>

…server log (#3870)" This reverts commit b437a96.

…og (#3870) Signed-off-by: Duo Zhang <[email protected]>

HBASE-26479: Print too slow/big scan's operation_id in region server log

9139b81

Apache9 reviewed Nov 22, 2021

View reviewed changes

hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RSRpcServices.java Show resolved Hide resolved

Add unit test for HBASE-26479

0da793e

Apache9 approved these changes Nov 23, 2021

View reviewed changes

Apache9 merged commit c148d5b into apache:master Nov 24, 2021

Apache9 pushed a commit that referenced this pull request Nov 24, 2021

HBASE-26479 Print too slow/big scan's operation_id in region server l…

b437a96

…og (#3870) Signed-off-by: Duo Zhang <[email protected]>

asfgit pushed a commit that referenced this pull request Nov 24, 2021

Revert "HBASE-26479 Print too slow/big scan's operation_id in region …

52ce877

…server log (#3870)" This reverts commit b437a96.

Apache9 pushed a commit that referenced this pull request Nov 25, 2021

HBASE-26479 Print too slow/big scan's operation_id in region server l…

2798ef1

…og (#3870) Signed-off-by: Duo Zhang <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

HBASE-26479: Print too slow/big scan's operation_id in region server log #3870

HBASE-26479: Print too slow/big scan's operation_id in region server log #3870

Uh oh!

liangxs commented Nov 22, 2021

Uh oh!

Apache-HBase commented Nov 22, 2021

Uh oh!

Apache9 left a comment

Uh oh!

Uh oh!

Apache-HBase commented Nov 22, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

HBASE-26479: Print too slow/big scan's operation_id in region server log #3870

HBASE-26479: Print too slow/big scan's operation_id in region server log #3870

Uh oh!

Conversation

liangxs commented Nov 22, 2021

Uh oh!

Apache-HBase commented Nov 22, 2021

Uh oh!

Apache9 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Apache-HBase commented Nov 22, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Apache-HBase commented Nov 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants