Skip to content

HBASE-25634 The client frequently exceeds the quota, which causes the… #3020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 1 commit into from
Closed

HBASE-25634 The client frequently exceeds the quota, which causes the… #3020

wants to merge 1 commit into from

Conversation

@SiCheng-Zheng
Copy link
Contributor

… meta table scan to be too high

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 52s Docker mode activated.
-0 ⚠️ yetus 0m 7s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.3 Compile Tests _
+1 💚 mvninstall 3m 45s branch-2.3 passed
+1 💚 compile 0m 31s branch-2.3 passed
+1 💚 shadedjars 5m 13s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 25s branch-2.3 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 29s the patch passed
+1 💚 compile 0m 28s the patch passed
+1 💚 javac 0m 28s the patch passed
+1 💚 shadedjars 5m 2s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 25s the patch passed
_ Other Tests _
+1 💚 unit 2m 20s hbase-client in the patch passed.
24m 52s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR #3020
Optional Tests javac javadoc unit shadedjars compile
uname Linux c21ac1624fde 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.3 / c05241d
Default Java AdoptOpenJDK-1.8.0_282-b08
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/testReport/
Max. process+thread count 338 (vs. ulimit of 12500)
modules C: hbase-client U: hbase-client
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 0m 31s Docker mode activated.
-0 ⚠️ yetus 0m 7s Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
_ Prechecks _
_ branch-2.3 Compile Tests _
+1 💚 mvninstall 4m 10s branch-2.3 passed
+1 💚 compile 0m 30s branch-2.3 passed
+1 💚 shadedjars 6m 9s branch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 30s branch-2.3 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 59s the patch passed
+1 💚 compile 0m 29s the patch passed
+1 💚 javac 0m 29s the patch passed
+1 💚 shadedjars 6m 2s patch has no errors when building our shaded downstream artifacts.
+1 💚 javadoc 0m 29s the patch passed
_ Other Tests _
+1 💚 unit 2m 26s hbase-client in the patch passed.
26m 39s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR #3020
Optional Tests javac javadoc unit shadedjars compile
uname Linux 615f7ddde838 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.3 / c05241d
Default Java AdoptOpenJDK-11.0.10+9
Test Results https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/testReport/
Max. process+thread count 290 (vs. ulimit of 12500)
modules C: hbase-client U: hbase-client
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/console
versions git=2.17.1 maven=3.6.3
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@Apache-HBase
Copy link

🎊 +1 overall

Vote Subsystem Runtime Comment
+0 🆗 reexec 1m 49s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 hbaseanti 0m 0s Patch does not have any anti-patterns.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
_ branch-2.3 Compile Tests _
+1 💚 mvninstall 3m 46s branch-2.3 passed
+1 💚 compile 1m 11s branch-2.3 passed
+1 💚 checkstyle 0m 34s branch-2.3 passed
+1 💚 spotbugs 1m 9s branch-2.3 passed
_ Patch Compile Tests _
+1 💚 mvninstall 3m 25s the patch passed
+1 💚 compile 1m 1s the patch passed
+1 💚 javac 1m 1s the patch passed
-0 ⚠️ checkstyle 0m 32s hbase-client: The patch generated 2 new + 5 unchanged - 0 fixed = 7 total (was 5)
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 hadoopcheck 17m 19s Patch does not cause any errors with Hadoop 2.10.0 or 3.1.2 3.2.1.
+1 💚 spotbugs 1m 14s the patch passed
_ Other Tests _
+1 💚 asflicense 0m 15s The patch does not generate ASF License warnings.
39m 38s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR #3020
Optional Tests dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname Linux 354be866ec2b 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/hbase-personality.sh
git revision branch-2.3 / c05241d
Default Java AdoptOpenJDK-1.8.0_282-b08
checkstyle https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-client.txt
Max. process+thread count 96 (vs. ulimit of 12500)
modules C: hbase-client U: hbase-client
Console output https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3020/1/console
versions git=2.17.1 maven=3.6.3 spotbugs=3.1.12
Powered by Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

@SiCheng-Zheng
Copy link
Contributor Author

SiCheng-Zheng commented Mar 5, 2021 •
edited
Loading

@jatsakthi review code thank you!

@Apache9
Copy link
Contributor

Apache9 commented Mar 5, 2021

This is a problem for branch-2.3 only?

The fix is in RpcRetryingCallerImpl, which has already been removed on master not sure if it is still a problem on master, but it should also be a problem for branch-2 I guess?

In HBase, usually we will open issue on dev branches first, i.e, master or branch-2, unless the problem is only for a specific branch.

Thanks.

@SiCheng-Zheng
Copy link
Contributor Author

SiCheng-Zheng commented Mar 8, 2021 •
edited
Loading

@Apache9 Thank you review code！
Branch-2 will have this problem, when the server frequently returns “msg=org.apache.hadoop.hbase.quotas.RpcThrottlingException: org.apache.hadoop.hbase.quotas.RpcThrottlingException: ClientTest:CPUTEST2-request size limit exceeded-wait 999hrs , 0sec in HBASE1.hadoop,16020,1614698288842”.

The client scan operation will frequently request the meta table, causing the meta table to become high.
To branch-2 new submission request. url
Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@SiCheng-Zheng @Apache-HBase @Apache9