HBASE-25739 TableSkewCostFunction need to use aggregated deviation #3067

clarax · 2021-03-19T03:55:30Z

…er way to evaluate resource distribution

clarax · 2021-03-19T03:56:07Z

This is the the fix that only update table skew cost function

Apache-HBase · 2021-03-19T04:42:00Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 49s	master passed
+1 💚	compile	3m 11s	master passed
+1 💚	checkstyle	1m 5s	master passed
+1 💚	spotbugs	2m 6s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 29s	the patch passed
+1 💚	compile	3m 9s	the patch passed
+1 💚	javac	3m 9s	the patch passed
-0 ⚠️	checkstyle	1m 4s	hbase-server: The patch generated 2 new + 84 unchanged - 17 fixed = 86 total (was 101)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	17m 21s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 14s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 16s	The patch does not generate ASF License warnings.
		45m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 636ec6478b6c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `585aca1`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-19T06:43:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 52s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 21s	master passed
+1 💚	compile	1m 5s	master passed
+1 💚	shadedjars	6m 47s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 42s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 4s	the patch passed
+1 💚	compile	1m 5s	the patch passed
+1 💚	javac	1m 5s	the patch passed
+1 💚	shadedjars	6m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	the patch passed
		_ Other Tests _
-1 ❌	unit	135m 50s	hbase-server in the patch failed.
		167m 22s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 96a75f5f8916 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `585aca1`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/testReport/
Max. process+thread count	3916 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-19T06:56:05Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	2m 20s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 50s	master passed
+1 💚	compile	0m 56s	master passed
+1 💚	shadedjars	6m 35s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 31s	the patch passed
+1 💚	compile	0m 58s	the patch passed
+1 💚	javac	0m 58s	the patch passed
+1 💚	shadedjars	6m 34s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 36s	the patch passed
		_ Other Tests _
-1 ❌	unit	151m 28s	hbase-server in the patch failed.
		179m 44s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux ae1e6effa9dd 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `585aca1`
Default Java	AdoptOpenJDK-1.8.0_282-b08
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/artifact/yetus-jdk8-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/testReport/
Max. process+thread count	4333 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/1/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-19T18:16:44Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 37s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	5m 29s	master passed
+1 💚	compile	3m 56s	master passed
+1 💚	checkstyle	1m 22s	master passed
+1 💚	spotbugs	2m 36s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 25s	the patch passed
+1 💚	compile	3m 55s	the patch passed
+1 💚	javac	3m 55s	the patch passed
-0 ⚠️	checkstyle	1m 31s	hbase-server: The patch generated 2 new + 84 unchanged - 17 fixed = 86 total (was 101)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	22m 57s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 41s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		61m 49s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 5d65d3b73c9f 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f405990`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-19T20:06:14Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 20s	master passed
+1 💚	compile	1m 6s	master passed
+1 💚	shadedjars	6m 48s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 4s	the patch passed
+1 💚	compile	1m 7s	the patch passed
+1 💚	javac	1m 7s	the patch passed
+1 💚	shadedjars	6m 48s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	the patch passed
		_ Other Tests _
-1 ❌	unit	142m 48s	hbase-server in the patch failed.
		171m 36s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux af4b0eeb9f17 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f405990`
Default Java	AdoptOpenJDK-11.0.10+9
unit	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/testReport/
Max. process+thread count	3906 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-19T21:20:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 47s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 10s	master passed
+1 💚	compile	0m 59s	master passed
+1 💚	shadedjars	7m 16s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 50s	the patch passed
+1 💚	compile	0m 59s	the patch passed
+1 💚	javac	0m 59s	the patch passed
+1 💚	shadedjars	7m 18s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 38s	the patch passed
		_ Other Tests _
+1 💚	unit	216m 32s	hbase-server in the patch passed.
		245m 47s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux e98a04f33ca7 4.15.0-126-generic #129-Ubuntu SMP Mon Nov 23 18:53:38 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f405990`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/testReport/
Max. process+thread count	3432 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/2/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-23T22:30:34Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 29s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 58s	master passed
+1 💚	compile	3m 12s	master passed
+1 💚	checkstyle	1m 6s	master passed
+1 💚	spotbugs	2m 6s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 36s	the patch passed
+1 💚	compile	3m 11s	the patch passed
+1 💚	javac	3m 11s	the patch passed
-0 ⚠️	checkstyle	1m 3s	hbase-server: The patch generated 2 new + 84 unchanged - 17 fixed = 86 total (was 101)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	18m 5s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 13s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		46m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 4e91af70deca 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `202b17f`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	95 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-24T00:32:55Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 35s	master passed
+1 💚	compile	1m 12s	master passed
+1 💚	shadedjars	7m 51s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 15s	the patch passed
+1 💚	compile	1m 11s	the patch passed
+1 💚	javac	1m 11s	the patch passed
+1 💚	shadedjars	7m 46s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 41s	the patch passed
		_ Other Tests _
+1 💚	unit	138m 38s	hbase-server in the patch passed.
		169m 18s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 1a071f82f35e 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `202b17f`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/testReport/
Max. process+thread count	4586 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-03-24T00:45:31Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 9s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 45s	master passed
+1 💚	compile	0m 59s	master passed
+1 💚	shadedjars	7m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 35s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 38s	the patch passed
+1 💚	compile	1m 0s	the patch passed
+1 💚	javac	1m 0s	the patch passed
+1 💚	shadedjars	7m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	the patch passed
		_ Other Tests _
+1 💚	unit	152m 40s	hbase-server in the patch passed.
		181m 50s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 25e1b948a5b7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `202b17f`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/testReport/
Max. process+thread count	4033 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/3/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

saintstack

Minor stuff to address else patch looks good.

hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java

hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java

Apache-HBase · 2021-04-06T22:50:23Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	4m 2s	master passed
+1 💚	compile	3m 12s	master passed
+1 💚	checkstyle	1m 6s	master passed
+1 💚	spotbugs	2m 6s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 37s	the patch passed
+1 💚	compile	3m 10s	the patch passed
+1 💚	javac	3m 10s	the patch passed
-0 ⚠️	checkstyle	1m 4s	hbase-server: The patch generated 1 new + 84 unchanged - 17 fixed = 85 total (was 101)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	18m 11s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 11s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 15s	The patch does not generate ASF License warnings.
		47m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux e3bcdda46afc 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `048ca4e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-04-07T00:49:51Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 11s	master passed
+1 💚	compile	1m 12s	master passed
+1 💚	shadedjars	7m 41s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 12s	the patch passed
+1 💚	compile	1m 11s	the patch passed
+1 💚	javac	1m 11s	the patch passed
+1 💚	shadedjars	7m 46s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 39s	the patch passed
		_ Other Tests _
+1 💚	unit	136m 39s	hbase-server in the patch passed.
		166m 40s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f440a0415345 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `048ca4e`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/testReport/
Max. process+thread count	4120 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-04-07T01:00:24Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 54s	master passed
+1 💚	compile	1m 1s	master passed
+1 💚	shadedjars	7m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 37s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	7m 40s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	148m 26s	hbase-server in the patch passed.
		177m 11s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 4315ee259dba 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `048ca4e`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/testReport/
Max. process+thread count	4154 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/4/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

saintstack

LGTM

Has this been tried in production @clarax ?

Leaving open another day or so in case @d-c-manning or @sunhelly have some input... Thanks.

saintstack · 2021-04-08T17:37:55Z

And how does this relate to #3011 ? Thanks.

clarax · 2021-04-08T18:44:28Z

LGTM

Has this been tried in production @clarax ?

Leaving open another day or so in case @d-c-manning or @sunhelly have some input... Thanks.

This was validated on a 12 node 110 region test cluster and a ~1000 node 400,000 region production cluster.

clarax · 2021-04-08T18:45:02Z

And how does this relate to #3011 ? Thanks.

It is a sub task of HBASE-25625: https://issues.apache.org/jira/browse/HBASE-25739?filter=-2

saintstack · 2021-04-08T18:52:20Z

Good. Thanks.

clarax · 2021-04-08T21:36:07Z

updated the commit message to clarify this is for subtask https://issues.apache.org/jira/browse/HBASE-25739?filter=-2

Apache-HBase · 2021-04-08T22:09:02Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 28s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	hbaseanti	0m 0s	Patch does not have any anti-patterns.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
		_ master Compile Tests _
+1 💚	mvninstall	3m 37s	master passed
+1 💚	compile	3m 14s	master passed
+1 💚	checkstyle	1m 5s	master passed
+1 💚	spotbugs	2m 4s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 36s	the patch passed
+1 💚	compile	3m 10s	the patch passed
+1 💚	javac	3m 10s	the patch passed
-0 ⚠️	checkstyle	1m 4s	hbase-server: The patch generated 1 new + 84 unchanged - 17 fixed = 85 total (was 101)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	hadoopcheck	17m 50s	Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0.
+1 💚	spotbugs	2m 15s	the patch passed
		_ Other Tests _
+1 💚	asflicense	0m 13s	The patch does not generate ASF License warnings.
		46m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile
uname	Linux 2f197963a33b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f9819f3`
Default Java	AdoptOpenJDK-1.8.0_282-b08
checkstyle	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/artifact/yetus-general-check/output/diff-checkstyle-hbase-server.txt
Max. process+thread count	96 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/console
versions	git=2.17.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-04-09T00:10:09Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 27s	Docker mode activated.
-0 ⚠️	yetus	0m 4s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	4m 25s	master passed
+1 💚	compile	1m 11s	master passed
+1 💚	shadedjars	7m 44s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 43s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	4m 11s	the patch passed
+1 💚	compile	1m 9s	the patch passed
+1 💚	javac	1m 9s	the patch passed
+1 💚	shadedjars	7m 39s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	the patch passed
		_ Other Tests _
+1 💚	unit	137m 1s	hbase-server in the patch passed.
		167m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux f9d9cd9471ff 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f9819f3`
Default Java	AdoptOpenJDK-11.0.10+9
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/testReport/
Max. process+thread count	4560 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2021-04-09T00:20:14Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 30s	Docker mode activated.
-0 ⚠️	yetus	0m 3s	Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck
		_ Prechecks _
		_ master Compile Tests _
+1 💚	mvninstall	3m 58s	master passed
+1 💚	compile	1m 0s	master passed
+1 💚	shadedjars	7m 40s	branch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 40s	master passed
		_ Patch Compile Tests _
+1 💚	mvninstall	3m 38s	the patch passed
+1 💚	compile	1m 3s	the patch passed
+1 💚	javac	1m 3s	the patch passed
+1 💚	shadedjars	7m 44s	patch has no errors when building our shaded downstream artifacts.
+1 💚	javadoc	0m 37s	the patch passed
		_ Other Tests _
+1 💚	unit	148m 27s	hbase-server in the patch passed.
		177m 32s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile
GITHUB PR	#3067
Optional Tests	javac javadoc unit shadedjars compile
uname	Linux 83db25d1360c 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `f9819f3`
Default Java	AdoptOpenJDK-1.8.0_282-b08
Test Results	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/testReport/
Max. process+thread count	4790 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3067/5/console
versions	git=2.17.1 maven=3.6.3
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

d-c-manning · 2021-04-09T15:43:54Z

I will look today.

d-c-manning

LGTM, thank you for the improvement!

ndimiduk · 2021-04-14T16:24:48Z

Hi there @d-c-manning , do you have a name an email address to which you'd like to receive attribution for your review? We try to honor the contributions made by reviewers through the use of Reviewed-by: and Signed-off-by: annotations on commit messages. For an example, see a59ac48. Thanks.

d-c-manning · 2021-04-15T16:59:31Z

oh wow thanks @ndimiduk! - is there a way to make this available for you automatically? David Manning <[email protected]>

apurtell · 2021-04-15T17:26:25Z

This can be merged now?

ndimiduk · 2021-04-15T20:17:12Z

Merged to master. Thank you for the patch, @clarax , and for the careful review, @saintstack and @d-c-manning !

ndimiduk · 2021-04-15T20:18:24Z

oh wow thanks @ndimiduk! - is there a way to make this available for you automatically? David Manning <[email protected]>

I don't know of a means for automated attribution no. I look for this information in the Github user profile, and one around to ask if I don't find it.

ndimiduk · 2021-04-15T20:19:51Z

@apurtell are you comfortable with this as a backport to 2.4? I'm thinking it'd be good for 2.3...

…pache#3067) Signed-off-by: Michael Stack <[email protected]> Reviewed-by: David Manning <[email protected]>

ndimiduk · 2021-04-15T21:03:22Z

@infraio are you comfortable with this as a backport to 2.2?

…pache#3067) Signed-off-by: Michael Stack <[email protected]> Reviewed-by: David Manning <[email protected]>

…ation (#3067)" This reverts commit 533c84d.

…ation (apache#3067)" This reverts commit 533c84d.

clarax force-pushed the HBASE-25625-table-skew branch from 9008479 to a456b01 Compare March 19, 2021 17:13

saintstack reviewed Mar 29, 2021

View reviewed changes

saintstack mentioned this pull request Apr 8, 2021

HBASE-25625 StochasticBalancer CostFunctions needs a better way to ev… #3011

Closed

saintstack approved these changes Apr 8, 2021

View reviewed changes

HBASE-25739 TableSkewCostFunction need to use aggregated deviation

87e4588

clarax force-pushed the HBASE-25625-table-skew branch from d7849df to 87e4588 Compare April 8, 2021 21:21

d-c-manning approved these changes Apr 9, 2021

View reviewed changes

ndimiduk merged commit 533c84d into apache:master Apr 15, 2021

ndimiduk changed the title ~~HBASE-25625 HBASE-25625 StochasticBalancer CostFunctions needs a bett…~~ HBASE-25739 TableSkewCostFunction need to use aggregated deviation Apr 15, 2021

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 15, 2021

HBASE-25739 TableSkewCostFunction need to use aggregated deviation (a…

5c5511e

…pache#3067) Signed-off-by: Michael Stack <[email protected]> Reviewed-by: David Manning <[email protected]>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 15, 2021

HBASE-25739 TableSkewCostFunction need to use aggregated deviation (a…

d337319

…pache#3067) Signed-off-by: Michael Stack <[email protected]> Reviewed-by: David Manning <[email protected]>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 15, 2021

HBASE-25739 TableSkewCostFunction need to use aggregated deviation (a…

12dc5c3

…pache#3067) Signed-off-by: Michael Stack <[email protected]> Reviewed-by: David Manning <[email protected]>

ndimiduk pushed a commit to ndimiduk/hbase that referenced this pull request Apr 15, 2021

HBASE-25739 TableSkewCostFunction need to use aggregated deviation (a…

2d5bea1

…pache#3067) Signed-off-by: Michael Stack <[email protected]> Reviewed-by: David Manning <[email protected]>

ndimiduk added a commit that referenced this pull request Apr 16, 2021

Revert "HBASE-25739 TableSkewCostFunction need to use aggregated devi…

6be4ecf

…ation (#3067)" This reverts commit 533c84d.

ndimiduk mentioned this pull request Apr 16, 2021

Revert "HBASE-25739 TableSkewCostFunction need to use aggregated deviation" #3178

Merged

ndimiduk added a commit that referenced this pull request Apr 16, 2021

Revert "HBASE-25739 TableSkewCostFunction need to use aggregated devi…

b65890d

…ation (#3067)" This reverts commit 533c84d.

sunhelly added a commit to sunhelly/hbase that referenced this pull request Apr 19, 2021

Revert "HBASE-25739 TableSkewCostFunction need to use aggregated devi…

5f156de

…ation (apache#3067)" This reverts commit 533c84d.

clarax mentioned this pull request Jun 9, 2021

HBASE-25739 TableSkewCostFunction need to use aggregated deviation #3370

Closed

HBASE-25739 TableSkewCostFunction need to use aggregated deviation #3067

HBASE-25739 TableSkewCostFunction need to use aggregated deviation #3067

Uh oh!

Conversation

clarax commented Mar 19, 2021

Uh oh!

clarax commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 19, 2021

Uh oh!

Apache-HBase commented Mar 23, 2021

Uh oh!

Apache-HBase commented Mar 24, 2021

Uh oh!

Apache-HBase commented Mar 24, 2021

Uh oh!

saintstack left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Apache-HBase commented Apr 6, 2021

Uh oh!

Apache-HBase commented Apr 7, 2021

Uh oh!

Apache-HBase commented Apr 7, 2021

Uh oh!

saintstack left a comment

Choose a reason for hiding this comment

Uh oh!

saintstack commented Apr 8, 2021

Uh oh!

clarax commented Apr 8, 2021

Uh oh!

clarax commented Apr 8, 2021

Uh oh!

saintstack commented Apr 8, 2021

Uh oh!

clarax commented Apr 8, 2021

Uh oh!

Apache-HBase commented Apr 8, 2021

Uh oh!

Apache-HBase commented Apr 9, 2021

Uh oh!

Apache-HBase commented Apr 9, 2021

Uh oh!

d-c-manning commented Apr 9, 2021

Uh oh!

d-c-manning left a comment

Choose a reason for hiding this comment

Uh oh!

ndimiduk commented Apr 14, 2021

Uh oh!

d-c-manning commented Apr 15, 2021

Uh oh!

apurtell commented Apr 15, 2021

Uh oh!

ndimiduk commented Apr 15, 2021

Uh oh!

ndimiduk commented Apr 15, 2021

Uh oh!

ndimiduk commented Apr 15, 2021

Uh oh!

ndimiduk commented Apr 15, 2021

Uh oh!

Reviewers

Assignees