HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 #3241

virajjasani · 2021-07-27T14:18:36Z

No description provided.

virajjasani · 2021-07-27T14:21:18Z

dev-support/Jenkinsfile

    options {
        buildDiscarder(logRotator(numToKeepStr: '5'))
-        timeout (time: 24, unit: 'HOURS')
+        timeout (time: 40, unit: 'HOURS')


Just in case full build exceeds 24 hr, I have done this change. Will revert this once we have confidence on QA results.

hadoop-yetus · 2021-07-28T10:19:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 8s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 25s		trunk passed
+1 💚	compile	21m 18s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 43s		trunk passed
+1 💚	mvnsite	24m 31s		trunk passed
+1 💚	javadoc	7m 41s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 45s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 21s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	47m 36s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 35s		Maven dependency ordering for patch
+1 💚	mvninstall	26m 30s		the patch passed
+1 💚	compile	24m 47s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	24m 47s	/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 10 new + 1921 unchanged - 0 fixed = 1931 total (was 1921)
+1 💚	compile	23m 44s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	23m 44s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 10 new + 1797 unchanged - 0 fixed = 1807 total (was 1797)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 45s	/results-checkstyle-root.txt	root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚	mvnsite	25m 59s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 15s		The patch has no ill-formed XML file.
+1 💚	javadoc	9m 36s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	9m 32s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 18s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	57m 59s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	776m 10s	/patch-unit-root.txt	root in the patch passed.
-1 ❌	asflicense	1m 35s	/results-asflicense.txt	The patch generated 48 ASF License warnings.
		1162m 3s

Reason	Tests
Failed junit tests	hadoop.ha.TestShellCommandFencer
	hadoop.http.TestSSLHttpServer
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
	hadoop.hdfs.TestRollingUpgrade
	hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/1/artifact/out/Dockerfile
GITHUB PR	#3241
Optional Tests	dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname	Linux f91e016ce10f 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `775340b`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/1/testReport/
Max. process+thread count	3315 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-07-28T10:28:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	shelldocs	0m 1s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 59s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 8s		trunk passed
+1 💚	compile	21m 11s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 30s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 36s		trunk passed
+1 💚	mvnsite	24m 23s		trunk passed
+1 💚	javadoc	7m 27s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 43s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 20s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	45m 6s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	25m 39s		the patch passed
+1 💚	compile	20m 55s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	20m 55s	/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 10 new + 1914 unchanged - 0 fixed = 1924 total (was 1914)
+1 💚	compile	19m 28s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	19m 28s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 10 new + 1793 unchanged - 0 fixed = 1803 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 42s	/results-checkstyle-root.txt	root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚	mvnsite	22m 13s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 13s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 56s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 15s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 19s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	48m 59s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	807m 18s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 18s		The patch does not generate ASF License warnings.
		1160m 45s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestFileChecksum
	hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/2/artifact/out/Dockerfile
GITHUB PR	#3241
Optional Tests	dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname	Linux 48c0ac257045 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `775340b`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/2/testReport/
Max. process+thread count	3586 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/2/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

This reverts commit 775340b.

aajisaka · 2021-07-29T07:56:30Z

If Curator 5.2 supports ZooKeeper 3.5.x, I'm +1 to upgrade.

virajjasani · 2021-07-29T13:49:31Z

If Curator 5.2 supports ZooKeeper 3.5.x, I'm +1 to upgrade.

@aajisaka you mean if (Curator 5.2 + Zookeeper 3.6) are compatible with Zookeeper 3.5 server? As per this doc, Curator 5 does not support ZK 3.4 but it does not mention if Curator 5 is wire compatible with ZK 3.5 servers.
@eolivelli would you be able to provide compatibility metrics of Curator 5.2 with ZK 3.5?
Thanks

Edit: Reg Zookeeper 3.4/3.5/3.6 compatibilities, found this from release doc here:
ZooKeeper clients from 3.4 and 3.5 branch are fully compatible with 3.6 servers. The upgrade from 3.5.7 to 3.6.0 can be executed as usual, no particular additional upgrade procedure is needed. ZooKeeper 3.6.0 clients are compatible with 3.5 servers as long as you are not using new APIs not present in 3.5.

eolivelli · 2021-07-29T14:18:29Z

would you be able to provide compatibility metrics of Curator 5.2 with ZK 3.5?

in Curator we have integration tests about using ZooKeeper 3.5 client and server on the classpath.
So it should work well even with ZK 3.5.x

https://github.com/apache/curator/tree/master/curator-test-zk35

eolivelli · 2021-07-29T14:54:53Z

hadoop-common-project/hadoop-auth/pom.xml

      <artifactId>zookeeper</artifactId>
    </dependency>
+    <dependency>
+      <groupId>io.dropwizard.metrics</groupId>


usually you should also add "snappy-java"

Done. Thanks

Hi,

Why did we decide to add io.dropwizard.metrics as part of this PR? Is it related with zk upgrade?

Yea. They are needed only for the server side part in case you use that metrics provider, that's the default

hadoop-yetus · 2021-07-29T20:51:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 34s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 9s		trunk passed
+1 💚	compile	24m 18s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	20m 41s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 53s		trunk passed
+1 💚	mvnsite	26m 37s		trunk passed
+1 💚	javadoc	8m 19s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 24s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 21s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	50m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	30m 16s		the patch passed
+1 💚	compile	24m 40s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	24m 40s	/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 10 new + 1921 unchanged - 0 fixed = 1931 total (was 1921)
+1 💚	compile	19m 40s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	19m 40s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 10 new + 1797 unchanged - 0 fixed = 1807 total (was 1797)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 42s	/results-checkstyle-root.txt	root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚	mvnsite	21m 26s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 13s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 54s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 11s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 19s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	51m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	309m 15s	/patch-unit-root.txt	root in the patch failed.
+0 🆗	asflicense	0m 41s		ASF License check generated no output?
		691m 17s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy
	hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
	hadoop.hdfs.TestCrcCorruption
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.blockmanagement.TestBlockInfoStriped
	hadoop.hdfs.TestEncryptionZones
	hadoop.hdfs.TestTrashWithEncryptionZones
	hadoop.hdfs.TestFileAppend2
	hadoop.hdfs.TestReplaceDatanodeOnFailure
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/3/artifact/out/Dockerfile
GITHUB PR	#3241
Optional Tests	dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname	Linux 88c228210564 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `46e1f6b`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/3/testReport/
Max. process+thread count	3158 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/3/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-07-31T04:30:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 54s		Maven dependency ordering for branch
+1 💚	mvninstall	24m 49s		trunk passed
+1 💚	compile	27m 11s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	compile	20m 49s	/branch-compile-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root in trunk failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-0 ⚠️	checkstyle	0m 41s	/buildtool-branch-checkstyle-root.txt	The patch fails to run checkstyle in root
+1 💚	mvnsite	29m 29s		trunk passed
+1 💚	javadoc	9m 24s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 58s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 21s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	45m 28s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	25m 34s		the patch passed
+1 💚	compile	20m 39s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	20m 39s	/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 10 new + 1921 unchanged - 0 fixed = 1931 total (was 1921)
+1 💚	compile	18m 29s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	18m 29s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 196 new + 1611 unchanged - 0 fixed = 1807 total (was 1611)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 38s	/results-checkstyle-root.txt	root: The patch generated 357 new + 0 unchanged - 0 fixed = 357 total (was 0)
+1 💚	mvnsite	20m 32s		the patch passed
+1 💚	shellcheck	0m 1s		No new issues.
+1 💚	xml	0m 13s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 30s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 41s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 21s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	45m 37s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	770m 28s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 34s		The patch does not generate ASF License warnings.
		1147m 55s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/6/artifact/out/Dockerfile
GITHUB PR	#3241
Optional Tests	dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname	Linux 75f0d31c6689 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `811277f`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/6/testReport/
Max. process+thread count	3459 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/6/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-07-31T13:33:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 5s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 53s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 24s		trunk passed
+1 💚	compile	23m 59s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 0s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 51s		trunk passed
+1 💚	mvnsite	26m 27s		trunk passed
+1 💚	javadoc	8m 13s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 8s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 19s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	49m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	27m 15s		the patch passed
+1 💚	compile	23m 6s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	23m 6s	/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 10 new + 1921 unchanged - 0 fixed = 1931 total (was 1921)
+1 💚	compile	20m 55s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	20m 55s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 10 new + 1797 unchanged - 0 fixed = 1807 total (was 1797)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 44s	/results-checkstyle-root.txt	root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚	mvnsite	22m 40s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 13s		The patch has no ill-formed XML file.
+1 💚	javadoc	8m 12s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 10s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 22s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	49m 34s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	821m 28s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 41s		The patch does not generate ASF License warnings.
		1198m 14s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/8/artifact/out/Dockerfile
GITHUB PR	#3241
Optional Tests	dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname	Linux 172ad69e501c 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `811277f`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/8/testReport/
Max. process+thread count	3151 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/8/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-07-31T13:37:13Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	12m 55s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 14s		trunk passed
+1 💚	compile	23m 59s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 8s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 50s		trunk passed
+1 💚	mvnsite	26m 32s		trunk passed
+1 💚	javadoc	8m 10s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 14s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 20s		branch/hadoop-project no spotbugs output file (spotbugsXml.xml)
+1 💚	shadedclient	49m 15s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 37s		Maven dependency ordering for patch
+1 💚	mvninstall	27m 21s		the patch passed
+1 💚	compile	23m 3s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
-1 ❌	javac	23m 3s	/results-compile-javac-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 generated 10 new + 1917 unchanged - 0 fixed = 1927 total (was 1917)
+1 💚	compile	20m 52s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	20m 52s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 10 new + 1793 unchanged - 0 fixed = 1803 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 46s	/results-checkstyle-root.txt	root: The patch generated 1 new + 356 unchanged - 2 fixed = 357 total (was 358)
+1 💚	mvnsite	22m 32s		the patch passed
+1 💚	shellcheck	0m 0s		No new issues.
+1 💚	xml	0m 13s		The patch has no ill-formed XML file.
+1 💚	javadoc	8m 11s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 13s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+0 🆗	spotbugs	0m 22s		hadoop-project has no data from spotbugs
+1 💚	shadedclient	49m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	824m 27s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 40s		The patch does not generate ASF License warnings.
		1201m 54s

Reason	Tests
Failed junit tests	hadoop.tools.fedbalance.TestDistCpProcedure
	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/9/artifact/out/Dockerfile
GITHUB PR	#3241
Optional Tests	dupname asflicense codespell shellcheck shelldocs compile javac javadoc mvninstall mvnsite unit shadedclient xml spotbugs checkstyle
uname	Linux 26930ed6d652 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `811277f`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/9/testReport/
Max. process+thread count	3149 (vs. ulimit of 5500)
modules	C: hadoop-project hadoop-common-project/hadoop-auth hadoop-common-project/hadoop-common hadoop-common-project/hadoop-registry hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-nfs hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-mawo/hadoop-yarn-applications-mawo-core . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3241/9/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

virajjasani · 2021-08-01T18:41:35Z

Addressed comment by @eolivelli.
It seems we have enough full build (~20 hr) QA results with latest revision and verified loaded data from PathChildrenCache (e.g keyCache and tokenCache). FYI @aajisaka @jojochuang.
Thanks

aajisaka

Thank you @virajjasani and @eolivelli

aajisaka · 2021-08-03T05:44:19Z

BTW, let's fix the deprecation warnings in a separate jira.

…#3241) Signed-off-by: Akira Ajisaka <[email protected]>

…o 3.6.3 and Curator to 5.2.0) (#5097) * HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (#3241) Signed-off-by: Akira Ajisaka <[email protected]> Co-authored-by: Viraj Jasani <[email protected]> Co-authored-by: Melissa You <[email protected]>

…o 3.6.3 and Curator to 5.2.0) (apache#5097) (apache#24) * HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache#3241) Signed-off-by: Akira Ajisaka <[email protected]> Co-authored-by: Viraj Jasani <[email protected]> Co-authored-by: Melissa You <[email protected]> Co-authored-by: Viraj Jasani <[email protected]>

virajjasani added 2 commits July 27, 2021 19:47

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0

cde957f

to be reverted

775340b

virajjasani commented Jul 27, 2021

View reviewed changes

Revert "to be reverted"

46e1f6b

This reverts commit 775340b.

eolivelli reviewed Jul 29, 2021

View reviewed changes

adding snappy-java dependency

2321c6c

fix checkstyle

811277f

aajisaka approved these changes Aug 2, 2021

View reviewed changes

eolivelli approved these changes Aug 2, 2021

View reviewed changes

aajisaka merged commit ccfa072 into apache:trunk Aug 3, 2021

virajjasani deleted the HADOOP-17612-trunk1 branch August 9, 2021 09:03

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache…

b12f3ec

…#3241) Signed-off-by: Akira Ajisaka <[email protected]>

steveloughran mentioned this pull request Jun 23, 2022

HADOOP-18311. Upgrade dependencies to address several CVEs #4491

Closed

4 tasks

melissayou pushed a commit to melissayou/hadoop that referenced this pull request Oct 19, 2022

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache…

679535c

…#3241) Signed-off-by: Akira Ajisaka <[email protected]>

melissayou pushed a commit to melissayou/hadoop that referenced this pull request Oct 20, 2022

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache…

f136ca0

…#3241) Signed-off-by: Akira Ajisaka <[email protected]>

melissayou pushed a commit to melissayou/hadoop that referenced this pull request Oct 23, 2022

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache…

d5982b1

…#3241) Signed-off-by: Akira Ajisaka <[email protected]>

melissayou mentioned this pull request Oct 28, 2022

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 #5067

Open

4 tasks

melissayou pushed a commit to melissayou/hadoop that referenced this pull request Nov 2, 2022

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 (apache…

2ab4794

…#3241) Signed-off-by: Akira Ajisaka <[email protected]>

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 #3241

HADOOP-17612. Upgrade Zookeeper to 3.6.3 and Curator to 5.2.0 #3241

Uh oh!

Conversation

virajjasani commented Jul 27, 2021

Uh oh!

virajjasani Jul 27, 2021

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jul 28, 2021

Uh oh!

hadoop-yetus commented Jul 28, 2021

Uh oh!

aajisaka commented Jul 29, 2021

Uh oh!

virajjasani commented Jul 29, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eolivelli commented Jul 29, 2021

Uh oh!

eolivelli Jul 29, 2021

Choose a reason for hiding this comment

Uh oh!

virajjasani Jul 29, 2021

Choose a reason for hiding this comment

Uh oh!

xinglin Oct 14, 2022

Choose a reason for hiding this comment

Uh oh!

eolivelli Oct 15, 2022

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jul 29, 2021

Uh oh!

hadoop-yetus commented Jul 31, 2021

Uh oh!

hadoop-yetus commented Jul 31, 2021

Uh oh!

hadoop-yetus commented Jul 31, 2021

Uh oh!

virajjasani commented Aug 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aajisaka left a comment

Choose a reason for hiding this comment

Uh oh!

aajisaka commented Aug 3, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

virajjasani commented Jul 29, 2021 •

edited

Loading

virajjasani commented Aug 1, 2021 •

edited

Loading