HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… #2762

xiaoyuyao · 2021-03-11T22:10:30Z

https://issues.apache.org/jira/browse/HADOOP-17578

Additional Debug Log for troubleshooting TokenCache related issues.

cxorm

Thank you @xiaoyuyao for the work.

Overall looks great, I'm +1.
Do you mind adding a empty-commit to trigger the CI-checks again ?

sadanand48

+1 . The changes look good to me. Please resolve the failing CI checks.

hadoop-yetus · 2021-03-15T23:05:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 33s		trunk passed
+1 💚	compile	20m 54s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	18m 0s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	1m 6s		trunk passed
+1 💚	mvnsite	1m 33s		trunk passed
+1 💚	javadoc	1m 5s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 35s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 22s		trunk passed
+1 💚	shadedclient	15m 45s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 52s		the patch passed
-1 ❌	compile	8m 4s	/patch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	root in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌	javac	8m 4s	/patch-compile-root-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	root in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
-1 ❌	compile	7m 13s	/patch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt	root in the patch failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
-1 ❌	javac	7m 13s	/patch-compile-root-jdkPrivateBuild-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.txt	root in the patch failed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 47s		the patch passed
+1 💚	mvnsite	1m 12s		the patch passed
-1 ❌	javadoc	0m 46s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
+1 💚	javadoc	1m 20s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 16s		the patch passed
-1 ❌	shadedclient	5m 52s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 25s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 32s		The patch does not generate ASF License warnings.
		141m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/2/artifact/out/Dockerfile
GITHUB PR	#2762
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 4ad9cd0410d5 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / a02fc561ec3aadf9f303aebcfa2fe754db180972
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/2/testReport/
Max. process+thread count	2003 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…e related issues.

hadoop-yetus · 2021-03-16T21:43:45Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 20s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 25s		trunk passed
+1 💚	compile	23m 42s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 29s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 30s		trunk passed
+1 💚	javadoc	1m 0s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 20s		trunk passed
+1 💚	shadedclient	18m 13s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 55s		the patch passed
+1 💚	compile	21m 51s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 51s		the patch passed
+1 💚	compile	18m 58s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	18m 58s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 57s	/results-checkstyle-hadoop-common-project_hadoop-common.txt	hadoop-common-project/hadoop-common: The patch generated 1 new + 71 unchanged - 0 fixed = 72 total (was 71)
+1 💚	mvnsite	1m 27s		the patch passed
-1 ❌	javadoc	1m 0s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
+1 💚	javadoc	1m 33s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 32s		the patch passed
+1 💚	shadedclient	18m 30s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 5s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 48s		The patch does not generate ASF License warnings.
		191m 7s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/3/artifact/out/Dockerfile
GITHUB PR	#2762
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 2a22a533a829 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `54b0ae4`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/3/testReport/
Max. process+thread count	2144 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-03-17T01:21:47Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 22s		trunk passed
+1 💚	compile	22m 38s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	19m 12s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	1m 0s		trunk passed
+1 💚	mvnsite	1m 31s		trunk passed
+1 💚	javadoc	0m 58s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	1m 32s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 20s		trunk passed
+1 💚	shadedclient	18m 2s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 53s		the patch passed
+1 💚	compile	22m 2s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	22m 2s		the patch passed
+1 💚	compile	19m 15s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	19m 15s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 3s		the patch passed
+1 💚	mvnsite	1m 25s		the patch passed
-1 ❌	javadoc	0m 57s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04.
+1 💚	javadoc	1m 32s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	spotbugs	2m 31s		the patch passed
+1 💚	shadedclient	18m 15s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 14s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 49s		The patch does not generate ASF License warnings.
		189m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/4/artifact/out/Dockerfile
GITHUB PR	#2762
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 206807b5ac97 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6655d22`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/4/testReport/
Max. process+thread count	2533 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2762/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

xiaoyuyao · 2021-03-17T17:56:56Z

Java doc issue is unrelated. Thanks @cxorm for the review. I'll merge the PR shortly.

aajisaka · 2021-03-18T03:49:33Z

Java doc issue is unrelated.

[ERROR] /home/jenkins/jenkins-home/workspace/hadoop-multibranch_PR-2762/src/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java:1923: error: exception not thrown: java.io.IOException
[ERROR]    * @throws IOException
[ERROR]              ^

throws IOException has been removed by the patch and it is related.

@xiaoyuyao @cxorm
Would you please fix this?

steveloughran · 2021-03-22T21:48:36Z

yeah, we do need a fix for this....

xiaoyuyao · 2021-03-22T23:52:08Z

Thanks @steveloughran and @aajisaka. I will open a new ticket for the javadoc fix.

xiaoyuyao · 2021-03-23T00:01:22Z

#2802 is opened to fix the javadoc issue.

apache#2762)

apache#2762) (cherry picked from commit b503de2) Change-Id: I9957dd1b80720966829dd49cff133223b90829cf

…okenCach… (apache#2762)" This reverts commit 3102f84. Reason for revert: <INSERT REASONING HERE> Change-Id: I159965c018f89ba7e495f0467034ce1fc2ea8bdd

apache#2762) (cherry picked from commit b503de2) Change-Id: I25ced4aebca6092d4f08c1d9d5e355419bd36b38

xiaoyuyao closed this Mar 12, 2021

xiaoyuyao reopened this Mar 12, 2021

cxorm approved these changes Mar 15, 2021

View reviewed changes

sadanand48 approved these changes Mar 15, 2021

View reviewed changes

xiaoyuyao added 3 commits March 16, 2021 11:11

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach…

0f98790

…e related issues.

refactor to trigger jenkins.

61fb9ed

Fix ut issue.

54b0ae4

xiaoyuyao force-pushed the HADOOP-17578 branch from a02fc56 to 54b0ae4 Compare March 16, 2021 18:31

fix checkstyle

6655d22

xiaoyuyao merged commit b503de2 into apache:trunk Mar 17, 2021

xiaoyuyao deleted the HADOOP-17578 branch March 19, 2021 20:22

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (

7f6c37f

apache#2762)

jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (

3102f84

apache#2762) (cherry picked from commit b503de2) Change-Id: I9957dd1b80720966829dd49cff133223b90829cf

jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… (

71e1232

apache#2762) (cherry picked from commit b503de2) Change-Id: I25ced4aebca6092d4f08c1d9d5e355419bd36b38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… #2762

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… #2762

Uh oh!

xiaoyuyao commented Mar 11, 2021

Uh oh!

cxorm left a comment

Uh oh!

sadanand48 left a comment

Uh oh!

hadoop-yetus commented Mar 15, 2021

Uh oh!

hadoop-yetus commented Mar 16, 2021

Uh oh!

hadoop-yetus commented Mar 17, 2021

Uh oh!

xiaoyuyao commented Mar 17, 2021

Uh oh!

aajisaka commented Mar 18, 2021

Uh oh!

steveloughran commented Mar 22, 2021

Uh oh!

xiaoyuyao commented Mar 22, 2021

Uh oh!

xiaoyuyao commented Mar 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… #2762

HADOOP-17578. Improve UGI debug log to help troubleshooting TokenCach… #2762

Uh oh!

Conversation

xiaoyuyao commented Mar 11, 2021

Uh oh!

cxorm left a comment

Choose a reason for hiding this comment

Uh oh!

sadanand48 left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Mar 15, 2021

Uh oh!

hadoop-yetus commented Mar 16, 2021

Uh oh!

hadoop-yetus commented Mar 17, 2021

Uh oh!

xiaoyuyao commented Mar 17, 2021

Uh oh!

aajisaka commented Mar 18, 2021

Uh oh!

steveloughran commented Mar 22, 2021

Uh oh!

xiaoyuyao commented Mar 22, 2021

Uh oh!

xiaoyuyao commented Mar 23, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants