HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart #2089

mmpataki · 2020-06-22T16:32:35Z

What changed

The sink registration starts the adapter if it doesn't exists.

hadoop-yetus · 2020-06-22T19:20:39Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 33s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	19m 40s	trunk passed
+1 💚	compile	19m 25s	trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
+1 💚	compile	16m 52s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	checkstyle	0m 53s	trunk passed
+1 💚	mvnsite	1m 28s	trunk passed
+1 💚	shadedclient	16m 39s	branch has no errors when building and testing our client artifacts.
-1 ❌	javadoc	0m 45s	hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.
+1 💚	javadoc	1m 1s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗	spotbugs	2m 13s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 11s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 50s	the patch passed
+1 💚	compile	18m 43s	the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
+1 💚	javac	18m 43s	the patch passed
+1 💚	compile	17m 7s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	javac	17m 7s	the patch passed
-0 ⚠️	checkstyle	0m 53s	hadoop-common-project/hadoop-common: The patch generated 2 new + 74 unchanged - 0 fixed = 76 total (was 74)
+1 💚	mvnsite	1m 27s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 38s	patch has no errors when building and testing our client artifacts.
-1 ❌	javadoc	0m 36s	hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.
+1 💚	javadoc	0m 58s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	findbugs	2m 19s	the patch passed
		_ Other Tests _
+1 💚	unit	9m 33s	hadoop-common in the patch passed.
+1 💚	asflicense	0m 50s	The patch does not generate ASF License warnings.
		149m 28s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/1/artifact/out/Dockerfile
GITHUB PR	#2089
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 552ee6355501 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `201d734`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
javadoc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/1/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/1/artifact/out/diff-checkstyle-hadoop-common-project_hadoop-common.txt
javadoc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/1/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/1/testReport/
Max. process+thread count	1427 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/1/console
versions	git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

What changed - The sink registration starts the adapter if it doesn't exists.

steveloughran · 2020-07-03T18:07:35Z

...oject/hadoop-common/src/test/java/org/apache/hadoop/metrics2/impl/TestMetricsSystemImpl.java

+    try {
+      ms.start();
+      ms.register(sinkName, "", ts);
+      assertNotNull("an adapter should exist for each sink", ms.getSinkAdapter(sinkName));


we're generally still 80 chars wide, I'm afraid...put the actual probe on a new line.
nice to see some text for the assertion tnough -appreciated

Fixed it, please review (added more info to the message)

steveloughran

LGTM, +1 pending that checkstyle fixup. [Don't worry about yetus reporting javadoc issues on java 11 BTW]

hadoop-yetus · 2020-07-04T10:28:33Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 31s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	19m 22s	trunk passed
+1 💚	compile	19m 15s	trunk passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
+1 💚	compile	16m 30s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	checkstyle	0m 52s	trunk passed
+1 💚	mvnsite	1m 25s	trunk passed
+1 💚	shadedclient	16m 24s	branch has no errors when building and testing our client artifacts.
-1 ❌	javadoc	0m 45s	hadoop-common in trunk failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.
+1 💚	javadoc	1m 3s	trunk passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+0 🆗	spotbugs	2m 10s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	2m 7s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 50s	the patch passed
+1 💚	compile	18m 35s	the patch passed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04
+1 💚	javac	18m 35s	the patch passed
+1 💚	compile	16m 28s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	javac	16m 28s	the patch passed
+1 💚	checkstyle	0m 53s	the patch passed
+1 💚	mvnsite	1m 24s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 56s	patch has no errors when building and testing our client artifacts.
-1 ❌	javadoc	0m 45s	hadoop-common in the patch failed with JDK Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.
+1 💚	javadoc	1m 4s	the patch passed with JDK Private Build-1.8.0_252-8u252-b09-1~18.04-b09
+1 💚	findbugs	2m 17s	the patch passed
		_ Other Tests _
+1 💚	unit	9m 13s	hadoop-common in the patch passed.
+1 💚	asflicense	0m 53s	The patch does not generate ASF License warnings.
		147m 3s

Subsystem	Report/Notes
Docker	ClientAPI=1.40 ServerAPI=1.40 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/3/artifact/out/Dockerfile
GITHUB PR	#2089
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 33ed3b2d9be2 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `f86f15c`
Default Java	Private Build-1.8.0_252-8u252-b09-1~18.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_252-8u252-b09-1~18.04-b09
javadoc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/3/artifact/out/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
javadoc	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/3/artifact/out/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.7+10-post-Ubuntu-2ubuntu218.04.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/3/testReport/
Max. process+thread count	1473 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-2089/3/console
versions	git=2.17.1 maven=3.6.0 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

steveloughran · 2020-07-06T15:26:04Z

+1 merged to trunk and about to pull into branch-3.3

#2089) Contributed by Madhusoodan P

steveloughran · 2020-07-06T15:33:05Z

and 3.2

BTW, if you are looking at metrics, why not look at #2069 . That's not about publishing externally, but it is designed to enable apps to collect statistics in detail about an instance of an IO object (stream etc) for collation

mmpataki changed the title ~~HADOOP-17081 : MetricsSystem doesn't start the sink adapters on restart~~ HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart Jun 23, 2020

mmpataki force-pushed the HADOOP-17081 branch 2 times, most recently from d759021 to eeed85b Compare June 23, 2020 01:10

HADOOP-17081 : MetricsSystem doesn't start the sink adapters on restart

e670b34

What changed - The sink registration starts the adapter if it doesn't exists.

mmpataki force-pushed the HADOOP-17081 branch from eeed85b to e670b34 Compare June 23, 2020 01:12

Stop the metric system post testing

572df34

steveloughran requested changes Jul 3, 2020

View reviewed changes

Fixed the 80char checkstyle error

6dc79df

mmpataki requested a review from steveloughran July 4, 2020 07:34

steveloughran merged commit 2f500e4 into apache:trunk Jul 6, 2020

asfgit pushed a commit that referenced this pull request Jul 6, 2020

HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart (

0789ae5

#2089) Contributed by Madhusoodan P

asfgit pushed a commit that referenced this pull request Jul 6, 2020

HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart (

dfb74a3

#2089) Contributed by Madhusoodan P

mmpataki deleted the HADOOP-17081 branch July 7, 2020 06:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart #2089

HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart #2089

Uh oh!

mmpataki commented Jun 22, 2020

Uh oh!

hadoop-yetus commented Jun 22, 2020

Uh oh!

steveloughran Jul 3, 2020

Uh oh!

mmpataki Jul 4, 2020

Uh oh!

steveloughran left a comment

Uh oh!

hadoop-yetus commented Jul 4, 2020

Uh oh!

steveloughran commented Jul 6, 2020

Uh oh!

steveloughran commented Jul 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart #2089

HADOOP-17081. MetricsSystem doesn't start the sink adapters on restart #2089

Uh oh!

Conversation

mmpataki commented Jun 22, 2020

Uh oh!

hadoop-yetus commented Jun 22, 2020

Uh oh!

steveloughran Jul 3, 2020

Choose a reason for hiding this comment

Uh oh!

mmpataki Jul 4, 2020

Choose a reason for hiding this comment

Uh oh!

steveloughran left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jul 4, 2020

Uh oh!

steveloughran commented Jul 6, 2020

Uh oh!

steveloughran commented Jul 6, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants