HDDS-1464. Client should have different retry policies for different exceptions. #785

swagle · 2019-04-29T21:24:35Z

@hanishakoneru

KeyOutputStream correctly does the handleWrite() from inside the retry so only the correct setting for the fixed delay between retries needs to be set appropriately.

hadoop-yetus · 2019-04-29T22:15:37Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	24	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	1054	trunk passed
+1	compile	48	trunk passed
+1	checkstyle	20	trunk passed
+1	mvnsite	28	trunk passed
+1	shadedclient	716	branch has no errors when building and testing our client artifacts.
+1	findbugs	34	trunk passed
+1	javadoc	19	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	30	the patch passed
+1	compile	21	the patch passed
+1	javac	21	the patch passed
-0	checkstyle	14	hadoop-ozone/client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	mvnsite	24	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	747	patch has no errors when building and testing our client artifacts.
+1	findbugs	42	the patch passed
+1	javadoc	20	the patch passed
		_ Other Tests _
+1	unit	27	client in the patch passed.
+1	asflicense	29	The patch does not generate ASF License warnings.
		2985

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-785/1/artifact/out/Dockerfile
GITHUB PR	#785
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux e829b67ce4ef 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24 10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `95790bb`
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/1/artifact/out/diff-checkstyle-hadoop-ozone_client.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/1/testReport/
Max. process+thread count	410 (vs. ulimit of 5500)
modules	C: hadoop-ozone/client U: hadoop-ozone/client
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/1/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyOutputStream.java

hadoop-yetus · 2019-04-29T23:07:51Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	816	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	1241	trunk passed
+1	compile	49	trunk passed
+1	checkstyle	21	trunk passed
+1	mvnsite	29	trunk passed
+1	shadedclient	689	branch has no errors when building and testing our client artifacts.
+1	findbugs	38	trunk passed
+1	javadoc	24	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	31	the patch passed
+1	compile	22	the patch passed
+1	javac	22	the patch passed
-0	checkstyle	14	hadoop-ozone/client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	mvnsite	23	the patch passed
-1	whitespace	0	The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>. Refer https://git-scm.com/docs/git-apply
+1	shadedclient	758	patch has no errors when building and testing our client artifacts.
+1	findbugs	44	the patch passed
+1	javadoc	20	the patch passed
		_ Other Tests _
+1	unit	27	client in the patch passed.
+1	asflicense	33	The patch does not generate ASF License warnings.
		4000

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-785/2/artifact/out/Dockerfile
GITHUB PR	#785
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 13ab06d49bcf 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 4b4200f
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/2/artifact/out/diff-checkstyle-hadoop-ozone_client.txt
whitespace	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/2/artifact/out/whitespace-eol.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/2/testReport/
Max. process+thread count	411 (vs. ulimit of 5500)
modules	C: hadoop-ozone/client U: hadoop-ozone/client
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/2/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-04-30T01:17:34Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	23	Docker mode activated.
		_ Prechecks _
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	1011	trunk passed
+1	compile	32	trunk passed
+1	checkstyle	18	trunk passed
+1	mvnsite	28	trunk passed
+1	shadedclient	645	branch has no errors when building and testing our client artifacts.
+1	findbugs	33	trunk passed
+1	javadoc	22	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	27	the patch passed
+1	compile	19	the patch passed
+1	javac	19	the patch passed
-0	checkstyle	12	hadoop-ozone/client: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1	mvnsite	21	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	696	patch has no errors when building and testing our client artifacts.
+1	findbugs	42	the patch passed
+1	javadoc	17	the patch passed
		_ Other Tests _
+1	unit	23	client in the patch passed.
+1	asflicense	27	The patch does not generate ASF License warnings.
		2782

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-785/3/artifact/out/Dockerfile
GITHUB PR	#785
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 061695c41da7 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2 17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 4b4200f
maven	version: Apache Maven 3.3.9
Default Java	1.8.0_191
findbugs	v3.1.0-RC1
checkstyle	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/3/artifact/out/diff-checkstyle-hadoop-ozone_client.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/3/testReport/
Max. process+thread count	444 (vs. ulimit of 5500)
modules	C: hadoop-ozone/client U: hadoop-ozone/client
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/3/console
Powered by	Apache Yetus 0.9.0 http://yetus.apache.org

This message was automatically generated.

…exceptions.

hadoop-yetus · 2019-05-02T17:45:07Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	46	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
-1	test4tests	0	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ trunk Compile Tests _
+1	mvninstall	434	trunk passed
+1	compile	214	trunk passed
+1	checkstyle	52	trunk passed
+1	mvnsite	0	trunk passed
+1	shadedclient	868	branch has no errors when building and testing our client artifacts.
+1	javadoc	126	trunk passed
0	spotbugs	298	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	501	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	444	the patch passed
+1	compile	203	the patch passed
+1	javac	203	the patch passed
+1	checkstyle	56	the patch passed
+1	mvnsite	0	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	727	patch has no errors when building and testing our client artifacts.
+1	javadoc	124	the patch passed
+1	findbugs	448	the patch passed
		_ Other Tests _
-1	unit	163	hadoop-hdds in the patch failed.
-1	unit	967	hadoop-ozone in the patch failed.
+1	asflicense	32	The patch does not generate ASF License warnings.
		5530

Reason	Tests
Failed junit tests	hadoop.ozone.client.rpc.TestOzoneClientRetriesOnException
	hadoop.hdds.scm.safemode.TestSCMSafeModeWithPipelineRules
	hadoop.ozone.scm.TestXceiverClientMetrics
	hadoop.ozone.web.TestOzoneVolumes
	hadoop.ozone.om.TestOmInit
	hadoop.ozone.client.rpc.TestContainerStateMachineFailures
	hadoop.ozone.web.client.TestKeys
	hadoop.ozone.scm.pipeline.TestSCMPipelineMetrics
	hadoop.ozone.web.client.TestOzoneClient
	hadoop.ozone.client.rpc.TestHybridPipelineOnDatanode
	hadoop.hdds.scm.container.TestContainerStateManagerIntegration
	hadoop.ozone.ozShell.TestOzoneDatanodeShell
	hadoop.ozone.TestStorageContainerManager
	hadoop.ozone.client.rpc.TestCloseContainerHandlingByClient
	hadoop.ozone.scm.pipeline.TestPipelineManagerMXBean
	hadoop.ozone.om.TestOzoneManagerRestInterface
	hadoop.ozone.container.common.statemachine.commandhandler.TestCloseContainerByPipeline
	hadoop.ozone.om.TestOzoneManagerHA
	hadoop.hdds.scm.pipeline.TestPipelineClose
	hadoop.ozone.ozShell.TestS3Shell
	hadoop.hdds.scm.pipeline.TestNode2PipelineMap
	hadoop.ozone.scm.TestGetCommittedBlockLengthAndPutKey
	hadoop.ozone.scm.TestSCMNodeManagerMXBean
	hadoop.ozone.scm.TestSCMMXBean
	hadoop.ozone.ozShell.TestOzoneShell
	hadoop.ozone.om.TestOzoneManagerConfiguration
	hadoop.ozone.web.TestOzoneWebAccess
	hadoop.ozone.client.rpc.TestCommitWatcher
	hadoop.ozone.om.TestOmMetrics
	hadoop.ozone.om.TestContainerReportWithKeys
	hadoop.ozone.client.rpc.TestOzoneRpcClient
	hadoop.ozone.scm.TestContainerSmallFile
	hadoop.hdds.scm.pipeline.TestSCMRestart
	hadoop.ozone.scm.node.TestQueryNode
	hadoop.ozone.scm.node.TestSCMNodeMetrics
	hadoop.ozone.client.rpc.TestBCSID
	hadoop.hdds.scm.pipeline.TestNodeFailure
	hadoop.ozone.scm.TestXceiverClientManager

Subsystem	Report/Notes
Docker	Client=17.05.0-ce Server=17.05.0-ce base: https://builds.apache.org/job/hadoop-multibranch/job/PR-785/4/artifact/out/Dockerfile
GITHUB PR	#785
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 0555deb12949 4.4.0-144-generic #170~14.04.1-Ubuntu SMP Mon Mar 18 15:02:05 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6a42745`
Default Java	1.8.0_191
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/4/artifact/out/patch-unit-hadoop-hdds.txt
unit	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/4/artifact/out/patch-unit-hadoop-ozone.txt
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/4/testReport/
Max. process+thread count	4710 (vs. ulimit of 5500)
modules	C: hadoop-ozone/client U: hadoop-ozone/client
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-785/4/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

swagle · 2019-05-02T18:07:11Z

/label ozone

hanishakoneru · 2019-05-03T19:31:21Z

The patch LGTM overall.
The CI unit test failure TestOzoneClientRetriesOnException#testMaxRetriesByOzoneClient looks related though.

swagle · 2019-05-04T00:54:52Z

Thanks for review @hanishakoneru. On my local machine, I get a 80% pass rate on this test.
I can see that it is failing elsewhere without these changes as well: #781

hanishakoneru · 2019-05-05T16:20:40Z

The test is flaky. I will merge this PR. Thanks @swagle for working on this.

Author: Jagadish <[email protected]> Reviewers: Jagadish<[email protected]> Closes apache#785 from vjagadish1989/website-reorg28

hadoop-yetus reviewed Apr 29, 2019

View reviewed changes

hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/io/KeyOutputStream.java Outdated Show resolved Hide resolved

swagle added 4 commits May 2, 2019 09:04

HDDS-1464. Client should have different retry policies for different …

b1dfdf1

…exceptions.

HDDS-1464. Add default RetryPolicy. Fix integration test.

3a4bb97

HDDS-1464. Fix for whitespace error.

e245865

HDDS-1464. Fix for checkstyle error.

21cc83e

swagle force-pushed the HDDS-1464 branch from e6649ea to 21cc83e Compare May 2, 2019 16:05

elek added the ozone label May 2, 2019

hanishakoneru merged commit 1d70c8c into apache:trunk May 5, 2019

shanthoosh pushed a commit to shanthoosh/hadoop that referenced this pull request Oct 15, 2019

atoomula and prateekm FYI..

de2b97f

Author: Jagadish <[email protected]> Reviewers: Jagadish<[email protected]> Closes apache#785 from vjagadish1989/website-reorg28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-1464. Client should have different retry policies for different exceptions. #785

HDDS-1464. Client should have different retry policies for different exceptions. #785

Uh oh!

swagle commented Apr 29, 2019 •

edited

Loading

Uh oh!

hadoop-yetus commented Apr 29, 2019

Uh oh!

Uh oh!

hadoop-yetus commented Apr 29, 2019

Uh oh!

hadoop-yetus commented Apr 30, 2019

Uh oh!

hadoop-yetus commented May 2, 2019

Uh oh!

swagle commented May 2, 2019

Uh oh!

hanishakoneru commented May 3, 2019

Uh oh!

swagle commented May 4, 2019

Uh oh!

hanishakoneru commented May 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HDDS-1464. Client should have different retry policies for different exceptions. #785

HDDS-1464. Client should have different retry policies for different exceptions. #785

Uh oh!

Conversation

swagle commented Apr 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Apr 29, 2019

Uh oh!

Uh oh!

hadoop-yetus commented Apr 29, 2019

Uh oh!

hadoop-yetus commented Apr 30, 2019

Uh oh!

hadoop-yetus commented May 2, 2019

Uh oh!

swagle commented May 2, 2019

Uh oh!

hanishakoneru commented May 3, 2019

Uh oh!

swagle commented May 4, 2019

Uh oh!

hanishakoneru commented May 5, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

swagle commented Apr 29, 2019 •

edited

Loading