HADOOP-16591 Fix S3A ITest*MRjob failures.

[sidseth]

NOTICE

Please create an issue in ASF JIRA before opening a pull request,
and you need to set the title of the pull request which starts with
the corresponding JIRA issue number. (e.g. HADOOP-XXXXX. Fix a typo in YYY.)
For more details, please see https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute

[sidseth]

Test command run: mvn -T 1C verify -Dparallel-tests -DtestsThreadCount=12 -Ds3guard -Dauth -Ddynamo
Region: us-east-2
There's the usual flaky test failures. Otherwise, all tests succeed.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	79	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1189	trunk passed
+1	compile	31	trunk passed
+1	checkstyle	24	trunk passed
+1	mvnsite	37	trunk passed
-1	shadedclient	98	branch has errors when building and testing our client artifacts.
+1	javadoc	24	trunk passed
0	spotbugs	60	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	58	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	32	the patch passed
+1	compile	26	the patch passed
+1	javac	26	the patch passed
+1	checkstyle	17	the patch passed
+1	mvnsite	30	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
-1	shadedclient	27	patch has errors when building and testing our client artifacts.
+1	javadoc	20	the patch passed
+1	findbugs	60	the patch passed
		_ Other Tests _
+1	unit	78	hadoop-aws in the patch passed.
+1	asflicense	23	The patch does not generate ASF License warnings.
		1927

Subsystem	Report/Notes
Docker	Client=19.03.2 Server=19.03.2 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1487/1/artifact/out/Dockerfile
GITHUB PR	#1487
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 732c1198d1c9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 3f223be
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1487/1/testReport/
Max. process+thread count	305 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1487/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	110	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1455	trunk passed
+1	compile	37	trunk passed
+1	checkstyle	28	trunk passed
+1	mvnsite	42	trunk passed
-1	shadedclient	116	branch has errors when building and testing our client artifacts.
+1	javadoc	34	trunk passed
0	spotbugs	67	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	65	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	38	the patch passed
+1	compile	28	the patch passed
+1	javac	28	the patch passed
+1	checkstyle	21	the patch passed
+1	mvnsite	37	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
-1	shadedclient	33	patch has errors when building and testing our client artifacts.
+1	javadoc	26	the patch passed
+1	findbugs	70	the patch passed
		_ Other Tests _
+1	unit	92	hadoop-aws in the patch passed.
+1	asflicense	30	The patch does not generate ASF License warnings.
		2355

Subsystem	Report/Notes
Docker	Client=18.09.7 Server=18.09.7 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1487/1/artifact/out/Dockerfile
GITHUB PR	#1487
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 314cce32d3ca 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / d7d6ec8
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1487/1/testReport/
Max. process+thread count	264 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1487/1/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

[sidseth]

The shadedclient failure is not related. More details at #1432 (comment)

[sidseth]

/rerun

[sidseth]

/rebuild

[steveloughran]

+1, committed to trunk. Thanks!

FWIW, in HADOOP-16570 I'm revisiting this change to see if it should also/should instead have the conf used to create the minicluster changed by forcing down the test properties by way of the relevant S3ATestUtils call. I'll leave this change here in though

[sidseth]

Thanks. I wasn't sure about that change - especially if the test configs set fs.defaultFS. OTOH, the MiniDFS cluster will likely overwrite that setting anyway - so should be safe.