HADOOP-16005: Add XAttr support to WASB and ABFS #452

c-w · 2018-12-22T19:25:15Z

As discussed in HADOOP-16005, this pull request implements getXAttr and setXAttr on hadoop-azure's WASB and ABFS file-systems.

The changes were tested against the following Azure storage account configurations:

WASB: StorageV2, RA-GRS replication in East US (primary) West US (secondary). WASB test session screenshot. All tests pass and the ABFS tests are skipped as expected.
ABFS: StorageV2 with Data Lake Storage Gen2 preview enabled, RA-GRS replication in East US (primary) West US (secondary). ABFS test session screenshot. All ABFS tests pass but the WASB tests fail since the storage account hasn't implemented the blob endpoints yet.

The test-patch script passed: test-patch output.

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java

AvroDataFileHdfsWriter should include avro as the file suffix as some pig jobs couldn't read the avro files if they don't come with the proper suffix Author: Hai Lu <[email protected]> Reviewers: Xinyu Liu <[email protected]> Closes apache#452 from lhaiesp/master

c-w · 2019-11-26T23:37:04Z

Apologies for the delay on this pull request. I resolved the merge conflicts and pushed an update.

All the unit testes pass, but when running the integration tests with my azure-auth-keys.xml file against StorageV2 accounts, I'm getting some failures. However, I'm also getting these failures when running against trunk so I doubt that the errors are related to this pull request.

I ran the tests by executing sudo mvn clean install -Pdist -Dtar -Ptest-patch -pl ./hadoop-tools/hadoop-azure | tee test.log.txt in the development environment Docker container running on WSL: see test.log.txt and a sample detailed test failure.

As a new contributor to this project, I'd appreciate any guidance on how to resolve the errors and properly integration test my change. Any ideas @DadanielZ? Thanks in advance for your help!

DadanielZ · 2019-11-26T23:51:54Z

@c-w the configuration for ABFS is not correct, the account name and related suffix should be "dfs.core.windows.net", not "blob.core.windows.net".

hadoop-yetus · 2019-11-27T00:33:18Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	3m 25s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	23m 59s	trunk passed
+1 💚	compile	0m 30s	trunk passed
+1 💚	checkstyle	0m 23s	trunk passed
+1 💚	mvnsite	0m 33s	trunk passed
+1 💚	shadedclient	15m 47s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 36s	trunk passed
+0 🆗	spotbugs	0m 57s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 55s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s	the patch passed
+1 💚	compile	0m 22s	the patch passed
+1 💚	javac	0m 22s	the patch passed
+1 💚	checkstyle	0m 21s	the patch passed
+1 💚	mvnsite	0m 27s	the patch passed
+1 💚	whitespace	0m 1s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 7s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 23s	the patch passed
+1 💚	findbugs	0m 53s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 18s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 29s	The patch does not generate ASF License warnings.
		67m 37s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/2/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 9e270c2aaaf6 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `ef950b0`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/2/testReport/
Max. process+thread count	305 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/2/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

c-w · 2019-11-27T04:34:23Z

Thanks for the help, @DadanielZ. I was able to run the tests by changing the ABFS endpoint to dfs as opposed to blob as you suggested.

All test now pass for the branch except for one test which also fails on trunk: comparison, branch log, trunk log. Let me know if this is sufficient for the purposes of this pull request or if there's further testing/verification you'd like me to perform.

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java

DadanielZ

Looks good to me, there are minor checkstyle issues, once they are fixed I will help to commit it. Thanks.

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java

...ols/hadoop-azure/src/test/java/org/apache/hadoop/fs/azure/NativeAzureFileSystemBaseTest.java

...op-azure/src/test/java/org/apache/hadoop/fs/azurebfs/ITestAzureBlobFileSystemAttributes.java

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java

virajith · 2019-11-30T02:24:07Z

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java

too many line breaks in this method. Please remove the extra ones.

Done in 54dcb1d.

Re-activating this comment as i don't see this fixed.

I removed some more newlines in 6ecbdc0. I'm finding it a bit difficult to discern what style to follow since the file seems to include multiple styles for newlines (e.g. my original implementation was close to setOwner in indentation style). If you have any additional feedback on-top of what is implemented now, I'd appreciate more concrete pointers on style/formatting. Thanks in advance!

Sorry about nit-picking but it would be good to just follow the formatting in other methods in this file. For example, you don't need new lines in the catch block and also remove 3588.

Thanks for the pointers. I was previously following the style of setOwner. I removed the newlines you mentioned in f7c0d26.

...tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystemStore.java

virajith · 2019-11-30T02:36:36Z

...-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/AzureNativeFileSystemStore.java

Is there a reason for using a different encoding when compared to the AzureBlobFileSystemStore?

The documentation for Azure Data Lake Storage Gen2 (backing AzureBlobFileSystemStore) states that x-ms-properties must be encoded in ISO-8859-1 (see path/update#request-headers). The documentation for Azure Blob Storage (backing AzureNativeFileSystemStore) however only states that x-ms-meta must follow the conventions for C# identifiers (see set-blob-metadata#request-headers) which may contain Unicode letters and characters (see identifier-names). As such I believe that the two classes should use different encodings. What am I missing?

I was mostly curious about this one. I assume @DadanielZ can confirm this one?

This change is mostly looking good. If @DadanielZ can confirm the encoding issue, i can happy to commit this one.

Sorry for being away so long, yes, this is fine

hadoop-yetus · 2019-11-30T04:46:52Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 15s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	23m 14s	trunk passed
+1 💚	compile	0m 27s	trunk passed
+1 💚	checkstyle	0m 24s	trunk passed
+1 💚	mvnsite	0m 31s	trunk passed
+1 💚	shadedclient	14m 31s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 22s	trunk passed
+0 🆗	spotbugs	0m 54s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 52s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 26s	the patch passed
+1 💚	compile	0m 22s	the patch passed
+1 💚	javac	0m 22s	the patch passed
+1 💚	checkstyle	0m 18s	the patch passed
+1 💚	mvnsite	0m 26s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 51s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 20s	the patch passed
+1 💚	findbugs	0m 54s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 17s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		62m 15s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/3/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 299679ebd1f0 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6b2d6d4`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/3/testReport/
Max. process+thread count	295 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/3/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-11-30T04:52:58Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 47s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	21m 47s	trunk passed
+1 💚	compile	0m 26s	trunk passed
+1 💚	checkstyle	0m 22s	trunk passed
+1 💚	mvnsite	0m 30s	trunk passed
+1 💚	shadedclient	14m 49s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 21s	trunk passed
+0 🆗	spotbugs	0m 53s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 50s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 28s	the patch passed
+1 💚	compile	0m 24s	the patch passed
+1 💚	javac	0m 24s	the patch passed
+1 💚	checkstyle	0m 16s	the patch passed
+1 💚	mvnsite	0m 25s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 0s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 20s	the patch passed
+1 💚	findbugs	0m 53s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 17s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 29s	The patch does not generate ASF License warnings.
		61m 53s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/4/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux e1f92a9e389d 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6b2d6d4`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/4/testReport/
Max. process+thread count	306 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/4/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-11-30T04:58:33Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 35s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	18m 0s	trunk passed
+1 💚	compile	0m 31s	trunk passed
+1 💚	checkstyle	0m 26s	trunk passed
+1 💚	mvnsite	0m 34s	trunk passed
+1 💚	shadedclient	13m 45s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 26s	trunk passed
+0 🆗	spotbugs	0m 51s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 48s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 27s	the patch passed
+1 💚	compile	0m 23s	the patch passed
+1 💚	javac	0m 23s	the patch passed
+1 💚	checkstyle	0m 19s	the patch passed
+1 💚	mvnsite	0m 26s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	13m 33s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 23s	the patch passed
+1 💚	findbugs	0m 54s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 21s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 31s	The patch does not generate ASF License warnings.
		54m 44s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/7/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 79ceaaf65ff9 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6b2d6d4`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/7/testReport/
Max. process+thread count	412 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/7/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-11-30T05:18:13Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	23m 43s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 1s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	23m 12s	trunk passed
+1 💚	compile	0m 27s	trunk passed
+1 💚	checkstyle	0m 23s	trunk passed
+1 💚	mvnsite	0m 30s	trunk passed
+1 💚	shadedclient	14m 32s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 22s	trunk passed
+0 🆗	spotbugs	0m 53s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 51s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 31s	the patch passed
+1 💚	compile	0m 28s	the patch passed
+1 💚	javac	0m 28s	the patch passed
+1 💚	checkstyle	0m 18s	the patch passed
+1 💚	mvnsite	0m 27s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 37s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 20s	the patch passed
+1 💚	findbugs	0m 54s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 21s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 31s	The patch does not generate ASF License warnings.
		84m 50s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/5/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 24241755d1fe 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6b2d6d4`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/5/testReport/
Max. process+thread count	295 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/5/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-11-30T05:20:21Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	24m 12s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	22m 8s	trunk passed
+1 💚	compile	0m 31s	trunk passed
+1 💚	checkstyle	0m 22s	trunk passed
+1 💚	mvnsite	0m 31s	trunk passed
+1 💚	shadedclient	14m 28s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 26s	trunk passed
+0 🆗	spotbugs	0m 56s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 55s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s	the patch passed
+1 💚	compile	0m 25s	the patch passed
+1 💚	javac	0m 25s	the patch passed
+1 💚	checkstyle	0m 17s	the patch passed
+1 💚	mvnsite	0m 26s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	15m 10s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 20s	the patch passed
+1 💚	findbugs	0m 58s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 17s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 27s	The patch does not generate ASF License warnings.
		84m 44s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/6/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux e4cb54c0831e 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6b2d6d4`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/6/testReport/
Max. process+thread count	309 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/6/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

virajith · 2019-11-30T18:39:21Z

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azure/NativeAzureFileSystem.java

Re-activating this comment as i don't see this fixed.

hadoop-yetus · 2019-12-01T01:39:26Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	29m 55s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	20m 31s	trunk passed
+1 💚	compile	0m 28s	trunk passed
+1 💚	checkstyle	0m 21s	trunk passed
+1 💚	mvnsite	0m 31s	trunk passed
+1 💚	shadedclient	14m 26s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 22s	trunk passed
+0 🆗	spotbugs	0m 48s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 47s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 26s	the patch passed
+1 💚	compile	0m 22s	the patch passed
+1 💚	javac	0m 22s	the patch passed
+1 💚	checkstyle	0m 17s	the patch passed
+1 💚	mvnsite	0m 25s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 59s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 20s	the patch passed
+1 💚	findbugs	0m 53s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 17s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 27s	The patch does not generate ASF License warnings.
		88m 9s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/8/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux cc953942734b 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `6b2d6d4`
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/8/testReport/
Max. process+thread count	305 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/8/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-12-10T18:28:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 10s	Docker failed to build yetus/hadoop:104ccca9169.

Subsystem	Report/Notes
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/9/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2019-12-10T18:44:44Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	0m 0s	Docker mode activated.
-1 ❌	docker	0m 9s	Docker failed to build yetus/hadoop:104ccca9169.

Subsystem	Report/Notes
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/10/console
versions	git=2.17.1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

c-w · 2019-12-20T20:27:13Z

@DadanielZ Any chance you could take another look at this and provide some guidance on @virajith's question regarding encodings #452 (comment)? Would love to get this wrapped up and merged soon.

hadoop-yetus · 2020-01-09T17:35:05Z

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	1m 13s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 2 new or modified test files.
		_ trunk Compile Tests _
+1 💚	mvninstall	20m 59s	trunk passed
+1 💚	compile	0m 29s	trunk passed
+1 💚	checkstyle	0m 25s	trunk passed
+1 💚	mvnsite	0m 31s	trunk passed
+1 💚	shadedclient	14m 28s	branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 24s	trunk passed
+0 🆗	spotbugs	0m 50s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	0m 48s	trunk passed
		_ Patch Compile Tests _
+1 💚	mvninstall	0m 26s	the patch passed
+1 💚	compile	0m 23s	the patch passed
+1 💚	javac	0m 23s	the patch passed
+1 💚	checkstyle	0m 17s	the patch passed
+1 💚	mvnsite	0m 27s	the patch passed
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	shadedclient	14m 59s	patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	0m 20s	the patch passed
+1 💚	findbugs	0m 52s	the patch passed
		_ Other Tests _
+1 💚	unit	1m 17s	hadoop-azure in the patch passed.
+1 💚	asflicense	0m 28s	The patch does not generate ASF License warnings.
		60m 14s

Subsystem	Report/Notes
Docker	Client=19.03.5 Server=19.03.5 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-452/11/artifact/out/Dockerfile
GITHUB PR	#452
JIRA Issue	HADOOP-16005
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux 72a3738d9792 4.15.0-70-generic #79-Ubuntu SMP Tue Nov 12 10:36:11 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / `a40dc9e`
Default Java	1.8.0_232
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/11/testReport/
Max. process+thread count	305 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-452/11/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.11.1 https://yetus.apache.org

This message was automatically generated.

snvijaya

Changes look fine for AzureBlobFileSystem.

DadanielZ · 2020-01-15T01:33:51Z

@c-w, @virajith @snvijaya thanks for your efforts, I will commit it shortly.

Author: Gabor Szadovszky <[email protected]> Closes apache#452 from gszadovszky/PARQUET-1198 and squashes the following commits: 5b7ee44 [Gabor Szadovszky] PARQUET-1198: Bump java source and target to java8

DadanielZ requested changes Feb 5, 2019

View reviewed changes

c-w requested a review from DadanielZ November 26, 2019 23:24

DadanielZ reviewed Nov 28, 2019

View reviewed changes

hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/AzureBlobFileSystem.java Outdated Show resolved Hide resolved

DadanielZ reviewed Nov 28, 2019

View reviewed changes

virajith reviewed Nov 30, 2019

View reviewed changes

virajith requested changes Nov 30, 2019

View reviewed changes

c-w added 8 commits January 9, 2020 11:33

Add setXAttr and getXAttr to WASB and ABFS

081c8c2

Remove linebreaks

3cf5168

Replace try/catch with intercept

e37cdd9

Fix import order

20e5dde

Make parameters final

ae5e947

Encapsulate XMS_PROPERTIES_ENCODING

939c142

Remove more newlines

b66d6c9

Remove newlines

a167c87

snvijaya approved these changes Jan 14, 2020

View reviewed changes

DadanielZ approved these changes Jan 15, 2020

View reviewed changes

DadanielZ closed this Jan 15, 2020

c-w deleted the hadoop-16005 branch May 23, 2020 01:15

HADOOP-16005: Add XAttr support to WASB and ABFS #452

HADOOP-16005: Add XAttr support to WASB and ABFS #452

Uh oh!

Conversation

c-w commented Dec 22, 2018

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

c-w commented Nov 26, 2019

Uh oh!

DadanielZ commented Nov 26, 2019

Uh oh!

hadoop-yetus commented Nov 27, 2019

Uh oh!

c-w commented Nov 27, 2019

Uh oh!

Uh oh!

DadanielZ left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

c-w Nov 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Nov 30, 2019

Uh oh!

hadoop-yetus commented Nov 30, 2019

Uh oh!

hadoop-yetus commented Nov 30, 2019

Uh oh!

hadoop-yetus commented Nov 30, 2019

Uh oh!

hadoop-yetus commented Nov 30, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Dec 1, 2019

Uh oh!

hadoop-yetus commented Dec 10, 2019

Uh oh!

hadoop-yetus commented Dec 10, 2019

Uh oh!

c-w commented Dec 20, 2019

Uh oh!

hadoop-yetus commented Jan 9, 2020

Uh oh!

snvijaya left a comment

Choose a reason for hiding this comment

Uh oh!

DadanielZ commented Jan 15, 2020

Uh oh!

Reviewers

Assignees

c-w Nov 30, 2019 •

edited

Loading