Skip to content

Conversation

@mehakmeet
Copy link
Contributor

Description of PR

Adding toggleable support for modification time during distcp -update between two stores with incompatible checksum comparison.

How was this patch tested?

Compiled and ran the added tests on ABFS and S3A.

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

…for file skip. (apache#5308)


Adding toggleable support for modification time during distcp -update between two stores with incompatible checksum comparison.

Contributed by: Mehakmeet Singh <[email protected]>
@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 10m 29s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 2 new or modified test files.
_ branch-3.3 Compile Tests _
+1 💚 mvninstall 43m 4s branch-3.3 passed
+1 💚 compile 0m 26s branch-3.3 passed
+1 💚 checkstyle 0m 27s branch-3.3 passed
+1 💚 mvnsite 0m 32s branch-3.3 passed
+1 💚 javadoc 0m 30s branch-3.3 passed
+1 💚 spotbugs 0m 55s branch-3.3 passed
+1 💚 shadedclient 28m 23s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 34s the patch passed
+1 💚 compile 0m 21s the patch passed
+1 💚 javac 0m 21s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 14s the patch passed
+1 💚 mvnsite 0m 23s the patch passed
+1 💚 javadoc 0m 16s the patch passed
+1 💚 spotbugs 0m 48s the patch passed
+1 💚 shadedclient 27m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 16m 3s hadoop-distcp in the patch passed.
+1 💚 asflicense 0m 33s The patch does not generate ASF License warnings.
132m 58s
Subsystem Report/Notes
Docker ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5387/1/artifact/out/Dockerfile
GITHUB PR #5387
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux a974ffad64bf 4.15.0-200-generic #211-Ubuntu SMP Thu Nov 24 18:16:04 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3 / 399de84
Default Java Private Build-1.8.0_352-8u352-ga-1~18.04-b08
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5387/1/testReport/
Max. process+thread count 618 (vs. ulimit of 5500)
modules C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5387/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@mehakmeet mehakmeet merged commit a2ceb09 into apache:branch-3.3 Feb 14, 2023
jojochuang pushed a commit to jojochuang/hadoop that referenced this pull request May 23, 2023
…le checking for file skip. (apache#5387)

Adding toggleable support for modification time during distcp -update between two stores with incompatible checksum comparison.

    Also Contains: "HADOOP-18633. fix test AbstractContractDistCpTest#testDistCpUpdateCheckFileSkip."
    Also Contains: "CDPD-28513. HADOOP-17628. Distcp contract test is really slow with ABFS and S3A; timing out. (apache#3240)" to ensure correctness in the tests.

Change-Id: I5ce8d630396c8ea525195841a5e3d3f6aaea7455
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants