Skip to content

Conversation

pan3793
Copy link
Member

@pan3793 pan3793 commented Jul 9, 2024

Description of PR

JIRA: HADOOP-19222. Switch yum repo baseurl due to CentOS 7 sunset.

Similar to HADOOP-18151 (which handled sunset for CentOS 8), CentOS 7 reached EOL on July 1, 2024

How was this patch tested?

This change will trigger Yetus to run the native library building.

Without this change, the CentOS 7 CI fails with:

[2024-07-01T13:07:41.573Z] #9 [ 5/16] RUN yum update -y     && yum groupinstall -y "Development Tools"     && yum install -y         centos-release-scl         python3     && yum install -y $(pkg-resolver/resolve.py centos:7)
[2024-07-01T13:07:42.135Z] #9 0.452 Loaded plugins: fastestmirror, ovl
[2024-07-01T13:07:42.135Z] #9 0.570 Determining fastest mirrors
[2024-07-01T13:07:42.696Z] #9 0.824 Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=container error was
[2024-07-01T13:07:42.696Z] #9 0.824 14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error"
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826  One of the configured repositories failed (Unknown),
[2024-07-01T13:07:42.696Z] #9 0.826  and yum doesn't have enough cached data to continue. At this point the only
[2024-07-01T13:07:42.696Z] #9 0.826  safe thing yum can do is fail. There are a few ways to work "fix" this:
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826      1. Contact the upstream for the repository and get them to fix the problem.
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826      2. Reconfigure the baseurl/etc. for the repository, to point to a working
[2024-07-01T13:07:42.696Z] #9 0.826         upstream. This is most often useful if you are using a newer
[2024-07-01T13:07:42.696Z] #9 0.826         distribution release than is supported by the repository (and the
[2024-07-01T13:07:42.696Z] #9 0.826         packages for the previous distribution release still work).
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826      3. Run the command with the repository temporarily disabled
[2024-07-01T13:07:42.696Z] #9 0.826             yum --disablerepo=<repoid> ...
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826      4. Disable the repository permanently, so yum won't use it by default. Yum
[2024-07-01T13:07:42.696Z] #9 0.826         will then just ignore the repository until you permanently enable it
[2024-07-01T13:07:42.696Z] #9 0.826         again or use --enablerepo for temporary usage:
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826             yum-config-manager --disable <repoid>
[2024-07-01T13:07:42.696Z] #9 0.826         or
[2024-07-01T13:07:42.696Z] #9 0.826             subscription-manager repos --disable=<repoid>
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826      5. Configure the failing repository to be skipped, if it is unavailable.
[2024-07-01T13:07:42.696Z] #9 0.826         Note that yum will try to contact the repo. when it runs most commands,
[2024-07-01T13:07:42.696Z] #9 0.826         so will have to try and fail each time (and thus. yum will be be much
[2024-07-01T13:07:42.696Z] #9 0.826         slower). If it is a very temporary problem though, this is often a nice
[2024-07-01T13:07:42.696Z] #9 0.826         compromise:
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826             yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
[2024-07-01T13:07:42.696Z] #9 0.826 
[2024-07-01T13:07:42.696Z] #9 0.826 Cannot find a valid baseurl for repo: base/7/x86_64
[2024-07-01T13:07:43.258Z] #9 ERROR: process "/bin/bash -o pipefail -c yum update -y     && yum groupinstall -y \"Development Tools\"     && yum install -y         centos-release-scl         python3     && yum install -y $(pkg-resolver/resolve.py centos:7)" did not complete successfully: exit code: 1
[2024-07-01T13:07:43.258Z] ------
[2024-07-01T13:07:43.258Z]  > [ 5/16] RUN yum update -y     && yum groupinstall -y "Development Tools"     && yum install -y         centos-release-scl         python3     && yum install -y $(pkg-resolver/resolve.py centos:7):
[2024-07-01T13:07:43.258Z] 0.826 
[2024-07-01T13:07:43.258Z] 0.826      5. Configure the failing repository to be skipped, if it is unavailable.
[2024-07-01T13:07:43.258Z] 0.826         Note that yum will try to contact the repo. when it runs most commands,
[2024-07-01T13:07:43.259Z] 0.826         so will have to try and fail each time (and thus. yum will be be much
[2024-07-01T13:07:43.259Z] 0.826         slower). If it is a very temporary problem though, this is often a nice
[2024-07-01T13:07:43.259Z] 0.826         compromise:
[2024-07-01T13:07:43.259Z] 0.826 
[2024-07-01T13:07:43.259Z] 0.826             yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true
[2024-07-01T13:07:43.259Z] 0.826 
[2024-07-01T13:07:43.259Z] 0.826 Cannot find a valid baseurl for repo: base/7/x86_64
[2024-07-01T13:07:43.259Z] ------

For code changes:

  • Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
  • Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
  • If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
  • If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/1/console in case of problems.

cd /etc/yum.repos.d/ || exit &&
sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* &&
sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* &&
sed -i 's|# *baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-* &&
Copy link
Member Author

@pan3793 pan3793 Jul 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change is necessary, this makes it support both

#baseurl=http://mirror.centos.org/...
# baseurl=http://mirror.centos.org/...

@pan3793
Copy link
Member Author

pan3793 commented Jul 9, 2024

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 22m 25s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 14s Maven dependency ordering for branch
-1 ❌ mvninstall 25m 48s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 mvnsite 0m 0s trunk passed
-1 ❌ shadedclient 43m 44s branch has errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 8s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 1s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shadedclient 0m 45s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+0 🆗 asflicense 0m 23s ASF License check generated no output?
68m 43s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/1/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux ef7335c239a8 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 1f8517f
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/1/testReport/
Max. process+thread count 77 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/1/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@pan3793
Copy link
Member Author

pan3793 commented Jul 9, 2024

investigation:

Yetus failed to create user "jenkins" on building run-specific image yetus/hadoop:tp-11600
https://ci-hadoop.apache.org/blue/rest/organizations/jenkins/pipelines/hadoop-multibranch/branches/PR-6932/runs/1/nodes/56/log/

[2024-07-09T12:35:10.546Z] #8 [ 4/10] RUN if [ "998" != -1 ]; then (groupadd --non-unique --gid 998 dockersock && adduser "jenkins" dockersock ) || true; fi
[2024-07-09T12:35:10.546Z] #8 0.359 Usage: adduser [options] LOGIN
[2024-07-09T12:35:10.546Z] #8 0.359        adduser -D
[2024-07-09T12:35:10.546Z] #8 0.359        adduser -D [options]
[2024-07-09T12:35:10.546Z] #8 0.359 
[2024-07-09T12:35:10.546Z] #8 0.359 Options:
[2024-07-09T12:35:10.546Z] #8 0.359   -b, --base-dir BASE_DIR       base directory for the home directory of the
[2024-07-09T12:35:10.546Z] #8 0.359                                 new account
[2024-07-09T12:35:10.546Z] #8 0.359   -c, --comment COMMENT         GECOS field of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -d, --home-dir HOME_DIR       home directory of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -D, --defaults                print or change default useradd configuration
[2024-07-09T12:35:10.546Z] #8 0.359   -e, --expiredate EXPIRE_DATE  expiration date of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -f, --inactive INACTIVE       password inactivity period of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -g, --gid GROUP               name or ID of the primary group of the new
[2024-07-09T12:35:10.546Z] #8 0.359                                 account
[2024-07-09T12:35:10.546Z] #8 0.359   -G, --groups GROUPS           list of supplementary groups of the new
[2024-07-09T12:35:10.546Z] #8 0.359                                 account
[2024-07-09T12:35:10.546Z] #8 0.359   -h, --help                    display this help message and exit
[2024-07-09T12:35:10.546Z] #8 0.359   -k, --skel SKEL_DIR           use this alternative skeleton directory
[2024-07-09T12:35:10.546Z] #8 0.359   -K, --key KEY=VALUE           override /etc/login.defs defaults
[2024-07-09T12:35:10.546Z] #8 0.359   -l, --no-log-init             do not add the user to the lastlog and
[2024-07-09T12:35:10.546Z] #8 0.359                                 faillog databases
[2024-07-09T12:35:10.546Z] #8 0.359   -m, --create-home             create the user's home directory
[2024-07-09T12:35:10.546Z] #8 0.359   -M, --no-create-home          do not create the user's home directory
[2024-07-09T12:35:10.546Z] #8 0.359   -N, --no-user-group           do not create a group with the same name as
[2024-07-09T12:35:10.546Z] #8 0.359                                 the user
[2024-07-09T12:35:10.546Z] #8 0.359   -o, --non-unique              allow to create users with duplicate
[2024-07-09T12:35:10.546Z] #8 0.359                                 (non-unique) UID
[2024-07-09T12:35:10.546Z] #8 0.359   -p, --password PASSWORD       encrypted password of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -r, --system                  create a system account
[2024-07-09T12:35:10.546Z] #8 0.359   -R, --root CHROOT_DIR         directory to chroot into
[2024-07-09T12:35:10.546Z] #8 0.359   -P, --prefix PREFIX_DIR       prefix directory where are located the /etc/* files
[2024-07-09T12:35:10.546Z] #8 0.359   -s, --shell SHELL             login shell of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -u, --uid UID                 user ID of the new account
[2024-07-09T12:35:10.546Z] #8 0.359   -U, --user-group              create a group with the same name as the user
[2024-07-09T12:35:10.546Z] #8 0.359   -Z, --selinux-user SEUSER     use a specific SEUSER for the SELinux user mapping
[2024-07-09T12:35:10.546Z] #8 0.359 
[2024-07-09T12:35:10.546Z] #8 DONE 0.4s

this should be a Yetus side issue, and I suppose it has been broken for a while ...

the corresponding code should be https://github.com/apache/yetus/blob/23ddb4e106785effb2e1154aed65329f722bfabe/precommit/src/main/shell/test-patch-docker/Dockerfile.patchspecific#L31

due to the incorrectly setup for user "jenkins", the PATH env is abnormal, then commands like pwd, bash become unavailable ...

@pan3793
Copy link
Member Author

pan3793 commented Jul 9, 2024

I have less knowledge of Yetus, not sure how to fix and verify that.

For reviewers, this PR change is orthogonal to the Yetus issue, it fixes the CentOS 7 YUM repo resolving issues. I lean toward merging it first (though Yetus is not happy ...) and then fixing the Yetus issue later.

@steveloughran
Copy link
Contributor

not my area of knowledge at all...can anyone else take a look at this?

@slfan1989
Copy link
Contributor

@GauthamBanasandra Can you help take a look at this issue? Thank you very much!

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/2/console in case of problems.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/3/console in case of problems.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/4/console in case of problems.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 23s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 0m 20s Maven dependency ordering for branch
-1 ❌ mvninstall 0m 22s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 mvnsite 0m 0s trunk passed
-1 ❌ shadedclient 1m 9s branch has errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 19s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
-1 ❌ shadedclient 0m 30s patch has errors when building and testing our client artifacts.
_ Other Tests _
+0 🆗 asflicense 0m 21s ASF License check generated no output?
4m 10s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/2/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux b19e9b90f4b4 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 9122bef
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/2/testReport/
Max. process+thread count 46 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/2/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 0m 18s Maven dependency ordering for branch
-1 ❌ mvninstall 0m 21s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 mvnsite 0m 0s trunk passed
-1 ❌ shadedclient 1m 38s branch has errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 26s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shadedclient 0m 45s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+0 🆗 asflicense 0m 22s ASF License check generated no output?
4m 59s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/3/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux e137bdb7f29d 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 9122bef
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/3/testReport/
Max. process+thread count 46 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/3/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 33m 32s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 41s Maven dependency ordering for branch
+1 💚 mvninstall 33m 14s trunk passed
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 80m 38s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 34s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shadedclient 32m 12s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
149m 14s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/4/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux d11f19503b05 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 9122bef
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/4/testReport/
Max. process+thread count 551 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/4/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/5/console in case of problems.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 23s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 0m 19s Maven dependency ordering for branch
-1 ❌ mvninstall 0m 22s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 1m 25s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 18s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shadedclient 0m 41s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+0 🆗 asflicense 0m 21s ASF License check generated no output?
4m 38s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/5/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux b4a2ea50c53f 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 4d27539
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/5/testReport/
Max. process+thread count 9 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/5/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/6/console in case of problems.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 0m 18s Maven dependency ordering for branch
-1 ❌ mvninstall 0m 21s /branch-mvninstall-root.txt root in trunk failed.
+1 💚 mvnsite 0m 0s trunk passed
-1 ❌ shadedclient 1m 12s branch has errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 20s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
-1 ❌ shadedclient 0m 53s patch has errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 32s The patch does not generate ASF License warnings.
4m 46s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/6/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux e7cdbd6b4c11 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 00fafe5
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/6/testReport/
Max. process+thread count 51 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/6/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/console in case of problems.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 19s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 15s Maven dependency ordering for branch
+1 💚 mvninstall 19m 49s trunk passed
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 53m 6s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 22s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shadedclient 18m 24s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 25s The patch does not generate ASF License warnings.
73m 49s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux d08d49f502f9 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0a815cd
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/testReport/
Max. process+thread count 661 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/console in case of problems.

# don't let these tests cause -1s because we aren't really paying that
# much attention to them
YETUS_ARGS+=("--tests-filter=checkstyle")
YETUS_ARGS+=("--tests-filter=checkstyle,test4tests")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opened HADOOP-19223 (#6937) to make this change independently.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, test passed, let me revert this change.

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 6m 53s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 shelldocs 0m 1s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-0 ⚠️ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for branch
+1 💚 mvninstall 17m 20s trunk passed
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 19m 1s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 21s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 hadolint 0m 2s No new issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shellcheck 0m 0s No new issues.
+1 💚 shadedclient 19m 17s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 22s The patch does not generate ASF License warnings.
65m 7s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux 3ae1a61dc308 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 0a815cd
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/testReport/
Max. process+thread count 550 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/console
versions git=2.25.1 maven=3.6.3 hadolint=1.11.1-0-g0e692dd shellcheck=0.7.0
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Member

@GauthamBanasandra GauthamBanasandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM +1

@slfan1989
Copy link
Contributor

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/7/console in case of problems.

LGTM +1

It's still concerning when these situations arise. Previously, I've had to roll back PRs twice due to this issue.

# Apply the script again because centos-release-scl creates new YUM repo files
RUN pkg-resolver/set-vault-as-baseurl-centos.sh centos:7

# hadolint ignore=DL3008,SC2046
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I readed our previous Dockerfiles and found that DL3008 and SC2046 were not ignored. Why we should ignore these two rules here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hadoop-yetus
Copy link

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/8/console in case of problems.

@pan3793
Copy link
Member Author

pan3793 commented Jul 11, 2024

The last change after approval is d59e005, this patch is ready to go.

@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 20s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+0 🆗 hadolint 0m 0s hadolint was not available.
+0 🆗 shellcheck 0m 0s Shellcheck was not available.
+0 🆗 shelldocs 0m 0s Shelldocs was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
-1 ❌ test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
_ trunk Compile Tests _
+0 🆗 mvndep 14m 31s Maven dependency ordering for branch
+1 💚 mvninstall 19m 49s trunk passed
+1 💚 mvnsite 0m 0s trunk passed
+1 💚 shadedclient 53m 12s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 23s Maven dependency ordering for patch
+1 💚 mvninstall 0m 0s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 0m 0s the patch passed
+1 💚 shadedclient 18m 28s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 asflicense 0m 25s The patch does not generate ASF License warnings.
74m 3s
Subsystem Report/Notes
Docker ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/8/artifact/out/Dockerfile
GITHUB PR #6932
Optional Tests dupname asflicense codespell detsecrets hadolint shellcheck shelldocs mvnsite unit
uname Linux cc1fcf128ea2 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / d59e005
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/8/testReport/
Max. process+thread count 680 (vs. ulimit of 5500)
modules C: U:
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6932/8/console
versions git=2.9.5 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@slfan1989 slfan1989 merged commit 4f0ee9d into apache:trunk Jul 11, 2024
@slfan1989
Copy link
Contributor

@pan3793 Thanks for the contribution! @GauthamBanasandra Thanks for the review!

slfan1989 pushed a commit that referenced this pull request Jul 11, 2024
…Contributed by Cheng Pan.

Reviewed-by: Gautham Banasandra <[email protected]>
Signed-off-by:  Shilun Fan <[email protected]>
KeeProMise pushed a commit to KeeProMise/hadoop that referenced this pull request Sep 9, 2024
…6932) Contributed by Cheng Pan.

Reviewed-by: Gautham Banasandra <[email protected]>
Signed-off-by:  Shilun Fan <[email protected]>
Hexiaoqiao pushed a commit to Hexiaoqiao/hadoop that referenced this pull request Sep 12, 2024
…6932) Contributed by Cheng Pan.

Reviewed-by: Gautham Banasandra <[email protected]>
Signed-off-by:  Shilun Fan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants