Skip to content

Commit 3ea6fd0

Browse files
committed
[SPARK-22377][BUILD] Use /usr/sbin/lsof if lsof does not exists in release-build.sh
## What changes were proposed in this pull request? This PR proposes to use `/usr/sbin/lsof` if `lsof` is missing in the path to fix nightly snapshot jenkins jobs. Please refer #19359 (comment): > Looks like some of the snapshot builds are having lsof issues: > > https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.1-maven-snapshots/182/console > >https://amplab.cs.berkeley.edu/jenkins/view/Spark%20Packaging/job/spark-branch-2.2-maven-snapshots/134/console > >spark-build/dev/create-release/release-build.sh: line 344: lsof: command not found >usage: kill [ -s signal | -p ] [ -a ] pid ... >kill -l [ signal ] Up to my knowledge, the full path of `lsof` is required for non-root user in few OSs. ## How was this patch tested? Manually tested as below: ```bash #!/usr/bin/env bash LSOF=lsof if ! hash $LSOF 2>/dev/null; then echo "a" LSOF=/usr/sbin/lsof fi $LSOF -P | grep "a" ``` Author: hyukjinkwon <[email protected]> Closes #19695 from HyukjinKwon/SPARK-22377. (cherry picked from commit c8b7f97) Signed-off-by: hyukjinkwon <[email protected]>
1 parent d905e85 commit 3ea6fd0

File tree

1 file changed

+9
-2
lines changed

1 file changed

+9
-2
lines changed

dev/create-release/release-build.sh

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,13 @@ else
121121
fi
122122
fi
123123

124+
# This is a band-aid fix to avoid the failure of Maven nightly snapshot in some Jenkins
125+
# machines by explicitly calling /usr/sbin/lsof. Please see SPARK-22377 and the discussion
126+
# in its pull request.
127+
LSOF=lsof
128+
if ! hash $LSOF 2>/dev/null; then
129+
LSOF=/usr/sbin/lsof
130+
fi
124131

125132
if [ -z "$SPARK_PACKAGE_VERSION" ]; then
126133
SPARK_PACKAGE_VERSION="${SPARK_VERSION}-$(date +%Y_%m_%d_%H_%M)-${git_hash}"
@@ -337,7 +344,7 @@ if [[ "$1" == "publish-snapshot" ]]; then
337344
-DskipTests $PUBLISH_PROFILES clean deploy
338345

339346
# Clean-up Zinc nailgun process
340-
lsof -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
347+
$LSOF -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
341348

342349
rm $tmp_settings
343350
cd ..
@@ -375,7 +382,7 @@ if [[ "$1" == "publish-release" ]]; then
375382
-DskipTests $PUBLISH_PROFILES clean install
376383

377384
# Clean-up Zinc nailgun process
378-
lsof -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
385+
$LSOF -P |grep $ZINC_PORT | grep LISTEN | awk '{ print $2; }' | xargs kill
379386

380387
./dev/change-version-to-2.10.sh
381388

0 commit comments

Comments
 (0)