Skip to content

Commit 1768bd5

Browse files
WangTaoTheTonicsrowen
authored andcommitted
[SPARK-4832][Deploy]some other processes might take the daemon pid
Some other processes might use the pid saved in pid file. In that case we should ignore it and launch daemons. JIRA is down for maintenance. I will file one once it return. Author: WangTaoTheTonic <[email protected]> Author: WangTaoTheTonic <[email protected]> Closes #3683 from WangTaoTheTonic/otherproc and squashes the following commits: daa86a1 [WangTaoTheTonic] some bash style fix 8befee7 [WangTaoTheTonic] handle the mistake scenario cf4ecc6 [WangTaoTheTonic] remove redundant condition f36cfb4 [WangTaoTheTonic] some other processes might take the pid
1 parent 1c8633f commit 1768bd5

File tree

1 file changed

+11
-9
lines changed

1 file changed

+11
-9
lines changed

sbin/spark-daemon.sh

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -129,8 +129,9 @@ case $option in
129129
mkdir -p "$SPARK_PID_DIR"
130130

131131
if [ -f $pid ]; then
132-
if kill -0 `cat $pid` > /dev/null 2>&1; then
133-
echo $command running as process `cat $pid`. Stop it first.
132+
TARGET_ID="$(cat "$pid")"
133+
if [[ $(ps -p "$TARGET_ID" -o args=) =~ $command ]]; then
134+
echo "$command running as process $TARGET_ID. Stop it first."
134135
exit 1
135136
fi
136137
fi
@@ -141,7 +142,7 @@ case $option in
141142
fi
142143

143144
spark_rotate_log "$log"
144-
echo starting $command, logging to $log
145+
echo "starting $command, logging to $log"
145146
if [ $option == spark-submit ]; then
146147
source "$SPARK_HOME"/bin/utils.sh
147148
gatherSparkSubmitOpts "$@"
@@ -154,7 +155,7 @@ case $option in
154155
echo $newpid > $pid
155156
sleep 2
156157
# Check if the process has died; in that case we'll tail the log so the user can see
157-
if ! kill -0 $newpid >/dev/null 2>&1; then
158+
if [[ ! $(ps -p "$newpid" -o args=) =~ $command ]]; then
158159
echo "failed to launch $command:"
159160
tail -2 "$log" | sed 's/^/ /'
160161
echo "full log in $log"
@@ -164,14 +165,15 @@ case $option in
164165
(stop)
165166

166167
if [ -f $pid ]; then
167-
if kill -0 `cat $pid` > /dev/null 2>&1; then
168-
echo stopping $command
169-
kill `cat $pid`
168+
TARGET_ID="$(cat "$pid")"
169+
if [[ $(ps -p "$TARGET_ID" -o args=) =~ $command ]]; then
170+
echo "stopping $command"
171+
kill "$TARGET_ID"
170172
else
171-
echo no $command to stop
173+
echo "no $command to stop"
172174
fi
173175
else
174-
echo no $command to stop
176+
echo "no $command to stop"
175177
fi
176178
;;
177179

0 commit comments

Comments
 (0)