-
Notifications
You must be signed in to change notification settings - Fork 656
Boot nginx from the backend app to fix graceful shutdown in production #3417
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
In production we aren't gracefully shutting down as expected. nginx does a quick shutdown when receiving a `SIGTERM` and once it exits, it appears that our process is aborted. Instead of spawning the backend from the shell script, the backend process is now responsible for spawning the shell script. This allows the backend to finish its shutdown work even if the script has already exited. `cat` is passed to the nginx buildpack script as our "server", which will block on `STDIN` so that `nginx` will run until receiving `SIGTERM`.
Deployed and tested on staging:
|
Thanks for the PR! Your approach works great, but I'm not sure spawning #!/bin/bash
set -ue
if [[ -z "${USE_FASTBOOT-}" ]]; then
unset USE_FASTBOOT
else
export USE_FASTBOOT
node --optimize_for_size --max_old_space_size=200 fastboot.js &
fi
# Since this script is launched separately, we tell the nginx
# buildpack (`bin/start-nginx`) that `cat` is our server.
bin/start-nginx cat &
target/release/server &
wait -n |
I haven't tried that exact script, but I don't think that will work. I tried a dozen different ways to launch it from the script, and couldn't get any of them to work. Launching I just couldn't convince bash to work. In particular,
I even had a few scripts that seemed to work fine locally, but just didn't work on Heroku. My guess is that maybe I was running into differences between interactive and non-interactive shells. (For example, I had to use As strange as it is to launch nginx from our backened app, it was the only approach I could find that ensured our process lives long enough to clean up. |
Ok then, let's get this fixed! @bors r+ |
📌 Commit 0061c5f has been approved by |
☀️ Test successful - checks-actions |
In production we aren't gracefully shutting down as expected. nginx
does a quick shutdown when receiving a
SIGTERM
and once it exits, itappears that our process is aborted.
Instead of spawning the backend from the shell script, the backend
process is now responsible for spawning the shell script. This allows
the backend to finish its shutdown work even if the script has already
exited.
cat
is passed to the nginx buildpack script as our "server", whichwill block on
STDIN
so thatnginx
will run until receivingSIGTERM
.r? @pietroalbini