Fix standalone server not catching SIGINT/SIGTERM - #3799
Merged
vigoo merged 8 commits intoSep 30, 2026
Merged
Conversation
✅ Deploy Preview for golemcloud canceled.
|
|
✅ All contributors have signed the CLA. |
|
Hi @baltasarblanco, thanks for your interest in contributing! This project requires that pull request authors are vouched, and you are not in the list of vouched users. This PR will be closed automatically. See https://github.com/golemcloud/golem/blob/main/CONTRIBUTING.md for more details. |
Contributor
Author
|
I have read the CLA Document and I hereby sign the CLA |
vigoo
reviewed
Sep 2, 2026
Amp-Thread-ID: https://ampcode.com/threads/T-01a0f15c-90c7-7576-bf7c-09ae4b194634 Co-authored-by: Amp <amp@ampcode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1603
Problem
golem server runnever installs a signal handler — the run path just awaits the services'JoinSetuntil completion. Running natively that is invisible: with no handler installed, the kernel applies SIGINT's default disposition and the process dies. But when the standalone server runs as PID 1 in a container (the Docker repro in the issue), the kernel only delivers a signal to a PID namespace's init process if init has installed a handler for it — default dispositions are never applied (pid_namespaces(7)). So SIGINT (Ctrl+C,kill -INT 1) and SIGTERM (docker stop) are silently discarded, which matches every observation in the issue thread.Fix
Wrap the
join_next()loop inServerSubcommand::Runin atokio::select!against a shutdown signal (SIGINT + SIGTERM viatokio::signal::unixon Unix,ctrl_c()elsewhere). On signal: log,join_set.shutdown().await, and return. Aborting the tasks drops the router task, which owns the worker executor'sRunDetails; itsDropcancels the graph-wide shutdown token and stops the epoch thread — the same mechanism already documented incli/golem/src/router.rs("Dropping its RunDetails cancels the shutdown token that drives background services").One file changed, no new dependencies (
tokioalready has thesignalfeature enabled workspace-wide).Verification
Reproduced the PID 1 condition without Docker, since the bug's trigger is being the init of a PID namespace:
sudo unshare --fork --pid --mount-proc ./target/debug/golem server run -vv.Before (main):
kill -INT <pid>kills it instantly (kernel default action, no shutdown of any kind).kill -INTis silently ignored; the server keeps running and can only be stopped with SIGKILL.After (this branch):
Received shutdown signal, stopping Golem server, with exit code 0.Shutdown requested, stopping scheduler background loopbefore the process exits.cargo build -p golemandcargo make fixpass.Note: on unpatched versions,
docker run --initworks around the issue, since tini becomes PID 1 and forwards signals.