Skip to content

Gracefully handle receipt of multiple termination signals#207

Merged
mperham merged 1 commit intosidekiq:masterfrom
dazuma:prevent-multiple-signals
May 21, 2012
Merged

Gracefully handle receipt of multiple termination signals#207
mperham merged 1 commit intosidekiq:masterfrom
dazuma:prevent-multiple-signals

Conversation

@dazuma
Copy link
Contributor

@dazuma dazuma commented May 21, 2012

This is a fix for an issue I came across yesterday when deploying a sidekiq worker as a daemon.

The issue is that when you send a SIGTERM or SIGINT to a running worker, it initiates shutdown by raising an Interrupt in the main thread. The shutdown process takes a finite amount of time because it waits for the manager to shut down, and then Celluloid registers an at_exit process that does its own cleanup of running actors. If another signal is received during that cleanup process, the cleanup itself is interrupted by an exception, and this causes the whole system to go crazy. In my case, the second signal was received during Celluloid's at_exit cleanup, and, on Mac OS Lion, this (for reasons I don't care to understand fully) caused the process to spin at 100% cpu until you kill -9 it.

This patch simply ensures that at most one Interrupt exception is raised as a result of receiving a termination signal.

mperham added a commit that referenced this pull request May 21, 2012
Gracefully handle receipt of multiple termination signals
@mperham mperham merged commit 8755dc0 into sidekiq:master May 21, 2012
@mperham
Copy link
Collaborator

mperham commented May 21, 2012

Awesome fix, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants