Skip to content

Switch celery to thread runners for dev for the Macs#5793

Merged
bjester merged 1 commit intolearningequality:unstablefrom
bjester:forking-macs
Mar 31, 2026
Merged

Switch celery to thread runners for dev for the Macs#5793
bjester merged 1 commit intolearningequality:unstablefrom
bjester:forking-macs

Conversation

@bjester
Copy link
Copy Markdown
Member

@bjester bjester commented Mar 30, 2026

Summary

Team members are having issues running the celery task workers. Investigation found issues with process forking:

On Apple Silicon + macOS 26, billiard's prefork pool is fundamentally unreliable regardless of what's loaded. The grpcio chase was a red herring; the SIGSEGV would likely happen even with a minimal task.

The fix is --pool=solo. Looking at the pnpm run celery script, it already uses -c 1 (single worker), so solo is equivalent in behavior — just without the fork. The simplest thing is to run it directly

References

https://learningequality.slack.com/archives/C0LK8QS9J/p1774881025194369

Reviewer guidance

  1. Start up the dev server and celery make devceleryworkers
  2. Create a channel
  3. Add some content to the channel
  4. Publish the channel
  5. Verify that the task workers ran the tasks

@bjester bjester requested a review from marcellamaki March 30, 2026 19:05
@bjester bjester merged commit 021c2f4 into learningequality:unstable Mar 31, 2026
22 checks passed
@bjester bjester deleted the forking-macs branch March 31, 2026 15:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants