Add fiber worker execution mode by crmne · Pull Request #728 · rails/solid_queue

crmne · 2026-04-04T14:02:58Z

Summary

Hi @rosa, I finally had some time to work on this after our earlier conversation about async worker execution mode.

This PR is a first implementation of async worker execution mode for Solid Queue. I'm also running benchmarks in parallel and can follow up with numbers once I have stable results.

Workers can now be configured with execution_mode: :async (or :fiber), which runs claimed jobs as fibers on a single async reactor thread and bounds concurrency with capacity / fibers instead of a thread pool.

This is separate from supervisor async mode. Supervisor mode still controls whether managed processes run in forks or threads; this change adds an async execution backend for workers themselves.

What Changed

added an execution pool abstraction for workers
introduced SolidQueue::ExecutionPools::AsyncPool
moved existing thread-pool behavior into SolidQueue::ExecutionPools::ThreadPool
added execution_mode: :async and :fiber as a configuration alias
added capacity / fibers as the clearer async-worker concurrency options
updated workers to build execution pools after fork/boot rather than during initialize
updated worker metadata to report execution mode, capacity, and inflight work across both backends
wake workers when capacity becomes available again so async execution integrates cleanly with the poll loop
treat async pool cancellation as fatal so reactor failures are surfaced instead of being silently ignored

Configuration / Validation

This PR also tightens async worker validation:

async workers require fiber-scoped isolated execution state
async workers fail early if the async gem is not available
async workers reject threads: and require capacity or fibers instead

Database pool guidance is now Rails-version-aware:

Rails 7.1 keeps the conservative async DB-pool validation
Rails 7.2+ uses the lower async-worker bound, since ordinary Active Record query paths can release connections between async waits

The README now documents the version-specific DB-pool guidance and calls out sticky Active Record APIs that can still pin connections.

Why

The goal is to support cooperative, mostly I/O-bound job execution with lower thread overhead and a clearer concurrency model, while keeping the feature explicit and safe to configure.

Tests

Added and updated coverage for:

async pool behavior
async worker execution
worker metadata and wake-up behavior
async boot/shutdown/error handling
configuration validation
DB-pool validation and docs around Rails version differences

seanharmer · 2026-04-04T14:30:16Z

I've pushed my initial implementation for this to #729 for comparison. Feel free to take anything you deem useful. :)

crmne · 2026-04-05T18:20:20Z

Hi @rosa, I've published a benchmark harness that answers three questions:

Within Solid Queue: how much do I/O-heavy workloads benefit from async execution compared to thread mode?
DB pool ceiling: at what concurrency does thread mode exhaust the database connection pool, and how much further does async go?
Across backends: how does Solid Queue compare to Async::Job + Redis when both run through ActiveJob?

https://github.com/crmne/solid_queue_bench

Solid Queue: async vs thread

async wins the majority of headline tests. Strongest gains by workload:

Workload	Best delta	Win rate
`sleep`	+27.2%	6/9
`async_http`	+26.0%	5/9
`cpu`	+5.1%	7/9
`ruby_llm_stream`	+20.2%	9/9

The cleanest result is ruby_llm_stream (real RubyLLM streaming + Turbo broadcasts) -- async wins every test. cpu staying roughly neutral is the expected control, which makes the I/O gains credible.

DB pool ceiling (stress suite)

The headline suite caps total concurrency to keep comparisons fair. The stress suite removes that cap. thread mode hit the DB pool wall after the baseline cap=25, proc=2 test and failed every higher-concurrency cell. async completed all 10/10 planned tests per workload -- it multiplexes fibers over a much smaller connection pool, so it survives where threads cannot.

Async::Job vs Solid Queue

Async::Job + Redis is faster across all shared tests (+7% to +213%), but that's a different backend entirely -- a throughput ceiling reference, not a same-backend comparison.

Bottom line

The main async win is not raw throughput. It's good I/O performance without thread-sized DB pools and the connection pressure that comes with them. Under stress, that difference becomes binary: threads fail, fibers keep going.

crmne · 2026-04-07T16:30:26Z

I'm running this code in production at Chat with Work right now. Switched from Async::Job largely because of operational advantages, especially visibility from Mission Control Jobs when debugging failures.

bradgessler · 2026-04-09T15:36:37Z

Also running this in production and can confirm it works very well.

trevorturk · 2026-04-09T19:04:21Z

I wanted to highlight something to make sure it's more visible to others... it looks like this setup would allow for separately configured worker pools, so one could imagine putting "mostly waiting on LLMs or slow APIs" jobs into an async pool, and the "regular" work could be left on a thread pool. For example, in the PR changes here, you can see a config example like so:

- queues: "llm*"
  execution_mode: async
- queues: [ real_time, background ]
  execution_mode: thread

crmne · 2026-04-17T16:14:30Z

Just a heads-up: I'm planning to write a blog post about this PR! :)

rosa · 2026-04-17T16:29:44Z

Hey @crmne, sorry for the delay here. I'm swamped with other work and haven't had the time to look into this at all. It's a big diff so I need time. I also need to figure out the naming around this, because we already have an async execution mode unrelated to this, so the naming needs to be adjusted.

I'm really thankful for the contribution, but I'm afraid I won't have time to review this in the near term.

Of course, you're welcome to write whatever blog post you want 😊

crmne · 2026-04-17T16:32:04Z

Hey @rosa, didn't mean to add any pressure - I totally get being swamped with other committments. Does execution_mode: :fiber work better for you? I can do the change pretty quickly and before the blog post goes out!

rosa · 2026-04-17T18:32:22Z

Oh, no, no worries! I didn't think you were trying to add pressure, it's just that I felt really bad about seemingly ignoring this 😅

Yes! :fiber is what I had in mind! But also execution_mode... not completely sure about that... 🤔 Thinking of another possible names that reflect what this is doing.

seanharmer · 2026-04-17T18:37:28Z

How about:

- queues: llm
      threads: 2
      concurrency_model: fiber
      fibers: 50
      polling_interval: 0.1

crmne · 2026-04-17T18:48:03Z

How about:

- queues: llm
      threads: 2
      concurrency_model: fiber
      fibers: 50
      polling_interval: 0.1

I like that, but mixing fibers and threads wouldn't make sense.

seanharmer · 2026-04-17T18:52:01Z

How about:
- queues: llm
      threads: 2
      concurrency_model: fiber
      fibers: 50
      polling_interval: 0.1
I like that, but mixing fibers and threads wouldn't make sense.

Right, sorry, that was a copy/paste from my early implementation. If this one doesn't expose threads, just ignore that line. :)

rosa · 2026-04-17T18:53:13Z

Maybe it could just be threads: N or fibers: N, make it so that only one is allowed, and that determines whether you use threads or fibers?

crmne · 2026-04-17T18:55:04Z

Maybe it could just be threads: N or fibers: N, make it so that only one is allowed, and that determines whether you use threads or fibers?

I like it. Will implement that during the weekend.

crmne · 2026-04-19T14:20:55Z

I updated it accordingly, avoiding the need of a separate execution_mode config option.

I also renamed the worker implementation/docs away from "async", using "fiber" and "worker concurrency model" to avoid confusion.

crmne · 2026-04-21T08:18:25Z

https://paolino.me/solid-queue-doesnt-need-a-thread-per-job/ in case people are interested

gobijan · 2026-04-21T10:03:54Z

+          ActiveSupport::IsolatedExecutionState.isolation_level == :fiber
+        end
+
+        def ensure_io_timeout_compatibility!(io_class = IO)


Not sure about monkey patching here. Maybe just raise the required ruby version?

gobijan · 2026-04-21T10:17:14Z

+
+            # Older versions of the async gem don't support waking the reactor from another
+            # thread reliably, so we cooperatively poll for newly posted work.
+            sleep(IDLE_WAIT_INTERVAL) if pending_executions.empty?


This creates a busy-poll loop at 100hz per worker?
Would it be better to raise the min required async gem version to async >= 2.29 (thread safety improvements) so you can replace the poll loop with Async::Notification or Async::Queue?

gobijan

Raise async gem version to avoid busy-poll loop?

ollym · 2026-05-23T10:17:49Z

Similarly would be cool to see this work be introduced into Rails core so we could set config.active_record.async_query_executor = :global_fiber_pool for async queries. Good work otherwise, looking forward to seeing this and batch support merged.

rsamoilov

In my opinion, if the configuration is called fibers, it needs to be backend-agnostic. Which means checking Fiber.scheduler instead of defined?(Async), using Fiber.schedule {} instead of Async {}, and SizedQueue instead of Async::Semaphore - thus completely relying on the standard library and being implementation-agnostic.

Good design is the one that survives change, but this executor can't survive change by definition, as it locks the implementation to a specific backend. Async is the de facto standard, but there are other implementations, and hopefully there will be many more. IMO using Async belongs to the application layer, and tying the executor to the specific implementation of the scheduler would lock users out from using other (perfectly functional) implementations in the future. To me, this coupling doesn't make sense in the framework code, especially when the standard library exposes clean hooks that allow the executor to remain implementation-agnostic.

Essentially, I'd suggest either using a backend-agnostic implementation or updating the fibers configuration name to specifically point out that it's powered by Async.

rosa · 2026-06-16T14:16:03Z

That's a good point, @rsamoilov 🤔

will-s-stone · 2026-06-30T14:08:45Z

Were there any updates on the ETA for this PR? It would be really helpful to have this functionality!

rosa · 2026-06-30T15:07:16Z

Hey @will-s-stone, no ETA. I'm still swamped with other work and not sure when I'll have time to work on this.

abuisman · 2026-07-06T09:33:05Z

This seems like exactly what we need. Thanks @crmne for the work!

rosa · 2026-07-20T11:04:50Z

Hey! I'm finally catching up with PRs and issues this week, so I'll get to this one in the coming days. Thanks @crmne and everyone else for your patience 🙏🏻

rosa · 2026-07-22T16:13:19Z

+
+        def ensure_io_timeout_compatibility!(io_class = IO)
+          unless io_class.method_defined?(:timeout) && io_class.method_defined?(:timeout=)
+            # Async 2.24, which Ruby 3.1 resolves to, expects Ruby's newer IO


I think I'm going to stop supporting Ruby 3.1. It's time, it's been EOL since 2025-03-26. In that way, we can simplify this I think.

rosa · 2026-07-22T17:00:36Z

@rsamoilov, @crmne, I've thought about the explicit Async vs backend-agnostic question, and I agree with @crmne here. Async is already tested and benchmarked for this PR, so I think we should go with it. The only exposed configuration, fibers: N, doesn't prevent going for that backend-agnostic approach in the future if needed, so that people can use other backends when they're available. If/when this is needed, we can work on that in a separate, future PR. If this PR made that impossible or a breaking change or something like that, it'd be different but it doesn't seem to be the case.

Thanks a lot for your perspective, @rsamoilov, really appreciate it! I don't work with fibers, so your insight has been very useful for me.

Workers can now be configured with `fibers: N` as an alternative to `threads: N` (specifying both is invalid). In fiber mode, claimed jobs execute as fibers on a single Async reactor thread, bounded by the configured fiber count. This suits mostly I/O-bound jobs, where cooperative scheduling allows much higher in-flight capacity with fewer database connections than a thread pool. The existing pool was extracted into ExecutionPools::ThreadPool (SolidQueue::Pool remains as an alias), with the new ExecutionPools::FiberPool sharing the same interface. Pools are built on worker boot, after forking, since a reactor thread wouldn't survive a fork. Fiber mode requires the async gem (>= 2.24), which is not a runtime dependency: workers configured with `fibers` raise a clear error if it isn't available. It also requires fiber-scoped isolated execution state (ActiveSupport::IsolatedExecutionState.isolation_level = :fiber), validated at boot and configuration-check time. The database pool size guidance is fiber-aware and Rails-version aware: on Rails 7.2+, fibers release connections between queries, so a fiber worker needs far fewer connections than its fiber count.

Now that the minimum supported Ruby version is 3.2, IO#timeout, IO#timeout= and IO::TimeoutError, which async expects, are always available natively, and we no longer need to patch them in. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The fiber pool's reactor looped with a 10ms sleep to check for newly posted work, and again while waiting for in-flight executions to finish during shutdown, on the assumption that the async gem can't reliably wake the reactor from another thread. It can: Thread::Queue#pop is fiber-scheduler-aware, so a fiber blocking on it suspends properly while other fibers run, and a push from the poller thread wakes the reactor through the scheduler (via Scheduler#unblock and the selector's cross-thread wakeup, reliable since io-event 1.3, which all supported async versions require). This removes the idle CPU cost of the busy-poll and the up-to-10ms latency between claiming a job and executing it. Shutdown now closes the queue, which wakes the blocked pop, drains any already-queued executions and then returns nil, ending the loop. The separate wait for in-flight executions is unnecessary: they run in fibers that are children of the reactor's top-level task, which already waits for all its children before exiting. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Cover the full lifecycle of a supervised, forked fiber worker: jobs executing in fibers, graceful termination draining in-flight jobs, and abrupt termination releasing claimed jobs. The unit tests exercise the fiber pool in-process only, which wouldn't catch a pool built before forking, whose reactor thread wouldn't survive the fork. Also restore the generic first worker in the configuration example, adding the fiber worker as an extra entry instead of replacing it, and note that the fiber isolation level is an application-wide setting, not scoped to Solid Queue's processes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rosa · 2026-07-23T06:51:34Z

+
+      def validate_execution_options!(options)
+        if options.key?(:threads) && options.key?(:fibers)
+          raise ArgumentError, "Workers can specify either `threads` or `fibers`, but not both."


I think this belongs in the configuration validation.

rosa · 2026-07-23T07:32:12Z

+      def mark_metadata_dirty
+        metadata_state_mutex.synchronize { @metadata_dirty = true }
+      end
+
+      def metadata_state_mutex
+        @metadata_state_mutex
+      end
+
+      def reload_metadata_if_needed(executions_claimed)
+        needs_reload = metadata_state_mutex.synchronize do
+          claimed_or_dirty = executions_claimed || @metadata_dirty
+          @metadata_dirty = false
+          claimed_or_dirty
+        end
+
+        reload_metadata if needs_reload


I think we can simplify all this, and just rely on ClaimedExecution records, instead of having to keep this denormalized metadata up to date. I don't think it's worth it. I'll push some changes to this.

Nothing references it anymore, inside or outside the gem: it was internal API, always instantiated by the worker and never something apps would configure or subclass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Keeping the inflight gauge fresh on every claim and job completion required a mutex and dirty-tracking in the worker, an extra metadata update per claiming poll (up to 10 per second per worker at the default polling interval), and had the poller and heartbeat threads racing to update the same row. The gauge doesn't need that freshness: it's a monitoring convenience, the exact count is always available from claimed executions, and the heartbeat already refreshes it with bounded staleness. With the dirty-tracking gone, the pool callback's only job is waking the worker when capacity frees up, so it also gets its old name back: on_idle. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The configuration already validates that workers specify either threads or fibers but not both, and that's where option validation belongs; the worker doesn't validate any of its other options. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

In the default fork supervisor mode, workers are instantiated by the supervisor before forking, so a reactor thread started when the pool is built wouldn't survive the fork. Starting it when the first execution is posted makes the fiber pool safe to build at any point, matching the thread pool, which already builds its executor lazily on first use. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Build the execution pool directly when the worker is initialized, which is safe now that both pools start their threads lazily. Merge the worker defaults unconditionally and choose the pool type with a single check on the fibers option, relying on the configuration's validation to prevent threads and fibers from being combined. If a worker is instantiated directly with no pool options, it defaults to a thread pool; if it somehow gets both options, fibers win. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The in-flight gauge was refreshed on heartbeats only, so it could show counts that were stale by up to a heartbeat interval; better not to show it than to show stale information, especially given the exact count is always available from claimed executions. The pool type and size are constant, and the pool knows both, so the worker can store them directly when registering, as it did before, and the pools don't need to provide changing metadata at all. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The rescue inside the future covers job execution, but an error raised when restoring capacity or waking up the worker would escape it and reject the future with nobody watching. Attach the on_rejection callback the old pool had as a backstop, so those errors are also reported via handle_thread_error. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Instead of tracking whether capacity had been reserved with a local flag, scope the rescue to the code that runs after the reservation, which is the only part that needs to restore it on failure. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Both pools share the capacity ledger: the reserve/restore accounting, the idle notification, and the reserve-then-schedule shape of post. Move all of that to ExecutionPools::Base, leaving each pool with just its scheduling mechanism: the thread pool schedules executions on a Concurrent::ThreadPoolExecutor via schedule, and the fiber pool hands them to its reactor, adding its fatal-error and shutdown guards on top of the base's post. The default way of performing an execution, wrapped in the app executor, reporting errors and restoring capacity, also lives in the base class; the fiber pool overrides it to treat Async::Stop as fatal. The pool type is derived from the class name, and the factory performs the lookup in reverse, trusting the type it receives: the worker only ever passes :thread or :fiber. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

The async dependency and isolation level checks were surfaced through Configuration's validations but implemented in the fiber pool, which also ran them when built. Configuration is where these belong: it validates every supervised path before any process is instantiated, and reports through valid? and check. The pool now just requires the async gem lazily when starting its reactor, keeping setups without fiber workers from ever loading it, and otherwise trusts its caller, like the rest of the internal classes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Replace the ExecutionPools module with a Pool abstract class that the thread and fiber pools inherit from, and that builds the right pool for a worker, mirroring how Supervisor relates to ForkSupervisor and AsyncSupervisor. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rosa · 2026-07-23T10:12:15Z

@crmne, I prepared a few changes on top of your work. It was getting too big to just push it here as I did with a few smaller changes before, so I've opened crmne#1 for you to check. Core functionality doesn't change except for some defensive code and nicer errors for private APIs, which I think we can do without. The largest change is removing the dynamic metadata from workers and the need to keep the inflight capacity in sync.
The rest is mostly refactoring and reorganizing code. Feel free to review that one and let me know if you have objections over on that PR. I'll tidy up the history once everything is over here, before merging.

Simplify the worker and reorganize the execution pools

crmne · 2026-07-29T16:03:36Z

Thanks, @rosa! I reviewed and merged the changes from crmne#1. No objections from me. I left a more detailed note there, and this PR now includes the updated implementation. Thanks again for taking the time to work through it so carefully.

This was referenced Apr 4, 2026

Support for async streaming in Solid Queue for AI response handling #596

Closed

How to make solid_queue work well with fibers? #150

Closed

Optimizing for used Connections #627

Open

seanharmer mentioned this pull request Apr 4, 2026

Introduce async fiber-based worker mode for IO-bound jobs #729

Open

mhenrixon mentioned this pull request Apr 10, 2026

feat(process): add async/fiber execution mode for workers mhenrixon/pgbus#112

Merged

5 tasks

dbackeus mentioned this pull request Apr 11, 2026

There should be a way to bypass the connection pool vs worker threads check #736

Closed

crmne changed the title ~~Add async worker execution mode~~ Add fiber worker execution mode Apr 19, 2026

gobijan reviewed Apr 21, 2026

View reviewed changes

MadBomber mentioned this pull request Apr 29, 2026

Add RobotLab::Job base class for fiber-safe Solid Queue integration MadBomber/robot_lab#20

Closed

7 tasks

rsamoilov reviewed Jun 16, 2026

View reviewed changes

rosa reviewed Jul 22, 2026

View reviewed changes

crmne and others added 4 commits July 22, 2026 20:48

rosa force-pushed the async-worker-execution-mode branch from 2f845aa to 3559d3e Compare July 22, 2026 19:13

rosa reviewed Jul 23, 2026

View reviewed changes

rosa and others added 2 commits July 23, 2026 09:35

Remove the SolidQueue::Pool alias

34990a6

Nothing references it anymore, inside or outside the gem: it was internal API, always instantiated by the worker and never something apps would configure or subclass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rosa mentioned this pull request Jul 23, 2026

Simplify the worker and reorganize the execution pools crmne/solid_queue#1

Merged

rosa and others added 9 commits July 23, 2026 10:01

trevorturk mentioned this pull request Jul 25, 2026

feat: add config.hub_isolation_level for fiber-safe hub storage (Falcon/async) getsentry/sentry-ruby#3018

Merged

Merge pull request #1 from rails/fiber-worker-refinements

eb3e068

Simplify the worker and reorganize the execution pools

crmne force-pushed the async-worker-execution-mode branch from a25ad3a to eb3e068 Compare July 29, 2026 15:59

Uh oh!

Conversation

crmne commented Apr 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What Changed

Configuration / Validation

Why

Tests

Uh oh!

seanharmer commented Apr 4, 2026

Uh oh!

crmne commented Apr 5, 2026

Solid Queue: async vs thread

DB pool ceiling (stress suite)

Async::Job vs Solid Queue

Bottom line

Uh oh!

crmne commented Apr 7, 2026

Uh oh!

bradgessler commented Apr 9, 2026

Uh oh!

trevorturk commented Apr 9, 2026

Uh oh!

crmne commented Apr 17, 2026

Uh oh!

rosa commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

crmne commented Apr 17, 2026

Uh oh!

rosa commented Apr 17, 2026

Uh oh!

seanharmer commented Apr 17, 2026

Uh oh!

crmne commented Apr 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seanharmer commented Apr 17, 2026

Uh oh!

rosa commented Apr 17, 2026

Uh oh!

crmne commented Apr 17, 2026

Uh oh!

crmne commented Apr 19, 2026

Uh oh!

crmne commented Apr 21, 2026

Uh oh!

gobijan Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

gobijan Apr 21, 2026

Choose a reason for hiding this comment

Uh oh!

gobijan left a comment

Choose a reason for hiding this comment

Uh oh!

ollym commented May 23, 2026

Uh oh!

rsamoilov left a comment

Choose a reason for hiding this comment

Uh oh!

rosa commented Jun 16, 2026

Uh oh!

will-s-stone commented Jun 30, 2026

Uh oh!

rosa commented Jun 30, 2026

Uh oh!

abuisman commented Jul 6, 2026

Uh oh!

rosa commented Jul 20, 2026

Uh oh!

rosa Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

rosa commented Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rosa Jul 23, 2026

Choose a reason for hiding this comment

crmne commented Apr 4, 2026 •

edited

Loading

rosa commented Apr 17, 2026 •

edited

Loading

crmne commented Apr 17, 2026 •

edited

Loading

rosa commented Jul 22, 2026 •

edited

Loading

rosa commented Jul 23, 2026 •

edited

Loading