LaunchKit
← All posts
· 14 min read · by The LaunchKit team · 3 views

Solid Queue vs Sidekiq

Rails 8 picked a background job backend for you. The Solid Queue README states it without hedging: "Solid Queue is configured by default in new Rails 8 applications." Sidekiq has answered this question since 2012, it is genuinely faster per job, and its web UI ships in the free gem. So the real question is not whether to adopt Solid Queue. It is whether you have a reason to take it back out.

The versions under discussion are solid_queue 1.7.0, released 2026-08-21 under MIT, and sidekiq 8.1.7, released 2026-08-17 under LGPL-3.0. Everything below comes from those two codebases, their READMEs and Sidekiq's own pricing page.

What each one makes you run

Sidekiq 8 needs a Redis. The README is specific about which: "Redis 7.0+, Valkey 7.2+ or Dragonfly 1.27+", on "MRI 3.2+ or JRuby 9.4+", with "Rails and Active Job 7.0+". On top of that you run a sidekiq process. On Heroku that is a second dyno; under Kamal it is a second role; in docker-compose it is a second service. Two processes and a datastore you would not otherwise have.

Solid Queue needs a database that you already run, and a supervisor. The supervisor can be its own process through bin/jobs, or it can live inside your web server through the Puma plugin:

# config/puma.rb
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"]

That one line is the whole "one fewer server" argument, and it comes with a restriction the README names directly: "phased restarts are not supported currently because the plugin requires app preloading to work". If your deploy is a phased restart today, the Puma plugin changes your deploy, not just your Procfile. In fork mode the plugin forks a process per worker and dispatcher, which is the recommended setting; solid_queue_mode :async keeps them in the Puma process and silently ignores your processes: configuration.

The wider version of this trade, covering cache and WebSockets as well as jobs, is in Running Rails 8 without Redis.

Where a job waits between enqueue and pickup

Sidekiq blocks. Sidekiq::BasicFetch#retrieve_work calls conn.blocking_call(TIMEOUT, "brpop", *qs, TIMEOUT) with TIMEOUT = 2, so a job pushed to Redis is handed to a waiting thread as soon as it lands. Latency between enqueue and pickup is a network hop.

Solid Queue polls. Workers run one of exactly two queries against solid_queue_ready_executions, quoted in full in the README:

SELECT job_id
FROM solid_queue_ready_executions
ORDER BY priority ASC, job_id ASC
LIMIT ?
FOR UPDATE SKIP LOCKED;

The second form adds WHERE queue_name = ?. SKIP LOCKED is what lets several workers poll the same table without queueing behind each other's row locks, and it is why the README asks for PostgreSQL 9.5+, MySQL 8+ or MariaDB 10.6+.

The gap between those two designs is a poll interval. Solid Queue's library default for workers is 0.1 seconds, but the file the installer writes is not the library default: lib/generators/solid_queue/install/templates/config/queue.yml sets polling_interval: 1 for the worker and threads: 3. A fresh Rails 8 app therefore has up to a second of latency on an empty queue, which nobody told you about and which is one line to change. For a welcome email that is invisible. For a job whose result a user is watching a spinner for, it is the whole complaint.

Throughput, and the point where the database becomes the limit

Solid Queue's ceiling is higher than the usual objection assumes. 37signals published their numbers with the 1.0 release on 2024-09-26: "Currently in HEY we're processing about 20 million jobs per day, using 800 workers, 4 dispatchers and 2 schedulers, spread over 74 VMs running in two datacenters", with Basecamp 4 still to come at roughly four times that volume. Twenty million jobs a day is not a toy.

Read the rest of that sentence, though. Seventy four VMs. At that size the database backing the queue is a database sized for a queue, and the "one fewer server" argument that sold you the design is gone. What replaced it is a different argument, that the queue is transactional and backed up with your data, which is a real argument but not the same one.

Between those poles, the pressure shows up as connections. The README recommends setting threads "less than or equal to the queue database's connection pool size minus 2, as each worker uses connections for polling and heartbeat". Every worker process is holding connections permanently, polling on a timer, whether or not there is work. Postgres does not care about your job count, it cares about concurrent connections and write volume, and a queue is a write-heavy table where every job is an insert, an update, a delete and, if you keep them, a row that lives for a day.

One configuration choice costs more than it looks. Naming queues by prefix, queues: beta*, makes Solid Queue first ask which queues exist, which PostgreSQL answers with a recursive CTE emulating a loose index scan. Paused queues trigger the same lookup. The README's own summary: "if you want to ensure optimal performance on polling, the best way to do that is to always specify exact names for them, and not have any queues paused."

The dashboard question, and what Mission Control actually is

Solid Queue ships no user interface at all. The README's entire answer is a recommendation: "we recommend taking a look at mission_control-jobs, a dashboard where, among other things, you can examine and retry/discard failed jobs." Mission Control Jobs is a separate Rails engine under the rails organisation, supporting Resque and Solid Queue, mounted by you:

mount MissionControl::Jobs::Engine, at: "/jobs"

It includes HTTP basic auth, disabled until you set mission_control.http_basic_auth_user and http_basic_auth_password in credentials, and you can swap the whole thing for your own base_controller_class. So the honest comparison is: Sidekiq gives you a queue and a web UI in one gem line, and Solid Queue gives you a queue plus a second gem, a route, a credential and a decision about who can see it.

That gap is real enough that this site's own product runs Solid Queue with no dashboard mounted at all, and inspects failures from a console. Not a recommendation, just what happens by default when nobody makes the decision.

Retries and the shape of a failure

Sidekiq owns its retry policy. The default is 25 retries spread over roughly 20 days, with the delay computed as (retry_count ** 4) + 15 + (rand(10) * (retry_count + 1)) seconds, giving 15, 16, 31, 96, 271 and so on. After the last one the job moves to the Dead set, "limited by default to 10,000 jobs or 6 months so it doesn't grow infinitely".

Solid Queue owns none of it. The README is blunt: "Solid Queue doesn't include any automatic retry mechanism, it relies on Active Job for this." Your retry_on and discard_on declarations are the policy, which means the behaviour is identical on the test adapter, on :async in development, and in production. A job that exhausts its Active Job retries leaves a row in solid_queue_failed_executions, and the recovery is a console:

failed_execution = SolidQueue::FailedExecution.find(...)
failed_execution.error
failed_execution.retry   # re-enqueued as if new
failed_execution.discard # deleted

Which of these you prefer is mostly a question of where you want the policy written. Sidekiq's schedule is well tuned and applies to everything, including the jobs where you never thought about it. Active Job's is nothing until you write it, which is worse on the day you forget and better on the day you need a job to stop after two attempts instead of twenty five.

What happens when the worker is killed

A SIGKILL is where the two designs separate hardest, and it is the failure that actually happens, because the container runtime does it when a job exhausts the memory limit. Sidekiq's own reliability page states the outcome for the free gem: "if Sidekiq crashes while processing that job, it is lost forever." The fix, super_fetch, which keeps the job in Redis using LMOVE, is Sidekiq Pro.

Solid Queue never had the job anywhere except the database. When a process dies, another process notices the missing heartbeats, prunes the registration, and marks its in-flight jobs failed with SolidQueue::Processes::ProcessPrunedError. The job is a row in solid_queue_failed_executions, visible and retryable.

Then Solid Queue declines to retry it, on purpose, and the README explains why: "the job itself might be what's killing the process (for example, a job that exhausts the container's memory), and retrying it blindly would just kill the next worker too." If you want automatic recovery you subscribe to the fail_many_claimed.solid_queue event, check for that error class, and write your own cap so the loop terminates. The supervisor reaping a crashed fork raises a sibling, SolidQueue::Processes::ProcessExitError.

Durability that costs nothing extra is the strongest single argument for Solid Queue, and it is worth being precise about what it is: not fewer moving parts, but the same ACID guarantee that already covers your users' data now covering the work queued against it.

Recurring jobs, concurrency limits and bulk enqueue

All three are in Solid Queue 1.7.0 today, which is the answer to the most common objection written in 2024. The README's opening paragraph lists "delayed jobs, concurrency controls, recurring jobs, pausing queues, numeric priorities per job, priorities by queue order, and bulk enqueuing (enqueue_all for Active Job's perform_all_later)".

Recurring tasks live in config/recurring.yml, parsed by Fugit, so schedule: every day at 9am America/New_York works and so does a plain cron string. Multiple schedulers are safe: a row in solid_queue_recurring_executions is written in the same transaction as the job, with a unique index on task_key and run_at. That guarantee has a condition the README states and people miss: it "only works if you have preserve_finished_jobs set to true (the default), and the guarantee applies as long as you keep the jobs around". Finished jobs default to one day of retention.

Concurrency controls are declared per job:

limits_concurrency to: 2, key: ->(contact) { contact.account }, duration: 5.minutes

In Sidekiq, the equivalent features are the Enterprise tier: cron jobs, unique jobs and rate limiting are all listed there, not in Pro and not in the free gem.

The honest caveat comes from the Solid Queue README itself, which recommends against its own feature for the common case: "Concurrency controls introduce significant overhead (blocked executions need to be created and promoted to ready, semaphores need to be created and updated) so you should consider carefully whether you need them." For throttling, it tells you to use a dedicated queue with one worker thread instead. And mixing concurrency controls with perform_all_later "has no benefit", because limited jobs must be enqueued one at a time, so you keep the API and lose the batching.

The option that reads like a timeout and is not

duration: in limits_concurrency is the trap. Reading that line above, everyone concludes the job gets five minutes to run and then the lock is released. The README says otherwise: "the duration is not really about the job that's enqueued or being run, it's about the jobs that are blocked waiting, or about the jobs that would get discarded while the semaphore is closed."

So a job holding the semaphore is not interrupted at five minutes. Nothing kills it. What happens instead is that blocked jobs become eligible for release, and they are released by the dispatcher's concurrency maintenance pass, whose interval is a separate setting defaulting to 600 seconds. A duration: 5.minutes sitting in front of a 10 minute failsafe sweep is not a five minute anything. The default duration is SolidQueue.default_concurrency_control_period, three minutes, against that same 600 second sweep.

The README's advice follows from the mechanism: "you should set duration in a way that all your jobs would finish well under that duration and think of the concurrency maintenance task as a failsafe in case something goes wrong."

The test that stays green while the limit does nothing

Concurrency controls are enforced at enqueue, by Solid Queue, and only by Solid Queue. The limits_concurrency macro comes from ActiveJob::ConcurrencyControls, which the engine mixes into ActiveJob::Base whenever the gem is loaded, and all it does is assign class attributes: concurrency_key, concurrency_limit, concurrency_duration, concurrency_on_conflict. The enforcement is somewhere else entirely, in SolidQueue::Job, where acquire_concurrency_lock calls Semaphore.wait(self), reached only through SolidQueueAdapter#enqueue.

Run your specs on the :test adapter, which is what Rails configures for the test environment, and none of that code runs. The macro parses, the job enqueues, your assertions pass, and the limit you wrote is enforced by nothing. The same is true of :async in development, where the concurrency declaration is decoration. Nothing warns you at boot, because from the gem's point of view nothing is wrong.

If a concurrency limit is load bearing, the test has to run against Solid Queue itself, with the queue schema loaded, asserting on SolidQueue::BlockedExecution rows rather than on have_enqueued_job.

What each one costs

Both are free to start. Sidekiq OSS is LGPL-3.0 and gives you scheduled jobs, retries, the web UI and the Ruby API. Solid Queue is MIT with no tier above it.

The Sidekiq numbers matter when you need what the free gem leaves out. Pro is "$99/mo or $995/yr", adding batches, reliability including super_fetch, and worker metrics. Enterprise starts "at $269/mo per 100 threads", adding rate limiting, cron, unique jobs, rolling restarts, historical metrics and web UI authorization. A five worker process deployment at 20 threads each is 100 threads, so $269 a month is the realistic entry point for a mid-sized app that wants cron and unique jobs.

Add the Redis itself, whatever your platform charges for a managed instance with enough memory to hold your backlog, plus the worker dynos, which you pay for under either design.

Against that, Solid Queue's bill is capacity on a database you were already paying for, plus the connections and write volume from the previous section, plus an afternoon wiring up Mission Control.

The verdict, and what would flip it

Start on Solid Queue and keep it until you can name the thing it will not do for you. The reasoning is not cost and not simplicity, both of which are arguments a sufficiently stubborn person can win in either direction. The reasoning is that a job is a row in the same database as the record it is about, so a restore is consistent, a crash loses nothing, and the failure mode where your queue and your data disagree does not exist.

Three things flip it, and only one is throughput.

Latency flips it first and earliest. Any workload where a user is waiting on the job wants brpop, not a poll loop, and no amount of tuning polling_interval down turns polling into blocking. Redis already being in your stack flips it next, because the entire operational argument for Solid Queue was the datastore you did not have to run, and if you run one anyway for rate limiting or WebSockets then Sidekiq is free of extra infrastructure too. Throughput flips it last, later than most people assume, and it is a gradient rather than a line: as the queue's write volume starts to move your web request latency, the fix is to give the queue its own database, and after that you are operating two datastores, which was the thing you were avoiding.

What would not flip it: missing features. Recurring jobs, concurrency controls and bulk enqueue all shipped. Choosing Sidekiq in 2026 for cron means choosing Enterprise for cron.

What this page does not cover

Benchmarks are absent here on purpose. No jobs per second figure appears above, because a credible one needs a fixed job payload, a fixed database, and both systems tuned by somebody who wants the other to win, and a number without that setup is worse than no number.

Also missing: GoodJob and Que, both database-backed and both with adherents who will tell you Solid Queue reinvented them; Sidekiq's Rails-free usage, since everything above assumes Active Job; and migration mechanics, which the Solid Queue README covers under incremental adoption by setting self.queue_adapter = :solid_queue on one job class at a time.

#rails #jobs

Comments

No comments yet. Be the first.

Only used to confirm and publish your comment. Never shown publicly, never shared.

Markdown: **bold**, `code`, ```fenced blocks```, > quotes, [links](url). HTML and images are not rendered.