after_commit, and the job that runs before the row exists
A background job that fails intermittently with "couldn't find Order with id=5", on an order you can see in the database right now, is nearly always this. The callback that enqueued it ran inside the transaction, and the worker got there before the commit did.
Here's the order things actually happen in, measured on Rails 8.1.4 by logging from every callback on a model.
The order, and the one that surprises people
Invoice.create!(name: "a")
# ["before_save", "after_create", "after_save", "after_commit", "after_create_commit"]
Two things to take from that line.
after_create fires before after_save, which reads backwards and catches people out. The
create callbacks are nested inside the save callbacks, so the specific one runs first and the
general one wraps it.
And the two commit callbacks come last, after everything else. That's the boundary, and the next measurement shows where it actually sits.
Where the transaction boundary is
Invoice.transaction do
Invoice.create!(name: "b")
# at this point the log reads:
# ["before_save", "after_create", "after_save"]
end
# and now it reads:
# ["before_save", "after_create", "after_save", "after_commit", "after_create_commit"]
Inside the block, the row exists as far as your connection is concerned and does not exist as far
as anybody else is concerned. after_save has run. after_commit hasn't.
So an after_save that does this:
after_save { OrderConfirmationJob.perform_later(self) }
hands an id to a queue that another process is reading, while the transaction holding that id is still open. The worker is fast, the transaction is not yet committed, and the lookup fails. It works on your laptop every time, because your laptop runs the job inline or a second later.
What a rollback leaves behind
Invoice.transaction do
Invoice.create!(name: "c")
raise ActiveRecord::Rollback
end
# ["before_save", "after_create", "after_save", "after_rollback"]
# rows written: 0
Zero rows, and three callbacks that already ran. Whatever after_save did, it did. If it wrote to
Redis, sent a webhook or incremented a counter somewhere, that side effect survived a transaction
that produced no row at all, and nothing will go back and undo it.
after_rollback exists to clean up after exactly that, and it's a worse place to be than not
having done the thing. This is the second argument for after_commit, and it's the one that
matters when something goes wrong rather than when everything is merely fast.
Nested transactions commit once
Invoice.transaction do
Invoice.transaction { Invoice.create!(name: "d") }
# after the inner block: ["before_save", "after_create", "after_save"]
end
# after the outer block: [..., "after_commit", "after_create_commit"]
The inner block isn't a transaction, it's a no-op that joins the outer one. So a service object
that wraps its work in transaction do and relies on commit callbacks firing at the end of it
will behave completely differently when something else calls it inside a transaction of its own.
This is worth knowing before you debug it, because the symptom is a callback that fires "too late" in one code path and on time in another, with no difference in the model.
Raising in after_commit does nothing useful
Receipt.create!(name: "e")
# RuntimeError: boom from after_commit
# rows in the table afterwards: 1
The exception reached the caller, so the request 500s, and the row is in the database because the transaction committed before the callback ran. There is nothing left to roll back.
Which means after_commit has no safety net. If your webhook call raises, you have a committed
record, a failed request and no retry. The fix is the same one that applies to every side effect:
put it in a job, and let the queue's retry handle it, so the callback's only job is to enqueue
something that can fail safely.
The rule, and the setting that may already handle it
Put reads and derived columns in before_save. Put anything that leaves the process in
after_commit: jobs, emails, webhooks, cache busting, search index updates.
after_create_commit, after_update_commit and after_destroy_commit are the narrower versions
and they're usually what you want, because after_commit alone fires on all three and a callback
that meant "when a record is created" firing on every update is a quiet bug.
Worth checking before you write any of this: Rails 7.2 added
config.active_job.enqueue_after_transaction_commit, which makes perform_later wait for the
commit on its own. If that's on in your app, the job case is already handled and an after_commit
callback to do it manually is belt and braces. Everything else on the list above still needs the
callback.
What this page doesn't cover
No job ran. The failure described above is a race between two processes and nothing here reproduces it under load, so the ordering is measured and the race is reasoned about.
Nothing here touches after_commit on: :destroy with dependent: associations, where the order
across a tree of records gets genuinely confusing, nor what any of it does under
use_transactional_tests, which is on by default and wraps every test in a transaction that never
commits. That last one is why commit callbacks are famously untested, and the tests behind this
article had to turn it off.
Comments
No comments yet. Be the first.