Rails system tests

A request spec renders the template and asserts against the string. That is most of what you need, and it is why the LaunchKit boilerplate has 141 spec files and not one of them opens a browser. The gap is narrow and specific: the template is correct and the page is broken, because a Stimulus controller never connected, a <turbo-frame> came back without its id, or an element that is invisible to you is on top of the button. A rails system test is the only thing in the box that catches those, and the price is a browser.

This post is about what that price actually is. Every number below was measured on a throwaway Rails 8.1.3.1 app against PostgreSQL 17.7 and Chrome 153.0.8010.53, and the failures are copied out of the runner.

What driven_by starts

Four things, in one process tree. The minitest process is the one you started. Capybara boots Puma in a thread of that same process, which ActionDispatch::SystemTesting::Server arranges in two lines:

def set_server
  Capybara.server = :puma, { Silent: self.class.silence_puma } if Capybara.server == Capybara.servers[:default]
end

Then Selenium starts chromedriver, and chromedriver starts Chrome. So visit articles_url is a Ruby method call that becomes an HTTP request to chromedriver, that becomes a CDP message to Chrome, that becomes a real HTTP request back to a Puma listening on a port Capybara picked. The run banner prints it:

Capybara starting Puma...
* Version 8.0.2, codename: Into the Arena
* Min threads: 0, max threads: 4
* Listening on http://127.0.0.1:53689

The driver line is one call, and the defaults are in the signature:

def self.driven_by(driver, using: :chrome, screen_size: [1400, 1400], options: {}, &capabilities)

using: :chrome means a visible window. The generated application_system_test_case.rb overrides it to :headless_chrome, and headless Chrome turns out to be two command line arguments in SystemTesting::Browser and nothing else:

options = ::Selenium::WebDriver::Chrome::Options.new
options.add_argument("--disable-search-engine-choice-screen")
options.add_argument("--headless") if name == :headless_chrome

Plain --headless, not --headless=new, which is correct on a current Chrome: the bare flag has meant the new headless mode since Chrome 128, and Chrome 132 removed the old implementation from the binary altogether, so the two spellings are now the same thing. Nothing downloads chromedriver in your Gemfile: Selenium Manager does it on first use and caches it, and on this machine the cache holds 153.0.8010.52 while the installed browser is 153.0.8010.53. A patch version apart, and it works.

One number worth knowing before you compare screenshots: screen_size: [1400, 1400] is the window, not the viewport. Every failure screenshot this post produced is 1400 by 1257.

Rails 8.1 stopped generating them

Rails 8 system tests have one thing the older ones did not, and it is not a feature. The railties 8.1 changelog has the entry, attributed to Eileen M. Uchitelle:

Don't generate system tests by default.

Rails scaffold generator will no longer generate system tests by default. To enable this pass --system-tests=true or generate them with bin/rails generate system_test name_of_test.

A second entry in 8.1.2 goes further: "Skip all system test files on app generation." The testing guide gained a whole section to explain the reasoning, and it is unusually blunt for a Rails guide:

System tests provide the most realistic testing experience as they test your application from a user's perspective. However, they come with important trade-offs:

• They are significantly slower than unit and integration tests
• They can be brittle and prone to failures from timing issues or UI changes
• They require more maintenance as your UI evolves

Given these trade-offs, system tests should be reserved for critical user paths rather than being created for every feature.

That is official Rails telling you to write fewer of these. The boilerplate agrees in a different dialect, with config.generators.system_tests = nil at config/application.rb:69.

There is a second default nobody remembers, and it predates all of this: bin/rails test does not run test/system. The guide says so in a NOTE. In the demo app bin/rails test reported 2 runs in 0.138 s and bin/rails test:all reported 10 runs in 9.24 s, which is a good way to discover that your CI has never run a single browser test.

Waiting on the wrong thing

Capybara's finders retry. Capybara::Node::Base#synchronize, at capybara/node/base.rb:76, wraps a block, catches a list of errors, sleeps default_retry_interval and tries again until a timer expires. The budget is set in capybara.rb:500:

config.default_max_wait_time = 2
config.default_retry_interval = 0.01
config.ignore_hidden_elements = true

Two seconds, retried every ten milliseconds. Everything Capybara does that the documentation calls "waiting" is that loop. Everything else in your test is plain Ruby that runs once.

Here is the whole flakiness class in six lines. A button reveals a panel that says "Loading...", and 1200 ms later the panel says "Loaded page 2".

test "waiting on the wrong thing" do
  visit articles_url
  click_on "Load more"
  assert_equal "Loaded page 2", find("#late-text").text
end

Failure:
ArticlesTest#test_waiting_on_the_wrong_thing:
Expected: "Loaded page 2"
  Actual: "Loading..."

find waited, correctly, for an element matching #late-text, and that element existed immediately. Then .text read it once and assert_equal compared it once. The retry happened around the wrong half of the line. Move the expectation inside the finder and the same page passes:

assert_selector "#late-text", text: "Loaded page 2"

The rule that falls out of this is mechanical, not stylistic. Any assertion whose subject is a Capybara node method returning a Ruby value, .text, .value, [:class], .all(...).size, is a snapshot, and a snapshot of an asynchronous page is a coin flip. Push the condition into the selector, where synchronize can see it.

Capybara retries the following, and nothing else, per capybara/selenium/driver.rb:297:

[
  ::Selenium::WebDriver::Error::StaleElementReferenceError,
  ::Selenium::WebDriver::Error::ElementNotInteractableError,
  ::Selenium::WebDriver::Error::InvalidSelectorError,
  ::Selenium::WebDriver::Error::ElementClickInterceptedError,
  ::Selenium::WebDriver::Error::NoSuchElementError,
  ::Selenium::WebDriver::Error::InvalidArgumentError
]

Read that list as a list of flakes Capybara has already absorbed for you. The next section is about what happens when one of them takes longer than two seconds.

Animations spend the same two seconds

A modal that slides away over 200 ms never shows up in a test result, because the click on whatever was behind it raises ElementClickInterceptedError, Capybara catches it, and a later retry lands after the overlay has gone. The same modal at 2500 ms shows up every time:

Selenium::WebDriver::Error::ElementClickInterceptedError: element click intercepted:
Element <button id="load-more">...</button> is not clickable at point (47, 176).
Other element would receive the click:
<div id="modal" class="" style="transition-duration: 2500ms;">...</div>
  (Session info: chrome=153.0.8010.53)

Same markup, same test, one CSS property. That is the actual shape of "flaky": not random, just a duration sitting near a budget, and a machine under load moves the duration and not the budget.

Capybara has a switch for it. Capybara.disable_animation = true installs a Rack middleware on the test server that rewrites every HTML response, and what it injects is short enough to quote whole:

*, *::before, *::after {
   transition: none !important;
   animation-duration: 0s !important;
   animation-delay: 0s !important;
   scroll-behavior: auto !important;
}

RAILS_SYSTEM_TESTING_SCREENSHOT_HTML=1 saves the page next to the screenshot, and the block is right there before </head> in the saved file, which is how you confirm it is on. With it set, the 2500 ms modal test passes.

Two things about that middleware deserve to be said out loud. It edits your HTML with html.sub(%r{(</head>)}, ...), so a response without a </head> gets nothing, and it reads the CSP header to reuse your nonce, which means a page with a strict style policy and no nonce silently keeps its animations. And transition: none is a statement about CSS transitions only.

What Element.animate does to all of that

The Web Animations API is not a CSS transition and not a CSS animation. element.animate([...], { duration: 2500 }) is a JavaScript call, and no stylesheet declares it, so there is nothing for a transition: none rule to override. With Capybara.disable_animation = true still set, a full-screen overlay animated that way blocked the click for its whole duration:

Selenium::WebDriver::Error::ElementClickInterceptedError: element click intercepted:
Element <button id="load-more">...</button> is not clickable at point (47, 124).
Other element would receive the click: <div id="waapi" ...>...</div>

So the switch most guides present as the fix for animation flakiness covers the animations written in CSS and misses the ones written in JavaScript. If your interaction layer is a library that animates imperatively, disable_animation buys you nothing and the two-second budget is still the only thing standing between you and a red build.

The overlay you cannot see and Chrome can

The boilerplate's landing pages use a scroll reveal: a Stimulus controller adds is-visible when an IntersectionObserver fires, and app/assets/stylesheets/landing.css starts the element hidden.

.reveal {
  opacity: 0;
  transform: translateY(24px);
  transition: opacity 0.7s cubic-bezier(0.16, 1, 0.3, 1), transform 0.7s cubic-bezier(0.16, 1, 0.3, 1);
}

Reproduce that pattern and ask Capybara about the element before the observer fires:

visible?: false
text:     ""

ignore_hidden_elements = true means the default finder never sees it, so assert_text "Revealed content" fails with the message Capybara reserves for this exact case: "expected to find visible css ... but there were no matches. Also found \"\", which matched the selector but not all filters." Selenium's getText returns rendered text, and nothing at opacity: 0 renders. Every assertion about copy below the fold is subject to this, and the element is perfectly present in the DOM, which is why the same content passes a request spec.

Then the other half, which is worse. Put a button below that element, ask Capybara to click it, and:

Selenium::WebDriver::Error::ElementClickInterceptedError: element click intercepted:
Element <button id="open-dialog">...</button> is not clickable at point (51, 1238).
Other element would receive the click: <div class="reveal" id="reveal-me">...</div>

Invisible to a human, invisible to Capybara's finders, and fully solid to a click. Nothing about that is a Capybara defect: the element has layout, and hit testing does not consult opacity.

The fix is not disable_animation, which was already on for that run and did nothing, because opacity: 0 here is the resting state rather than a transition. The application already ships the right answer in a media query, and the test just has to ask for it:

driven_by :selenium, using: :headless_chrome, screen_size: [ 1400, 1400 ] do |options|
  options.add_argument("--force-prefers-reduced-motion")
end

With that flag the same two lines print visible?: true and text: "Revealed content", and the assertion passes. The app's own @media (prefers-reduced-motion: reduce) block, written for users who asked not to be moved, turns out to be the test configuration too. If your CSS does not have that block, this flag does nothing and you are back to scrolling the element into view yourself.

Native <dialog> behaves differently and correctly. showModal() puts the dialog in the top layer and makes the rest of the document inert, so a click on the page behind it raises the same intercepted error with <dialog id="native-dialog" open=""> named as the blocker, and the reduced motion flag changes nothing. That is not flakiness, that is a modal working, and the boilerplate's dialog_controller.js is eleven lines around showModal() and close(). Close it in the test.

One connection, two threads, one BEGIN

The oldest complaint about browser tests is that the server cannot see the records the test just created, because they are in an uncommitted transaction on a different connection. Modern Rails solved that, and the mechanism is worth reading because its edges are where the remaining surprises live.

ActiveRecord::TestFixtures calls pool.pin_connection!(lock_threads) with lock_threads defaulting to true, and the pool method is six lines of consequence:

def pin_connection!(lock_thread)
  @pinned_connection ||= (connection_lease&.connection || checkout)
  @pinned_connections_depth += 1
  ...
  @pinned_connection.lock_thread = ActiveSupport::IsolatedExecutionState.context if lock_thread
  @pinned_connection.pinned = true
  @pinned_connection.verify!
  @pinned_connection.begin_transaction joinable: false, _lazy: false
end

Measured on a live pool, pinning changes two observable things:

lock before: Module
same conn: true
lock after:  ActiveSupport::Concurrency::ThreadMonitor
tx open:     true
joinable:    false

Module there is ActiveSupport::Concurrency::NullLock, whose synchronize just yields. After pinning, the adapter holds a real monitor, so the Puma thread and the test thread take turns on one connection instead of racing on it. Every checkout in the process returns that connection, which is why a controller that spawns its own thread and calls with_connection still reads the test's unsaved rows: in the demo app the page printed thread saw: 3 for three records that had never been committed.

The visibility rule is therefore "same process, yes; anything else, no", and it is easy to prove from inside a test:

Article.create!(title: "Only in the transaction", published: true)
outside = `psql -h localhost -p 15432 -d sysdemo_test -tAc "select count(*) from articles"`.strip

INSIDE  the transaction: 1
OUTSIDE the transaction: 0

That is the real constraint on a system test, and it has nothing to do with browsers. Anything that reads your database from another process, a separate worker, a webhook replay tool, a second app sharing the schema, sees an empty table. On a primary and replica setup Rails papers over the same problem inside the process by pointing every role at the writing pool, in setup_shared_connection_pool, which means your reading-role code is not actually exercising a replica during system tests.

The race the transaction does not cover

Sharing a connection fixes visibility. It does not fix time. click_on returns when the click has been dispatched, not when the server has finished with it, so this test is a race:

visit articles_url
fill_in "title", with: "Written by the browser"
click_on "Create"
assert_equal 1, Article.where(title: "Written by the browser").count

Failure:
ConnectionTest#test_the_test_process_races_the_browser_on_a_write:
Expected: 1
  Actual: 0

The controller was still inside a one second sleep when the test counted. On a fast local machine with a fast action this passes more often than it fails, which is the worst possible outcome, because it means the test earns its place in the suite and then starts failing on CI once a month.

The fix is to assert on something the browser can only show you after the server answered, and only then to look at the database:

click_on "Create"
assert_selector "li.article", text: "Written by the browser"
assert_equal 1, Article.where(title: "Written by the browser").count

Now the first assertion is inside synchronize and the second one runs after it succeeded. A general version of the rule: in a system test the page is the clock. Anything you check in Ruby has to be checked after something you checked in the browser.

Screenshots, and what the screenshot is of

The screenshot helper is small and worth knowing exactly. take_failed_screenshot is wired into before_teardown, so no configuration is needed:

def take_failed_screenshot
  return unless failed? && supports_screenshot? && Capybara::Session.instance_created?

  take_screenshot
  metadata[:failure_screenshot_path] = relative_image_path if Minitest::Runnable.method_defined?(:metadata)
end

Files land in Capybara.save_path.presence || "tmp/screenshots", named "#{unique}_#{method_name}" where unique is the string "failures" for a failing test and a counter otherwise. The method name is sanitised with method_name.gsub(/[^\w]+/, "-") and cut at name[0...225], so a test called "disable_animation does not stop Element.animate" becomes failures_test_disable_animation_does_not_stop_Element-animate.png. One file per failing test name, overwritten each run, which keeps the directory tidy. supports_screenshot? is just Capybara.current_driver != :rack_test.

Two environment variables earn their keep. RAILS_SYSTEM_TESTING_SCREENSHOT_HTML=1 writes the DOM next to the image, which is how you tell "the element was missing" from "the element was there and hidden". RAILS_SYSTEM_TESTING_SCREENSHOT=inline prints the PNG into an iTerm2 session using the image protocol, and artifact emits the Buildkite escape sequence instead.

Now the part the documentation does not say. The screenshot is taken in teardown, which is after the failure, after Capybara has given up its two seconds, and after whatever the page was doing has finished doing it. For the assert_equal failure above that gap is milliseconds and the image is perfect: it shows "Loading..." under a "Load more" button, which is the whole bug in one frame. For the intercepted click on the 2500 ms modal, the image shows the overlay already most of the way off screen, in a position where the click it refused would have landed cleanly. The error message names the blocking element and the screenshot does not show it blocking anything.

Treat the image as context and the exception text as evidence. On RSpec the trade is slightly worse: rspec-rails builds method_name from the example description plus "_#{rand(1000)}", so every run writes a new file and tmp/screenshots grows until somebody empties it.

What sixty of them cost

One page, one assertion, sixty times, written twice. Integration tests:

Finished in 0.275966s, 217.4181 runs/s, 434.8362 assertions/s.

System tests, single process:

Finished in 7.799064s, 7.6932 runs/s, 7.6932 assertions/s.

4.6 ms against 130 ms, a factor of 28, for exactly the same claim about exactly the same HTML. The fixed cost is real but small: one passing system test took 1.47 s and ten took 1.84 s, so the browser and the server together are about 1.4 s once and roughly 40 ms per additional trivial test. The variable cost is where suites die, and it has one dominant term, which is waiting. Five tests that each fail on a missing selector took 11.44 s, about 2.3 s each, because each one spends the full default_max_wait_time before reporting. A suite of 200 system tests with 10 failing spends 23 seconds telling you about the 10.

Parallelism is the reflex, and the generated test_helper.rb reaches for it with parallelize(workers: :number_of_processors). For system tests that default is wrong, because each worker is a process with its own database, its own Puma, its own chromedriver and its own Chrome. The same 60 tests on a 12 core machine:

workers=1   Finished in 9.086386s
workers=2   Finished in 5.455846s
workers=4   Finished in 5.111102s
workers=8   Finished in 5.926412s
workers=12  Finished in 7.467231s

The default is 46% slower than four workers and barely better than one. Twelve browsers on twelve cores do not get twelve cores. Pick a number, measure it on the machine your CI actually runs on, and put it in PARALLEL_WORKERS.

Selenium, or Cuprite

Two ways to drive Chrome from Capybara, and they are in different shape.

Selenium is what Rails wires up and what driven_by :selenium means. selenium-webdriver 4.49.0 was released on 2026-09-09, Selenium Manager handles the driver binary, and the W3C protocol underneath it is what produces the error classes quoted throughout this post. The cost is a second hop: your Ruby talks HTTP to chromedriver, which talks CDP to Chrome.

Cuprite skips the hop. It drives Chrome over CDP directly through Ferrum, with no chromedriver and no Selenium, and it is a drop-in Capybara driver. Version 0.18 was released on 2026-09-03, which ends a gap since 0.17 in May 2025. It is maintained, it is faster in most reports, and it gives you CDP-level tools such as network interception and console log access that Selenium makes awkward. What you give up is the thing Rails assumes: driven_by knows :selenium and :rack_test, so Cuprite goes in as a registered Capybara driver and you lose the part of SystemTesting::Browser that builds options for you.

Capybara itself is the quiet one. 3.40.0 has been the current release since 2024-01-27, more than two and a half years, while master keeps taking merges: the most recent commits at the time of writing are from July 2026. Not abandoned, not shipping either. Everything in this post runs on that release.

The call, and what would change it

Write system tests for the paths where the browser is the thing under test, and the guide's list is the right list: sign up, checkout, and any interaction where Hotwire is doing the work. For the boilerplate that means the Stimulus controllers with real layout behaviour and the onboarding flow that renders inside a <turbo-frame>, because Turbo frame: Content missing is a 200 in the server log and a green request spec, and a system test is the only thing that reads the two words in bold. The same argument closes Stimulus controllers in practice, which ends on jsdom returning zeros for layout: a controller that measures the page cannot be unit tested, so either a browser runs it or nothing does.

Everything else stays a request spec. A system test that only checks that a page contains some text is a 130 ms version of a 4.6 ms test, and it can fail for reasons that have nothing to do with your application.

What would change the recommendation: a headless browser with a startup cost in the tens of milliseconds would make the per-test number stop mattering, and most of the section on waiting would survive anyway because the waiting is in the page, not in the browser. A Capybara release that made the synchronize budget per-assertion explicit at the call site, rather than a global with a wait: override, would remove a good share of the flakes above.

The cost of that position, stated plainly: a suite with six system tests has six chances to be flaky, and the six will not be the ones with the most coverage, they will be the ones touching the most JavaScript. You will spend afternoons on them. The alternative is not spending those afternoons and finding out from a user that the button does nothing.

What this post does not cover

The LaunchKit boilerplate has no system specs. It has the gems, in the test group, under a comment that says what they were for:

group :test do
  # System (end-to-end) specs driving Turbo/Stimulus in a real browser
  gem "capybara"
  gem "selenium-webdriver"
end

capybara 3.40.0 and selenium-webdriver 4.49.0 are in Gemfile.lock, there is no spec/system directory, none of the 141 spec files declares type: :system, and config.generators.system_tests = nil sits in config/application.rb:69. Two details would bite whoever writes the first one: config.infer_spec_type_from_file_location! is commented out in spec/rails_helper.rb:66, so a file at spec/system/foo_spec.rb needs an explicit type: :system, and rspec-rails 8.0.4 defaults to DEFAULT_DRIVER = :selenium_chrome_headless rather than the headful :chrome that ActionDispatch::SystemTestCase defaults to. The measurements in this post come from a scratch Minitest app, which is the honest provenance.

Also absent: served_by and remote browsers in Docker, which is a different set of failures; parallelize_setup and per-worker database naming; Playwright, which has a Capybara driver and which nobody in the Rails 8 ecosystem has converged on; visual regression tools; and any discussion of assert_text versus assert_selector on performance grounds, because the two go through the same synchronize loop and the difference is not measurable next to the 130 ms.