LaunchKit
← All posts
· 16 min read · by The LaunchKit team · 0 views

Rails development inside a Docker container

A Rails application in a bind-mounted development container answered a trivial request in 11.56 ms where the same application on the laptop it is mounted from answered in 3.06 ms, and almost all of that gap is one thing: the per-request check that asks whether any file changed. This page measures where the time goes, changes one line, and measures again.

Everything below ran on a scratch application generated with rails new blog --database=postgresql --skip-action-mailbox --skip-action-text --skip-jbuilder --skip-git, on an Apple M2 Max with 12 cores, macOS 26.5.1, Docker Desktop 4.37.2 (engine 27.4.0, compose v2.31.0-desktop.2), with the Docker VM holding 12 CPUs and 7.653 GiB. The rails binary is 8.1.3.1; the Gemfile it wrote resolved to rails (8.1.4), which is what actually booted. Ruby is 4.0.5 on both sides, Puma 8.0.2, bootsnap 1.26.0, listen 3.10.0. Host ports are 3051 and 3052 because 3000 and 3010 were already taken on this laptop.

Writing the compose file and the Postgres service is a separate subject, covered in Rails and Postgres in Docker Compose. This page starts after the stack is up and asks what the inner loop costs.

Where a development request's time actually goes

The suspicion is usually the bind mount, and the bind mount is guilty, but not in the way people reach for. Rails in development runs ActiveSupport::FileUpdateChecker#updated? on every request, which globs the autoload directories and calls File.mtime on every .rb file it finds. Each one of those stats crosses the host boundary.

Here is that call, timed on its own, inside bin/rails runner, over 200 iterations. The same application is measured three ways: mounted from the host, copied into the image, and running natively on the laptop.

dirs = Rails.autoloaders.main.dirs.index_with { [:rb] }
watcher = Rails.application.config.file_watcher.new([], dirs) { }
watched = watcher.send(:watched)
n = 200
t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
n.times { watcher.updated? }
ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) / n * 1000
puts "watcher=#{Rails.application.config.file_watcher}"
puts "autoload dirs=#{dirs.size}  files stat-ed per check=#{watched.size}"
puts format("updated? mean=%.3fms over %d calls", ms, n)

Run with docker exec -i web bin/rails runner - < watcher_bench.rb. On the generated app, six .rb files:

=== container, bind mount ===
autoload dirs=18  files stat-ed per check=6
updated? mean=2.999ms over 200 calls
=== container, baked into image ===
autoload dirs=18  files stat-ed per check=6
updated? mean=0.051ms over 200 calls
=== host ===
autoload dirs=18  files stat-ed per check=6
updated? mean=0.399ms over 200 calls

Six files. Three milliseconds. Then 300 empty classes were written into app/models/gen/ to make the app a more honest size, and the same three measurements repeated:

=== container, bind mount ===
autoload dirs=18  files stat-ed per check=306
updated? mean=8.138ms over 200 calls
=== container, baked ===
autoload dirs=18  files stat-ed per check=306
updated? mean=0.588ms over 200 calls
=== host ===
autoload dirs=18  files stat-ed per check=306
updated? mean=1.530ms over 200 calls

The image layer is 59 times faster than the bind mount at six files and 14 times faster at 306. The laptop's own APFS sits in between, and its advantage shrinks as the file count grows, which is the shape you would expect if the per-file cost is what differs. One check is not the whole bill either: Rails.application.reloaders.size is 5 on this app, two of them file update checkers over the autoload paths, the others Importmap::Reloader, ActionView::CacheExpiry::ViewReloader and Rails::Application::RoutesReloader.

The one line that gives most of it back

ActiveSupport::EventedFileUpdateChecker does not stat anything on the request path. It holds an inotify watch and answers from a flag the listener thread set. Rails uses it only when you ask for it, and rails new on 8.1 does not even leave the commented-out hint in config/environments/development.rb that older versions did.

Two edits. In the Gemfile:

gem "listen", "~> 3.9", group: :development

And at the top of config/environments/development.rb:

config.file_watcher = ActiveSupport::EventedFileUpdateChecker

The measurement is 300 keep-alive requests to a controller that renders a plain string, issued from inside the container so that port publishing is out of the picture, on the 306-file app, with DISABLE_BOOTSNAP=1 on both runs so that nothing but the watcher differs:

Configuration p50 p95 mean
bind mount, FileUpdateChecker 11.56 ms 31.16 ms 15.73 ms
bind mount, EventedFileUpdateChecker 2.35 ms 4.23 ms 2.61 ms
code in the image, FileUpdateChecker 2.01 ms 3.16 ms 2.34 ms
host, FileUpdateChecker 3.06 ms 3.88 ms 3.19 ms

The evented watcher on a bind mount beats the laptop. That is the whole answer to "Rails in Docker is slow": it was not Docker, it was 306 stat calls per request going through a file sharing layer.

inotify does cross the bind mount, which is newer than most advice about it

A lot of writing about Rails in Docker says the evented watcher cannot work, because filesystem events raised on the host do not reach a Linux container. That was true, and it is not true on Docker Desktop 4.37.2. The mount shows up in the container as

722 709 0:47 /tmp/.../blog /rails rw,nosuid,nodev,relatime - fakeowner /run/host_mark/private rw,fakeowner

and events come through it. Listen::Adapter.select(force_polling: false) returned Listen::Adapter::Linux, not the polling fallback. A direct rb-inotify watch on app/controllers, started inside the container, then a write to the file from the host shell:

watching
inotify event: [:modify] probe_controller.rb

So no force_polling: true, no LISTEN_POLLING, no config.file_watcher fallback. The watch budget is not a constraint either: /proc/sys/fs/inotify/max_user_watches in the container reads 1048576.

What the switch costs

Three things, and the first one is the reason to think before copying it.

Reloading gets slower to notice. With the stat watcher, an edit on the host was served by the container in a median of 0.036 s over 30 edits. With the evented watcher, the median was 0.133 s. Listen debounces, and the gain on every request is paid back on every save. 133 ms is under the threshold where a human notices a page load, so this is the right trade, but it is a trade.

The gem is not optional once the line is there. listen sits in the development group, which the generated production Dockerfile excludes with BUNDLE_WITHOUT="development". Boot that image with RAILS_ENV=development and you get:

listen is not part of the bundle. Add it to your Gemfile. (Gem::LoadError)
    from /usr/local/bundle/ruby/4.0.0/gems/activesupport-8.1.4/lib/active_support/evented_file_update_checker.rb:3:in '<main>'

And the listener is a thread holding watches on every autoload directory for the life of the process, which is fine on one container and worth remembering when the compose file runs a web, a jobs and a CSS watcher service off the same image.

postgres:18 refuses the volume path that worked through 17

volumes: - pgdata:/var/lib/postgresql/data is the mount that has been correct for every Postgres image up to and including postgres:17, and on postgres:18 it does not start. Not "loses data", not "warns": exits 1 on a brand new empty volume.

db-1  | Error: in 18+, these Docker images are configured to store database data in a
db-1  |        format which is compatible with "pg_ctlcluster" (specifically, using
db-1  |        major-version-specific directory names).
...
db-1  |        Counter to that, there appears to be PostgreSQL data in:
db-1  |          /var/lib/postgresql/data (unused mount/volume)

PGDATA was /var/lib/postgresql/data through postgres:17 and is /var/lib/postgresql/18/docker on postgres:18 (18.6 in this run). The entrypoint does not merely ignore the old path, it treats a mountpoint there as a misconfiguration to refuse, at /usr/local/bin/docker-entrypoint.sh lines 255 to 261:

if [ "${#OLD_DATABASES[@]}" -eq 0 ] && [ "$PG_MAJOR" -ge 18 ] && {
    mountpoint -q /var/lib/postgresql/data \
    || awk '$5 == "/var/lib/postgresql/data" { found = 1 } END { exit !found }' /proc/self/mountinfo
}; then
    OLD_DATABASES+=( '/var/lib/postgresql/data (unused mount/volume)' )

The fix is to mount one directory higher and let the image choose the subdirectory, which is what the message asks for:

    volumes:
      - pgdata:/var/lib/postgresql

A test in the suite below pins it, so that the next major bump is a failing assertion rather than a container that will not boot:

data_directory = ActiveRecord::Base.connection.select_value("show data_directory")
assert_equal "/var/lib/postgresql/18/docker", data_directory

DATABASE_URL in compose points your test suite at your development database

The DATABASE_URL line is the one that costs data, and it is the shortest way to give a container its database, so it is everywhere:

    environment:
      DATABASE_URL: postgres://postgres:postgres@db:5432/blog_development

DATABASE_URL is not scoped to an environment. It overrides the connection for all of them, and config/database.yml saying database: blog_test under test: changes nothing:

$ docker compose exec -e RAILS_ENV=test web bin/rails runner \
    'c=ActiveRecord::Base.connection_db_config; puts "env=#{Rails.env} db=#{c.database} host=#{c.host}"'
env=test db=blog_development host=db

What follows from that is not a warning, it is a demonstration. A canary table was created in development, then db:test:prepare was run in the test environment, then the table was counted again:

rows before: 1
--- running RAILS_ENV=test bin/rails db:test:prepare
/rails/db/schema.rb doesn't exist yet. Run `bin/rails db:migrate` to create it, then try again. If you do not intend to use a database, you should instead alter /rails/config/application.rb to limit the frameworks that will be loaded.
--- after
ActiveRecord::StatementInvalid: PG::UndefinedTable: ERROR:  relation "canaries" does not exist

The task failed and purged the database on its way out. Every fixture-loading run of the suite does the same thing more quietly.

The fix is to stop shipping a full URL and pass the parts, so that each environment keeps the database name database.yml gives it. In config/database.yml, under default: &default:

  host: <%= ENV.fetch("DATABASE_HOST", "localhost") %>
  username: <%= ENV.fetch("DATABASE_USER", nil) %>
  password: <%= ENV.fetch("DATABASE_PASSWORD", nil) %>

and in the compose file, DATABASE_HOST: db, DATABASE_USER: postgres, DATABASE_PASSWORD: postgres. After that, RAILS_ENV=test answers env=test db=blog_test host=db.

web-console is off in every container until you say otherwise

The interactive console on the Rails error page is the one development feature that silently does not work in a container, and the reason is in the log rather than on the page:

Cannot render console from 172.21.0.1! Allowed networks: 127.0.0.0/127.255.255.255, ::1, ::ffff:127.0.0.0/::ffff:127.255.255.255

The request arrives from the compose network's gateway, not from loopback. The gateway address is not stable: it was 172.21.0.1 on one run of this stack and 172.18.0.1 after a docker compose down recreated the network, which is why the permission is written as the block rather than the address.

config.web_console.permissions = "172.16.0.0/12"

With that line, an action that raises returns an error page containing id="console" and the log line is gone. Do not put it in config/application.rb: 172.16.0.0/12 is private space, but it is private space shared with whatever else can reach the app.

docker compose watch: faster requests, slower saves

The other way to get the code into the container is not to mount it. docker compose watch keeps the code in the image and copies changed files in, which means every read the application makes is an overlay2 read.

    develop:
      watch:
        - action: sync
          path: ./app
          target: /rails/app
        - action: sync+restart
          path: ./config
          target: /rails/config
        - action: rebuild
          path: Gemfile.lock

Same app, same evented watcher, bootsnap on in both, 300 keep-alive requests from inside the container: 0.84 ms p50 on the synced image against 1.96 ms p50 on the bind mount. Then the same 15 edits measured end to end, from writing the file on the host to the new string coming back over HTTP:

=== bind mount (3051) ===
port=3051 served=15 stuck=0
min 0.126s  median 0.128s  max 0.146s
=== compose watch (3052) ===
port=3052 served=15 stuck=0
min 0.618s  median 0.670s  max 0.695s

Requests are 2.3 times faster and saves are 5 times slower. For a page you reload by hand, 0.670 s is noticeable and 1.1 ms of request time is not, so the bind mount with the evented watcher wins for ordinary work. compose watch earns its place on a suite run or a load test, where the number of file reads is large and nobody is waiting on a save.

The dead end: the bind mount was not eating the edits

The first reload measurement on the bind mount came back with half the edits never appearing. Thirty edits, each one polled for 10 seconds:

served: 16  stuck: 14
min 0.023s  median 0.041s  max 0.168s

Every stale iteration was reproducible, alternating, and looked exactly like a file sharing bug. The container's copy of the file was checked while it was stale: content new, mtime new, stat -c "%Y %.9Y" showing nanoseconds, and Rails still serving the previous version.

Two hypotheses were tested and both were wrong. ActiveSupport::FileUpdateChecker#max_mtime skips any file whose mtime is in the future relative to the process clock, at file_update_checker.rb:137, so a container clock running behind the host would make fresh edits invisible. A probe that reported mtime - Time.now at the moment each host write became visible measured the skew at between -0.002 s and -0.015 s, the wrong sign and three orders of magnitude too small. Attribute caching was ruled out the same way: a poll inside the container saw each new mtime within 50 ms of the host write.

Then the same thirty edits were run against the same application served by bin/rails server on the laptop, no container involved:

served: 15  stuck: 15
min 0.013s  median 0.020s  max 0.059s

It was bootsnap. bs_cache_key in ext/bootsnap/bootsnap.c stores key->mtime = (uint64_t)statbuf.st_mtime, whole seconds, and cache_key_equal_fast_path returns hit when the Ruby version digest, the byte size and that second all match. Two edits inside one second that keep the file the same length are one cache key, so the second one loads the compiled bytecode of the first. The content digest in the key is only consulted on the slow path, which Bootsnap.default_setup leaves off: revalidation: bool_env("BOOTSNAP_REVALIDATE").

DISABLE_BOOTSNAP=1 took both runs to 30 served, 0 stuck. Changing the length of the written string by one byte on each iteration did the same thing, 15 of 15 on both stacks. Nothing about it is specific to Docker, and a normal human editing session rarely produces two same-length edits inside one second, but a script that does will spend an hour blaming the wrong layer.

The three files

Dockerfile.dev, kept separate because the generated Dockerfile is a production build and says so on its own fourth line:

# syntax=docker/dockerfile:1
ARG RUBY_VERSION=4.0.5
FROM docker.io/library/ruby:$RUBY_VERSION-slim

RUN apt-get update -qq && \
    apt-get install --no-install-recommends -y \
      build-essential git libpq-dev libvips libyaml-dev pkg-config postgresql-client && \
    rm -rf /var/lib/apt/lists /var/cache/apt/archives

WORKDIR /rails

ENV RAILS_ENV="development" \
    BUNDLE_PATH="/usr/local/bundle"

COPY Gemfile Gemfile.lock ./
RUN bundle install

CMD ["bin/rails", "server", "-b", "0.0.0.0"]

BUNDLE_PATH is the line that matters and it is the one most often written wrong. Point it at /rails/vendor/bundle, which is the idiomatic place, and the bind mount of the working tree covers the directory the gems were installed into. The container then boots to:

Could not find rails-8.1.4, propshaft-1.3.2, pg-1.6.3-aarch64-linux, [... the rest of the lockfile ...] in locally installed gems (Bundler::GemNotFound)
    from /rails/config/boot.rb:3:in 'Kernel#require'

Gems belong outside the mounted tree. /usr/local/bundle is already the image default and needs no help.

compose.yaml:

name: railsdockerdev

services:
  db:
    image: postgres:18
    environment:
      POSTGRES_PASSWORD: postgres
    volumes:
      - pgdata:/var/lib/postgresql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres"]
      interval: 2s
      timeout: 2s
      retries: 30

  web:
    build:
      context: .
      dockerfile: Dockerfile.dev
    command: bash -c "rm -f tmp/pids/server.pid && bin/rails server -b 0.0.0.0"
    environment:
      DATABASE_HOST: db
      DATABASE_USER: postgres
      DATABASE_PASSWORD: postgres
    volumes:
      - .:/rails
    ports:
      - "3051:3000"
    depends_on:
      db:
        condition: service_healthy

volumes:
  pgdata:

There is no setup service and no entrypoint doing db:prepare. The generated bin/docker-entrypoint only runs it when the last two arguments are exactly ./bin/rails and server, which -b 0.0.0.0 already breaks, so bin/rails db:prepare is run by hand once and the file stays honest. From docker compose down -v to a stack answering requests, with the gem layer cached:

docker compose up -d --build  0.13s user 0.12s system 1% cpu 18.944 total
Created database 'blog_development'
Created database 'blog_test'

The third file is config/environments/development.rb, which needs exactly two lines added:

  config.file_watcher = ActiveSupport::EventedFileUpdateChecker
  config.web_console.permissions = "172.16.0.0/12"

The suite

Six examples, run inside the web service, asserting the claims on this page that a machine can check: that the working tree really is a mount rather than an image layer, that the test environment is not pointed at the development database, that postgres:18 put its data where the version says, and that bootsnap does and does not reuse a compile depending on one byte of length.

$ docker compose exec -e RAILS_ENV=test web bin/rails test test/docker_dev_test.rb
Running 6 tests in a single process (parallelization threshold is 50)
Run options: --seed 41252

# Running:

......

Finished in 0.029064s, 206.4418 runs/s, 516.1046 assertions/s.
6 runs, 15 assertions, 0 failures, 0 errors, 0 skips

The bootsnap pair is the interesting one, because it is the page's dead end turned into an assertion. Both writes are forced to the same second with File.utime, which is what a fast editor does by accident:

File.write(file, "def bootsnap_probe = :aaa\n")
File.utime(second, second, file)
load file.to_s
assert_equal :aaa, bootsnap_probe

File.write(file, "def bootsnap_probe = :bbb\n")
File.utime(second, second, file)
assert_equal 26, File.size(file)
load file.to_s

assert_equal :aaa, bootsnap_probe,
  "expected the stale compile: same mtime second and same byte size is a cache hit"

The position

Run the whole application in Compose, mount the working tree, and change the watcher. The received advice, that Rails belongs on the host and only Postgres in a container, is answering a real measurement with the wrong fix: the 11.56 ms was the stat storm, the evented watcher takes it to 2.35 ms, and 2.35 ms is faster than the laptop managed at 3.06 ms. What would change this position is a Docker Desktop release that stops forwarding inotify, or a Rails release that makes FileUpdateChecker the only option again. Both are checkable in about two minutes with the rb-inotify probe above.

The image is the cost. 933 MB for the development image against 594 MB for the production one, and 17.7 s to rebuild after touching the Gemfile, of which 15.7 s is bundle install with nothing cached between builds.

What this page does not cover

Only Docker Desktop on Apple Silicon, one machine, one file sharing implementation. Colima, Orbstack, Podman and Docker on Linux all move the boundary the stat calls cross, and the whole argument above is about that boundary, so none of these numbers transfer without being re-run.

Nothing here is about Dev Containers, which the generated Dockerfile points at in a comment and which rails new --devcontainer generates: that is a different mechanism with a different editor story. Nothing here is about production, where none of this applies, because the reloader is off and the code is in the image; deploying Rails with Kamal is the other end of that. Nothing here is about JavaScript or CSS build watchers, which run as their own compose services and have their own polling question. And the compose file above has one web service and no jobs service, so the question of whether Solid Queue should share a container with Puma in development is left where it is.

#rails #docker #performance

Comments

No comments yet. Be the first.

Only used to confirm and publish your comment. Never shown publicly, never shared.

Markdown: **bold**, `code`, ```fenced blocks```, > quotes, [links](url). HTML and images are not rendered.