LaunchKit
← All posts
· 15 min read · by The LaunchKit team · 0 views

Rails health checks and what /up does not check

Rails 8 writes a health check into every new application, one line in config/routes.rb with a comment above it that already tells you most of the truth:

# Reveal health status on /up that returns 200 if the app boots with no exceptions, otherwise 500.
# Can be used by load balancers and uptime monitors to verify that the app is live.
get "up" => "rails/health#show", as: :rails_health_check

That route is doing real work in production for a lot of people, including anything deployed with Kamal, which polls it before it will send traffic to a new container. The question worth answering is not whether to keep it. It is what a 200 from it entitles you to believe, and what happens to your site the day somebody decides it should also check the database.

Everything below was run against Rails 8.1.3.1 on Ruby 4.0.5, with PostgreSQL 17.7 on port 15432, and the output is pasted as it came out.

The whole controller

Rails::HealthController lives in the railties gem, not in your application, and the file is 63 lines including the documentation comment. The class itself:

class HealthController < ActionController::Base
  rescue_from(Exception) { render_down }

  def show
    render_up
  end

  private
    def render_up
      respond_to do |format|
        format.html { render html: html_status(color: "green") }
        format.json { render json: { status: "up", timestamp: Time.current.iso8601 } }
      end
    end

There is no model, no connection, no configuration hook and no extension point. Reaching the action at all is the entire test, and the body is a page painted green so a human who opens the URL in a browser gets an answer without reading a status code.

Mounted in a throwaway application and driven with Rack::Test:

rails 8.1.3.1
HTML  -> 200 text/html; charset=utf-8 "<!DOCTYPE html><html><body style=\"background-color: green\"></body></html>"
JSON  -> 200 {"status":"up","timestamp":"2026-09-24T14:23:28Z"}
.json -> 200 {"status":"up","timestamp":"2026-09-24T14:23:28Z"}
csv   -> 406 ""

The JSON branch is new. It arrived in Rails 8.1 through rails/rails#55092, and if your monitoring tool wants a parseable body rather than a status code, that is now free. The 406 on anything else is worth knowing before you point a checker at /up.xml.

rescue_from(Exception) is what makes the 500 half of the promise. Anything raised inside the request cycle for that action turns the page red. Since HealthController inherits from ActionController::Base and not from your ApplicationController, the only application code that can raise there is code attached to ActionController::Base itself, which in practice means an initializer. One that does:

ActiveSupport.on_load(:action_controller_base) do
  before_action { raise ArgumentError, "an initializer put me here" }
end

produces exactly what the routes comment advertises:

html: 500 "<!DOCTYPE html><html><body style=\"background-color: red\"></body></html>"
json: 500 {"status":"down","timestamp":"2026-09-24T14:23:55Z"}

A failed boot has nobody to answer the phone

"Returns 500 if the app did not boot" is the part of the comment that is not quite right, and it is worth being precise because it changes what a monitor should alert on. An application that fails during initialization does not serve a red page. It does not serve anything. An initializer that raises, put in front of Puma 8.0.2:

Puma starting in single mode...
* Puma version: 8.0.2 ("Into the Arena")
...
exit=1

and the request from the other side:

curl /up -> http 000
curl exit: 7

Exit code 7 is CURLE_COULDNT_CONNECT. There is no process listening. So the two failure signals are different shapes: a boot failure is a refused connection, a request-time failure is a 500, and a checker that only treats non-200 responses as unhealthy catches both by accident while a checker written around response bodies catches one. The distinction matters more during a deploy than during steady state, because a deploy is when boot failures happen.

What a green page proves about the database

Nothing. The controller's own documentation says so, in the source at health_controller.rb:28:

# NOTE: This endpoint does not reflect the status of all of your application's
# dependencies, such as the database or Redis cluster. Replace
# "rails/health#show" with your own controller action if you have
# application specific needs.

Measured rather than quoted: an application whose config/database.yml points at 10.255.255.1, an address that swallows packets, with one controller that runs SELECT 1 and the stock health route next to it.

GET /up        -> 200 in 0.146 s  green
GET /dashboard -> 500 in 1.019 s

The health check is green while every page that reads a row is a 500. If your monitoring stops at /up, that is a site that is down and a dashboard that is not paging anybody.

This is the moment most teams reach for a health check that opens a connection, and the next two sections are about why that instinct is right about the problem and wrong about the place to fix it.

The arithmetic that takes the fleet down

A health check that touches the database is a request like any other. It occupies a Puma thread while it waits, and it waits for exactly as long as the database takes to not answer.

The measurement: Puma with -t 3:3, a /up that blocks for 5 seconds standing in for a connection checkout against a sick database, and /pricing, a page that needs no database at all. Three health checks in flight, then one real request:

warmup /pricing 0.021205s
/pricing while 3 health checks are stuck: 4.499601s
/pricing after they finish: 0.001190s

A page that went out in 21 milliseconds took four and a half seconds, for a reason that has nothing to do with the page. That is one server.

The blocking has two sources, and neither is hypothetical. When the pool has no free connection, the wait is checkout_timeout, 5 seconds by default, and the error carries the number:

ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool within 5.000 seconds (waited 5.005 seconds); all pooled connections were in use

When the pool tries to open a new connection to a host that is not answering, the wait is twice what you configured, because connection_retries defaults to 1 at active_record/connection_adapters/abstract_adapter.rb:222 and reconnect! retries once with a small backoff:

connect_timeout=1 -> 2.13 s  ActiveRecord::DatabaseConnectionError
connect_timeout=2 -> 4.11 s  ActiveRecord::DatabaseConnectionError
connect_timeout=4 -> 8.12 s  ActiveRecord::DatabaseConnectionError
raw PG.connect connect_timeout=2 -> 2.01 s  PG::ConnectionBad

Put the pieces together for a fleet. Ten servers, RAILS_MAX_THREADS at the generated default of 3 and the pool sized to match, a load balancer checking every server every second. The database gets slow, not dead, the kind of slow a lock queue produces. Every health check now sits in the pool queue for 5 seconds, each server's three threads fill with health checks, real requests queue behind them, and the balancer starts timing out on servers whose application code is fine. All ten fail in the same second, because they are all failing on the same shared thing, and there is no healthy capacity left to shift traffic to. A check added to catch a partial outage converted it into a total one.

That pattern is called correlated failure, and a dependency check on the health route is an unusually efficient way to build one: every server polls the shared thing constantly, and they all reach the same verdict within one interval.

Liveness and readiness are two different questions

The reason the advice "do not check the database in your health check" keeps producing arguments is that it collapses two questions with different consequences. Kubernetes separates them, and the wording in its documentation is the clearest statement of the stakes. On a liveness failure, "the kubelet restarts that container". On a readiness failure, "the EndpointSlice controller removes the Pod's IP address from the EndpointSlices of all Services that match the Pod".

Restart, against stop sending traffic. That is the whole distinction, and it decides the content of each check:

  • Liveness should answer "would restarting this process help?" A deadlocked Puma, a wedged worker, a process out of file descriptors: yes. A database that is down: no, and restarting all ten servers into a thundering herd of reconnects while the database is already struggling is actively worse. rails/health#show is a good liveness check almost by accident, because it checks the process and nothing else.
  • Readiness should answer "should this instance get the next request?" Here a dependency check can be defensible, and it is the only place it is defensible, because the consequence is bounded at losing traffic rather than losing processes. It is still a correlated signal, so it still needs a timeout well under the probe timeout and it still should not be your only alerting.

Rails ships one endpoint and takes no position on which of the two it is. Most deploy tooling also offers one hook, which means the decision is made for you: whatever /up returns is treated as both. Given one endpoint and two questions, the liveness answer is the safe one to give.

What a deploy tool wants from it

Kamal is the case this boilerplate is built around, and the deploy half of the story, including what kamal-proxy grades and the two failures that do not look like their cause, is in Deploying Rails with Kamal 2. The part that belongs here is what the proxy's defaults are, since nothing in your repository states them. From kamal-proxy's internal/server/service.go:

DefaultDeployTimeout         = time.Second * 30
DefaultDrainTimeout          = time.Second * 30

DefaultHealthCheckPath     = "/up"
DefaultHealthCheckInterval = time.Second
DefaultHealthCheckTimeout  = time.Second * 5

Kamal 2.12.0 only passes those flags when you set them. In configuration/proxy.rb:78, the three health check options come out of proxy_config.dig("healthcheck", ...) and the whole hash goes through .compact, so an unset value is an absent flag and the Go default stands. The default path matches the Rails default, which is why none of this ever needs configuring and also why nobody knows what the numbers are.

Two consequences follow from a 1-second interval with a 5-second timeout. First, a check that regularly takes more than 5 seconds is a failed deploy, so any dependency check you add has to be written with a timeout tighter than that, not left to the database's own. Second, the polling is only for the cutover: Kamal's documentation says "once the app is up, the proxy will stop hitting the healthcheck endpoint". Continuous monitoring is somebody else's job, which in most setups means an external uptime checker hitting the same URL forever.

The container that gets no health check at all is the one running jobs. Kamal's readiness delay is documented as applying "only to containers that do not run a proxy or specify a healthcheck", and the poller is honest about what it does instead:

if readiness_delay > 0
  info "Container is running, waiting for readiness delay of #{readiness_delay} seconds"
  sleep readiness_delay
  status = block.call
end

Seven seconds and a second look at whether Docker still calls it running. The job: role in this boilerplate's config/deploy.yml is commented out, and until you uncomment it Solid Queue runs inside Puma through plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"], which means the web health check covers the job supervisor by accident. Split jobs onto their own machine and that coverage disappears with no warning, because the new container has no HTTP server for anything to poll.

The silencing that stops one character short

One health check per second is 86,400 requests a day, three log lines each, describing nothing. Rails has a switch for it, and the generated config/environments/production.rb sets it, line 53 in this boilerplate:

config.silence_healthcheck_path = "/up"

The middleware behind it is one comparison, in rails/rack/silence_request.rb:

def call(env)
  if @path === env["PATH_INFO"]
    Rails.logger.silence { @app.call(env) }
  else
    @app.call(env)
  end
end

=== on a String is string equality, and the documentation for the middleware shows a Regexp being passed instead, path: /test$/, which is the hint that the String case is exact. With the config set to "/up" and a logger capturing output:

GET /up: status 200, 0 log lines
GET /up.json: status 200, 5 log lines
   | Started GET "/up.json" for 127.0.0.1 at 2026-09-24 16:26:38 +0200
   | Processing by Rails::HealthController#show as JSON
   | Completed 200 OK in 0ms (Views: 0.0ms | GC: 0.0ms)
   |
   |
String#=== check: false

So the JSON format that 8.1 just added is the format that defeats the silencer that arrived in Rails 8.0 through rails/rails#52789. If your uptime monitor asks for /up.json to get a parseable body, set config.silence_healthcheck_path = %r{\A/up(\.json)?\z} instead, or point the monitor at /up with an Accept: application/json header, which is silenced because the path still matches. Both were checked:

regexp silencer, /up: 200, 0 non-blank log lines
regexp silencer, /up.json: 200, 0 non-blank log lines
regexp silencer, /up Accept json: 200, 0 non-blank log lines

Two middlewares answer before the controller does

The health check runs at the end of the middleware stack, and two entries above it can turn a healthy application into a failing check. Both are one line to fix and both are invisible until a deploy fails.

Host authorization is the first. ActionDispatch::HostAuthorization compares the Host header against config.hosts, and a load balancer or a Docker health check usually connects by IP, which means the Host header is an IP address:

-- /up requested by container IP (Host: 10.0.2.15)
  403

The middleware exposes an escape hatch for exactly this, and Rails 8 generates it commented out at the bottom of production.rb:

# config.host_authorization = { exclude: ->(request) { request.path == "/up" } }

The second is ActionDispatch::SSL. With config.force_ssl = true, a plain HTTP request to /up is a 301 to the HTTPS URL, and a checker that does not follow redirects records that as a failure. This boilerplate ships the exclusion rather than the comment:

config.ssl_options = {
  hsts: { expires: 1.year, subdomains: true, preload: false },
  # Let load balancers hit the health check over plain HTTP (no redirect).
  redirect: { exclude: ->(request) { request.path == "/up" } }
}

Same application, two paths, one of them excluded:

-- force_ssl with exclude, correct Host
  200 nil
-- same app, another path
  301 -> "https://launchkit.codes/healthz"

Both exclusions are path equality against the String "/up", the same comparison the log silencer uses, so moving the health check to /healthz means editing three places and the compiler will tell you about none of them.

When a richer check earns its place

My position is that /up should stay exactly as dumb as it is, and that a dependency check belongs on a second route with a different name, called by different things. /up for the load balancer and for liveness, /up/deep or similar for the deploy pipeline, a post-deploy smoke test, and a dashboard a human reads. Different callers, different blast radius, and the deep one can afford to be slow because nothing restarts a process over it.

What would change my mind: an application that cannot serve a single useful response without its database. A pure JSON API over one Postgres has no degraded mode to protect, so an instance that cannot reach the database really is useless, and taking it out of rotation costs nothing you had. Even then the check belongs in readiness, never in liveness, and it needs its own timeout.

If you want the richer check off the shelf, okcomputer is the one still being maintained: 1.20.0 was released on 2026-09-11, it registers named checks and reports them individually, and it has a timeout per check. The health_check gem is widely deployed and its last release is 3.1.0 from May

  1. Writing the second route yourself is roughly fifteen lines, and the shape that matters is this one:
class DeepHealthController < ActionController::Base
  def show
    ActiveRecord::Base.lease_connection.select_value("SELECT 1")
    head :ok
  rescue ActiveRecord::ConnectionTimeoutError, ActiveRecord::DatabaseConnectionError => e
    render plain: e.class.name, status: :service_unavailable
  end
end

with connect_timeout and checkout_timeout set low in the connection this action uses, so the action fails in under a second rather than in the 4.11 seconds measured above. Against a database at 10.255.255.1 that route answers:

deep check, db unreachable: 503, body "ActiveRecord::DatabaseConnectionError"

Naming the two error classes rather than rescuing StandardError is deliberate: a NoMethodError in this action is a bug in your code, not an unhealthy database, and it should reach your exception tracker rather than being reported to a load balancer as a sick Postgres.

Not covered here

Nothing above says what to do about a slow dependency that is not the database. Redis, an external payment API and a mail provider all have the same correlated-failure property and none of them belong in a liveness check either, but the timeouts and the degraded modes are specific to each client library. The boilerplate has no Redis at all, for reasons that are their own post. Kubernetes probe configuration, startup probes and terminationGracePeriodSeconds are also out of scope, since this deploys with Kamal.

#rails #deployment

Comments

No comments yet. Be the first.

Only used to confirm and publish your comment. Never shown publicly, never shared.

Markdown: **bold**, `code`, ```fenced blocks```, > quotes, [links](url). HTML and images are not rendered.