Active Record encryption and the query that finds nothing

Active Record encryption is four characters of configuration and it does what it says: the column holds ciphertext, the attribute holds plaintext, and nothing in between needs to know. The part that catches people is not the encryption. It is that a database column stops being a database column the moment you encrypt it, and Rails does not announce which of your existing queries just stopped working.

Everything below ran on activerecord 8.1.3.1 and Ruby 4.0.5 against PostgreSQL 17.7 on port 15432. The ciphertexts, the plans and the byte counts are copied out of the runs.

What encrypts writes into the column

encrypts :ssn decorates the attribute type. Nothing else changes: the column stays whatever you declared, the model stays a model, and the encryption happens in serialize and deserialize on ActiveRecord::Encryption::EncryptedAttributeType. So the INSERT looks entirely normal and the bound parameter is the only surprise:

Person Create  INSERT INTO "people" ("name", "email", "ssn", ...) VALUES ($1, $2, $3, ...)
  [["name", "a"],
   ["email", "{\"p\":\"sVFI4YJ9GROMUu7LJV7I\",\"h\":{\"iv\":\"NDHiRXi8pm2mpKua\",\"at\":\"UpvEIpOT4da66U0Qwecc4A==\"}}"],
   ["ssn",   "{\"p\":\"LgWSj7Mrt3mYs2Y=\",\"h\":{\"iv\":\"oCYBDY41WbWjEF+r\",\"at\":\"PCr2dFVZ2TfyWWL5LMjP4w==\"}}"]]

Three fields, all base64. p is the payload, iv is the 12 byte initialization vector, at is the 16 byte GCM authentication tag. The cipher is aes-256-gcm, hardcoded as CIPHER_TYPE in active_record/encryption/cipher/aes256_gcm.rb, and ActiveRecord::Encryption.cipher.key_length answers 32 with iv_length 12.

"ada@example.com" is 15 bytes of plaintext and 90 bytes in the column. That ratio is the whole sizing problem, and it has its own section below.

One thing you get for free and should know about: encrypts registers the attribute with ActiveRecord::Encryption::AutoFilteredParameters, which appends "person.ssn" to config.filter_parameters and adds :ssn to the model's filter_attributes. So the column is redacted in the log and in inspect without you writing a filter. Turn it off with config.active_record.encryption.add_to_filter_parameters = false, which you will want approximately never.

The three keys and which column each one reaches

Generating the Rails encryption keys is one rake task, and the task is small enough to read. bin/rails db:encryption:init is one puts of a heredoc in railties/databases.rake, and the whole of it is three calls to SecureRandom.alphanumeric(32) printed inside a YAML block:

active_record_encryption:
  primary_key: YehXdfzxVKpoLvKseJMJIEGs2JxerkB8
  deterministic_key: uhtk2DYS80OweAPnMLtrV2FhYIXaceAy
  key_derivation_salt: g7Q66StqUQDQk9SJ81sWbYZXgiRogBwS

The task generates and prints. It does not write anything, so paste them into bin/rails credentials:edit yourself. None of the three is an AES key: each is a password that gets run through PBKDF2. Context#build_default_key_provider returns DerivedSecretKeyProvider.new(config.primary_key), which calls ActiveSupport::KeyGenerator#generate_key, which is OpenSSL::PKCS5.pbkdf2_hmac at 2**16 iterations with key_derivation_salt as the salt and 32 as the length.

The division of labour is the fact worth memorising. primary_key derives the key for every non-deterministic column. deterministic_key goes through DeterministicKeyProvider, a subclass that adds one guard and nothing else, and serves every column declared deterministic: true. key_derivation_salt is the salt on both paths. The two key spaces never meet, which you can watch: write a row, restart with a different primary_key, and the non-deterministic column raises while the deterministic one reads fine.

non-deterministic column: ActiveRecord::Encryption::Errors::Decryption
deterministic column:     "vip"

Note that the guide says the recommended minimum is "12 bytes for the primary key and 20 bytes for the salt", so 32 is generous rather than required. Note also that a missing key is a boot-time problem only if something reads it: Config raises Missing Active Record encryption credential: active_record_encryption.deterministic_key lazily, on first use, which in practice means the first request that touches an encrypted column in an environment where nobody ran the task.

Deterministic mode is twelve bytes of HMAC

The entire difference between the two modes is one private method:

def generate_deterministic_iv(clear_text)
  OpenSSL::HMAC.digest(OpenSSL::Digest::SHA256.new, @secret, clear_text)[0, ActiveRecord::Encryption.cipher.iv_length]
end

Non-deterministic encryption calls cipher.random_iv. Deterministic encryption derives the IV from the key and the plaintext, so the same input always produces the same twelve bytes, and AES-GCM with a fixed key and a fixed IV is a pure function. Same plaintext, same ciphertext, forever.

That identity is global, not per row and not per table. Two different models with a deterministic email produced byte-identical column values:

authors.email == readers.email ? true
{"p":"EK74G1vPYWPTkNVSqEe/pvrAYg==","h":{"iv":"POVaFyxbo+igRvFk","at":"BVK+2cPBf2tR7FXle6OhDg=="}}

Which is the feature. Equality comparison in SQL works because the server is comparing two byte strings that Ruby produced from the same input, and it has no idea either of them is ciphertext. It is also the cost, and the guide is blunt about it: "The :deterministic option allows for querying by trading off lesser security. The data is still encrypted but the determinism makes crypto-analysis easier."

The query that finds nothing and raises nothing

Here is the behaviour that costs people an afternoon. A where on a non-deterministic column is accepted, runs, and returns zero rows.

class Person < ActiveRecord::Base
  encrypts :email, deterministic: true
  encrypts :ssn
end

Person.create!(name: "a", email: "ada@example.com", ssn: "111-22-3333")
Person.create!(name: "b", email: "ada@example.com", ssn: "111-22-3333")

Person.where(ssn: "111-22-3333").count  # => 0

Two rows hold that exact value. The count is 0. No exception, no warning, no log line saying anything is wrong. Active Record did precisely what you asked: it serialized the search term through the attribute type, which encrypted it with a fresh random IV, and handed PostgreSQL a bound parameter that has never existed in that column and never will.

Watch the two statements in one process, both searching for the same string:

Person Load   SELECT "people".* FROM "people" WHERE "people"."ssn" = $1
  [["ssn", "{\"p\":\"cCjthKJnsbbEHQI=\",\"h\":{\"iv\":\"X+Ysvsvso17eKkPm\",\"at\":\"QzPtXA9UJdm5L6YY3p1lZA==\"}}"]]
Person Count  SELECT COUNT(*) FROM "people" WHERE "people"."ssn" = $1
  [["ssn", "{\"p\":\"6KUBUs/E1YHW+Ps=\",\"h\":{\"iv\":\"cxLJACldCIIzHgm4\",\"at\":\"kWXBks9OK1Y4l4vh5GJRiQ==\"}}"]]

Different bytes, same string, milliseconds apart. Randomised IV is the point of non-deterministic encryption and it is working exactly as designed.

What makes this expensive is that it is a silent behaviour change on code that already exists. You add encrypts :ssn to a model, every find_by, every where, every uniqueness validation on that column quietly starts answering "no such record", and the specs that covered them only fail if a fixture actually matched. A spec that asserts an empty result stays green. A spec that asserts one result fails, and the failure says "expected 1, got 0", which reads like a fixture problem.

There is no framework-level guard here. deterministic: false columns are write-only as far as SQL is concerned, and you are expected to know that.

What the index on an encrypted column still does

The index does not break. That is the problem with it.

Fifty thousand contacts, the same email in two columns, one plain and one deterministic, a plain B-tree index on each:

rows: 50000
avg plaintext bytes:  23
avg ciphertext bytes: 102
index_contacts_on_email_plain: 3504 kB
index_contacts_on_email_enc:   8632 kB

Two and a half times the index for the same 50,000 values, which follows from indexing 102 bytes instead of 23. Equality lookups work and are fast, because that is what deterministic encryption buys. Everything else about a B-tree is now decorative.

A prefix search, on both columns, same statement shape:

-- email_plain
Index Scan using index_contacts_on_email_plain on contacts (actual rows=11 loops=1)
  Index Cond: (((email_plain)::text >= 'person4999'::text) AND ((email_plain)::text < 'person499:'::text))

-- email_enc
Index Scan using index_contacts_on_email_enc on contacts (actual rows=0 loops=1)
  Index Cond: (((email_enc)::text >= 'person4999'::text) AND ((email_enc)::text < 'person499:'::text))

Identical plans. Both use their index. Both are fast. One returns 11 rows and one returns 0, and nothing in the plan suggests which is which. The planner is doing correct range work over a sort order that has no relationship to the sort order of the plaintext, because base64 of AES-GCM output is noise.

ORDER BY has the same shape of failure. Ordering 20,000 rows by the encrypted email column returned ids [17395, 5500, 10337], which is a real, stable, repeatable order, and it is the alphabetical order of the ciphertexts. Sorting a user list by email is now sorting it by nothing.

Unique indexes are the exception and they work properly, because a unique index only ever asks about equality. So does validates_uniqueness_of: ActiveRecord::Validations::UniquenessValidator needed no help to reject a duplicate on a deterministic column, and answered Email has already been taken. Just make sure the column is deterministic before you rely on either, because on a non-deterministic column the unique index is satisfied by every row and the validation passes always.

What a deterministic column gives away

Determinism leaks the frequency distribution of your data to anyone holding a copy of the table, and they do not need the key to read it.

Eight rows, a deterministic plan column, and one GROUP BY:

5 rows share {"p":"FgA6sg==","h":{"iv":"kDHmlq7xOvxgV5i9","at":"d...
2 rows share {"p":"o0Nz","h":{"iv":"Vlu8RLm2B8FAbv4T","at":"Hjnd+...
1 rows share {"p":"S5WvT7OBI9wJSw==","h":{"iv":"K412vv2H+zHlUZP1"...

Nothing is decrypted and the whole shape of the column is visible. On a low cardinality column such as a plan name, a country, a status or a boolean-ish flag, that histogram plus one known value plus public knowledge of your pricing page is usually enough to label every group. The payload lengths are visible too: p is base64 of a GCM ciphertext, which below the compression threshold is exactly as long as the plaintext, so every row advertises the length of its secret.

This is why the default is non-deterministic and why the guide says non-deterministic "is recommended for all data unless you need to query by the encrypted attribute". Read that as a per-column decision, not a per-application one. An email address on a deterministic column is defensible, because the reason you encrypted it was the dump and not the frequency analysis, and you genuinely need find_by. A diagnosis column on eleven possible values is not.

Sizing the column, and where the 255 bytes come from

Measured overhead, real English prose, non-deterministic, in a text column:

plain    stored   ratio    compressed?
11       86       7.82     false
40       126      3.15     false
139      258      1.86     false
141      231      1.64     true
300      231      0.77     true
1000     239      0.24     true
4000     259      0.06     true

Two regimes, and the boundary is a constant in Encryptor:

# This threshold cannot be changed.
THRESHOLD_TO_JUSTIFY_COMPRESSION = 140.bytes

Below it, you pay roughly 70 bytes of envelope plus a third again for base64. Above it, Zlib runs first and long text gets cheaper than it was. The comment explains why the number is frozen: change it and the same plaintext would produce a message with or without the "c" header depending on which version of Rails wrote it, and deterministic lookups on existing data would stop matching.

The guide's "worst-case overhead to be around 255 bytes" is not the default configuration. The default DerivedSecretKeyProvider put a 19 byte plaintext in 98 bytes. Swapping in EnvelopeEncryptionKeyProvider, which generates a random data key per operation and stores it encrypted in a k header, put the same value in 240 bytes, and adding store_key_references = true put it in exactly 255. That is where the number comes from, and if you are on the default provider the practical rule for short Western text is about 120 bytes, not 255.

The guide's other point stands whatever provider you use, and it is the one that surprises people. string(255) in PostgreSQL counts 255 characters, and a UTF-8 character is up to four bytes, so that declaration was holding up to 1020 bytes of Cyrillic. Ciphertext is ASCII base64, where a character is a byte, so the same declaration now buys you 255. That is why the guide's table recommends string(510) for email addresses and string(1020) for a short sequence of emojis. Size encrypted string columns in bytes, and when the answer is awkward use text and stop counting.

The length validation that checks the wrong string

config.active_record.encryption.validate_column_size defaults to true and sounds like it protects you from the previous section. It does not. The implementation, in EncryptableRecord#validate_column_size, is one line:

validates_length_of attribute_name, maximum: limit

That validates the attribute, which is the plaintext. On a string(40) column holding an encrypted attribute, a 41 character plaintext is rejected in Ruby:

41 plaintext chars valid? false -> ["Short is too long (maximum is 40 characters)"]

and a 5 character plaintext passes validation and then hits the database:

5 plaintext chars: ActiveRecord::ValueTooLong: PG::StringDataRightTruncation: ERROR:  value too long for type character varying(40)

Five characters, 40 character column, ValueTooLong. The validation enforces a limit the stored value was never going to respect, and it enforces it against the one string that is not going into the column. Treat it as a leftover from the unencrypted schema rather than as a guard, and size the column yourself.

While you are looking at the column: encrypts on a non-string column does not do anything useful. An integer column got PG::InvalidTextRepresentation: ERROR: invalid input syntax for type integer: "{"p":"zYc=",...}", because the ciphertext is text and the column is not. Encrypted columns are string, text or binary.

Turning it on for a table that already has rows

Two settings exist for the migration window and both default to false.

support_unencrypted_data = true makes reads tolerate plaintext. The mechanism is a fallback in EncryptedAttributeType#handle_deserialize_error: decryption raises, and if the attribute supports unencrypted data the raw value is returned as-is. Without it, the same row raises ActiveRecord::Encryption::Errors::Decryption, and that error is worth seeing once before you meet it in production, because e.message is "ActiveRecord::Encryption::Errors::Decryption". The class is the whole message. Nothing tells you which record, which attribute or which key was tried.

extend_queries = true is the one people miss, and it is what makes queries find the rows that have not been backfilled yet. With it off, a deterministic lookup matched the 5 encrypted rows and missed the 1 plaintext row. With it on it found all 6, by putting both forms in the statement:

SELECT "notes".* FROM "notes"
WHERE "notes"."plan" IN ('{"p":"FgA6sg==","h":{"iv":"kDHmlq7xOvxgV5i9","at":"dq29q8L9aV4slAvIZiwTVQ=="}}', 'free')

Its default is false and the code says why, in a comment on Config#set_defaults: "TODO: Setting to false for now as the implementation is a bit experimental". ExtendedDeterministicQueries.install_support prepends a module onto ActiveRecord::Relation and includes one into ActiveRecord::Base, so it is a global patch on query building, and the file carries its own note that support for every kind of query is pending. Turn it on for the migration, turn it off after.

The backfill itself is record.encrypt, one row at a time. It goes through update_columns, so it writes one UPDATE per record, skips validations and callbacks, and leaves updated_at where it was:

Lead Update  UPDATE "leads" SET "email" = $1 WHERE "leads"."id" = $2
after: [[1, 90, 2020-01-01 00:00:00 UTC], [2, 90, 2020-01-01 00:00:00 UTC]]

A leftover timestamp is usually what you want, for the reason Counter caches by hand gives about <lastmod>: a backfill is not a content change. It also means a half-finished backfill leaves no trace on the row, so track progress with an id watermark rather than with updated_at.

The key you can rotate and the key you cannot

primary_key accepts an array, and the last entry is the one new writes use. Written under the old key, read back under both, across separate processes:

both:    "written under the old key"
newonly: ActiveRecord::Encryption::Errors::Decryption

So a rotation is: add the new key to the end of the list, deploy, re-encrypt every row with record.encrypt, then drop the old key from the list. Skip the middle step and the second deploy turns every unmigrated row into a Decryption error at read time.

Deterministic columns do not get this. DeterministicKeyProvider is a DerivedSecretKeyProvider with one added line:

raise ActiveRecord::Encryption::Errors::Configuration, "Deterministic encryption keys can't be rotated" if passwords.length > 1

It has to refuse, and the reason is the feature. Determinism means the ciphertext for a value is a function of the key, so two keys means two ciphertexts for one value, and every equality query and every unique index would have to know about both. Putting two values in deterministic_key raises at configuration time rather than corrupting anything, which is the right call and also means "rotate the deterministic key" is not a maintenance task you can plan. It is a full table rewrite with a migration window, designed in advance or not at all.

Rails already had encryption, and it is a different thing

The LaunchKit boilerplate does not call encrypts anywhere. Its secrets live in per-environment Rails credentials, config/credentials/<env>.yml.enc with a matching key file, read through one AppConfig module. That is ActiveSupport::EncryptedConfiguration, which encrypts a file at rest and decrypts it once at boot, and it is the right tool for a Stripe key because a Stripe key is configuration.

The two get conflated in search results and they solve opposite problems. Encrypted credentials protect a value that every request needs and no request writes, and the whole file is decrypted in memory for the process lifetime. Active Record encryption protects values that arrive from users at runtime, and the point is that a database dump, a replica, a backup or a log line does not contain them. If somebody asks for "rails encryption" and means an API token, they want credentials.

Something in the framework does use encrypts, and it is worth reading as the shortest real example. Solid Cache's encrypt: true resolves to a single call in SolidCache::Entry::Encryption that encrypts the value column and leaves key in the clear, which Solid Cache vs Redis pulls apart along with the custom encryption context it passes. Same mechanism as everything above, with different serializer and compression choices, for reasons specific to a cache table.

The call, and what would change it

Use Active Record encryption, non-deterministic, for anything you would not want in a database dump and never query by: notes, tokens, bank details, free text a user typed. It is one word in the model and there is no second system to run.

Reach for deterministic: true only on columns you actually look up, accept the frequency leak explicitly, and prefer high cardinality values. Email address, yes. Plan name, no.

Skip it entirely when you need real search over the encrypted values. Prefix, range, sort and ILIKE are not coming: the ciphertext has no order and no substrings. The established answer is Lockbox 2.2.0, released 2026-04-04, MIT, Ruby 3.3 or newer, 48.5 million downloads, "Modern encryption for Ruby and Rails", paired with Blind Index 2.8.1 from 2026-06-29, whose one line description is the entire pitch: "Securely search encrypted database fields". Lockbox's README sends you there in a sentence: "If you need to query encrypted fields, check out Blind Index." A blind index is, in its README's words, "a keyed hash of the sensitive data" stored in a column of its own, Argon2id by default at 3 iterations and 4 MB of memory. It is deterministic on purpose and it lets you keep the ciphertext itself non-deterministic, so you get lookups without the ciphertext being a histogram. That is a better shape than deterministic: true, and it costs a gem, a column and a backfill.

The cost of recommending the framework is that you inherit its ceiling. No blind index, no searchable encryption, no per-tenant keys without writing a key provider, and a deterministic key you cannot rotate. What would change it: a deterministic mode built on a separate hash column rather than on a fixed IV would remove both the frequency leak and the rotation dead end, and it would make most of this post about a historical design.

attr_encrypted, the pre-7.0 answer, is at 4.2.0 from 2025-01-23 with 80.5 million downloads, and it is not the one to start with in 2026. Nothing has been released in over eighteen months and the framework now covers its use case.

What this post does not cover

Custom key providers, which is where per-tenant and per-record keys live, and the ActiveRecord::Encryption::Key and KeyProvider contracts you implement to build one. KMS integration, which is what EnvelopeEncryptionKeyProvider exists to be combined with. Encrypting binary columns and the MessagePackMessageSerializer that makes them worthwhile. previous: schemes and the fixed: option, which are how you change encryption properties on a column that already has data. And the ignore_case: true option, which needs a second column named original_<name>, encrypts both, and therefore doubles the storage for that attribute, which is a tradeoff that deserves its own measurements rather than a footnote.

No timings appear anywhere above. Row counts, byte counts, index sizes and ciphertexts reproduce on your machine; a millisecond figure from mine would not.