Parsing JSON straight into objects, with a hook almost nobody uses
Text version
You receive a data file from a third-party API or an SFTP drop. It's a list of records, each one tagged with what it is and carrying its own attributes:
[
{ "resource_type": "Legend", "data": { "email": "jim@example.com", "fullname": "Jim Weirich" } },
{ "resource_type": "Legend", "data": { "email": "chris@example.com", "fullname": "Chris Seaton" } }
]
The usual answer is two passes. JSON.parse turns the file into an array of hashes, then you walk
that array and turn each hash into a model. Both loops run over the same data, and the first one
builds something you throw away.
Ruby's JSON library has a hook that removes the second pass. It's in the standard library, it's been there for years, and almost nobody reaches for it.
Everything below is the json 2.x API, and the gem's 3.0 line removed it: on json 3.0.2,
JSON.create_id is undefined and JSON.parse answers ArgumentError: unknown keyword:
create_additions. json is a default gem, so which line you're on is decided by what your
Gemfile.lock resolved, not by your Ruby version.
The key that names the class
Start in IRB, because the default is the thing worth seeing:
require "json"
JSON.create_id
# => "json_class"
create_id is the key the parser looks for in each object to decide which class to hand it to.
The default is json_class, which is what Ruby's own serialisation writes. It's also a setter,
and that's the whole trick:
JSON.create_id = "resource_type"
Now the parser looks for resource_type, the key their file already carries. You didn't negotiate
anything with the provider and you didn't rewrite anything on the way in. You pointed the parser
at their vocabulary and made it your dispatch key.
Opting a class in
A class joins in by defining one class method:
class Legend
attr_reader :email, :fullname
def initialize(json)
@email = json["email"]
@fullname = json["fullname"]
end
def self.json_create(object)
new(object["data"])
end
end
json_create receives the whole entry, so it can reach wherever the payload keeps the attributes.
Here they sit under data, which is why the hook passes object["data"] to the constructor
rather than the entry itself. That indirection is what you're buying: the shape of their file and
the shape of your initialiser don't have to match any more, and this one method is the only place
the mapping lives.
Watch what it hands back, too. Nothing forces it to return an instance of the class it's defined on. It returns whatever you decide that entry should become.
The flag without which none of it runs
JSON.parse(source, create_additions: true)
# => [#<Legend jim@example.com>, #<Legend chris@example.com>]
Drop create_additions: true and you get an array of hashes. No error, no warning, no hint that a
hook existed and was skipped. The flag is opt-in because the feature instantiates classes named
inside the document, and a parser that did that by default on any input would be a liability.
One bad record doesn't sink the file
Take the same file and remove the key from the second entry:
[
{ "resource_type": "Legend", "data": { "email": "jim@example.com" } },
{ "data": { "email": "chris@example.com" } }
]
JSON.parse(source, create_additions: true).map(&:class)
# => [Legend, Hash]
The first becomes a Legend, the second stays a Hash. You degrade per record, not per file: one
unrecognised entry doesn't take the import down, and it reaches you in a form you can inspect and
route yourself.
Where the mapping ends up living
You save one iteration, which on a large import is worth having. What you really get is somewhere to put the mapping.
Without the hook, the knowledge of how their records become your objects lives in whatever loop happens to consume the parse, and you'll copy it the second time someone imports the same feed. With it, that mapping sits in one class method on the class that cares.
What this does not do
It isn't a schema validator. json_create hands your constructor whatever the payload contains,
including nothing, so validation is still yours to write.
It isn't for untrusted input either. Turn it on for data whose shape you control or whose provider you trust, because it's a tool for an agreed format, not for parsing the open internet.
And it isn't the Ruby-to-Ruby round trip that json/add/core gives you, where Time, Symbol,
Range and friends serialise themselves and come back as themselves. That's the same machinery
pointed at your own data rather than at somebody else's.
Comments
No comments yet. Be the first.