An HTML generator in one hook, and the Ruby method lookup path
Text version
The goal of this mini project is the best DSL we can get for generating HTML. Here is the result we are aiming for:
HTML.html(
HTML.head,
HTML.body(
HTML.ul(
HTML.li("first li"),
HTML.li("second li")
)
)
)
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"
By the end of this part, the entire generator is one method, and not one of html, head, body,
ul or li is defined anywhere. Getting there means walking the whole Ruby object model, because
the trick that writes those methods for us is a hook at the very end of the method lookup path.
The draft, one method per tag
Take the approach the video calls the DHH one: the first version of a piece of code is always a draft, so start modestly and get a decent result.
module HTML
def self.html(*content) = "<html>#{content.join}</html>"
def self.head(*content) = "<head>#{content.join}</head>"
def self.body(*content) = "<body>#{content.join}</body>"
def self.ul(*content) = "<ul>#{content.join}</ul>"
def self.li(*content) = "<li>#{content.join}</li>"
end
HTML.html(HTML.head, HTML.body(HTML.ul(HTML.li("first li"), HTML.li("second li"))))
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"
That is the target output, on the target DSL, in seven lines. It also has one obvious problem: the HTML standard defines well over a hundred elements, and every one of those methods is the same method with a different name in it. Nobody is writing that by hand.
So the real question of part 1 is not "how do I generate HTML". It is "how do I get Ruby to define those hundred methods for me".
Why a module and not a class
The first decision is what container the generator lives in. It is a module, and the reason is worth being precise about.
A module in Ruby is a data structure that cannot be instantiated. It is a container that holds methods, constants, other modules and classes. Modules hold behavior only. Classes hold behavior and state.
The three usual reasons to reach for one:
- As a mixin, mixed into another module or class with
include,extendorprepend. Each keyword puts the module at a different place in the ancestor chain. - As a namespace, to prevent name clashing and to make the code read better.
- As a standalone entity, self-sufficient, holding module functions and nothing else.
Mathis the one everybody has used.
The generator is the third kind. It produces HTML on demand: there is no state to carry from one call to the next, no object whose life cycle matters, nothing to instantiate. A module of module functions is the exact shape of the problem.
That is also why the hook is defined on self. Every call in the DSL is HTML.something, a method
call on the module object itself, not on an instance of anything.
Classes are objects too
To understand the hook we need the ancestor chain, and to understand the ancestor chain we need the object model. In Ruby, classes are first class objects: they can appear in an expression, be assigned to a variable, and be passed as an argument.
class OtherClass; end
OtherClass.class # => Class
OtherClass.superclass # => Object
So OtherClass is an instance of the Class class, stored in a constant. What you call "a
class" is an object like any other, and the class OtherClass; end syntax is sugar for creating one
and assigning it to a constant.
Class itself has a parent:
Class.superclass # => Module
Module.superclass # => Object
Object.superclass # => BasicObject
A class is a specialization of a module. Everything a module can do, a class can do, plus one thing: it can be instantiated.
OtherClass.new # => #<OtherClass>
OtherModule = Module.new
OtherModule.new # NoMethodError: undefined method 'new' for an instance of Module
That is the whole difference between the two, and it is why the choice in the previous section was a real choice rather than a style preference.
Three names sit at the root of all of this:
BasicObjectis the top parent of every class. It holds the bare minimum: object creation and object comparison.Kernelis a module included inObject. It holds the bulk of the object manipulation logic, and it is whereputs,p,sendandformatactually live.Objectinherits fromBasicObjectand includesKernel. BecauseKernelcarries most of the methods,Objectis used mostly as the interface its name gives to all of its children.
Object.superclass # => BasicObject
Object.include?(Kernel) # => true
method(:puts).owner # => Kernel
That last line is the one to keep. puts is not defined on Object. It is defined on Kernel, and
you can call it anywhere because Object includes Kernel and everything is an Object.
The main object
When a program starts, Ruby creates the main object, an instance of Object. It is the top
level context of any program, most likely a nod to C, where main is the entry point of every
program.
self # => main
self.class # => Object
So a bare puts "hi" at the top of a file is a method call on main, resolved through Object to
Kernel. Nothing is special about the top level; it is just an object you did not have to create.
A method call is a message
What you call a method call is, behind the scenes, a message. A message has a name, usually a symbol, and an optional payload, the argument list. It needs a receiver: an object that responds to that message through a message handler. The sender is always the calling context, which is the main object when you are outside any class.
You can send one explicitly:
"a b c".send(:split, " ") # => ["a", "b", "c"]
Receiver: the string. Message name: :split. Payload: " ". The dot syntax you write every day is
the same thing with nicer punctuation:
"a b c".split(" ") # => ["a", "b", "c"]
This matters here because the hook we are about to define receives exactly those pieces: the message name, the payload, and the block.
The ancestor chain
The ancestor chain is the class hierarchy of an object, in order. It contains the class itself, the modules it includes, its parent class, that parent's included modules, and so on up.
Array.ancestors # => [Array, Enumerable, Object, Kernel, BasicObject]
class OtherClass; end
OtherClass.ancestors # => [OtherClass, Object, Kernel, BasicObject]
OtherClass.included_modules # => [Kernel]
Array includes Enumerable, inherits from Object, which includes Kernel and inherits from
BasicObject. Reading that array top to bottom is reading the order in which Ruby will look for a
method.
The method lookup path
When you call a method, Ruby first checks whether it exists in the context of self. If it does
not, it walks up the ancestor chain until it finds one.
class OtherClass; end
a = OtherClass.new
b = OtherClass.new
a.equal?(b) # => false
equal? is defined on BasicObject. The lookup tries OtherClass, then Object, then Kernel,
then BasicObject, where it finally finds it. That mechanism is the method lookup path.
So what happens when the chain runs out? The responsibility is handed to the method_missing hook,
which is itself looked up the same way, along the same chain. BasicObject ships an implementation
at the end of it, and that implementation is the one that raises:
module Plain; end
Plain.nope # NoMethodError: undefined method 'nope' for module Plain
The familiar NoMethodError is not the absence of a method. It is the return value of the last
method_missing on the chain. Override it earlier on that chain and the error never happens.
The whole generator
module HTML
def self.method_missing(tag, *args, &block)
"<#{tag}>#{args.join}</#{tag}>"
end
end
HTML.html(HTML.head, HTML.body(HTML.ul(HTML.li("first li"), HTML.li("second li"))))
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"
That is it. Every tag in the DSL, from one method.
method_missing is defined on self because the DSL calls module methods. The first parameter is
the method id, a symbol naming the call Ruby could not resolve: :html, :head, :body. It is
already the tag name, which is why no mapping table is needed. Then the argument list, and a block
that part 1 ignores.
args.join is what makes nesting work, and it is worth being explicit about why. Ruby evaluates
arguments before it evaluates the call, so the innermost tags are generated first and the outer call
receives strings. Adding a trace line to the hook shows the order:
method_missing(:head, [])
method_missing(:li, ["first li"])
method_missing(:li, ["second li"])
method_missing(:ul, ["<li>first li</li>", "<li>second li</li>"])
method_missing(:body, ["<ul><li>first li</li><li>second li</li></ul>"])
method_missing(:html, ["<head></head>", "<body><ul>...</ul></body>"])
Six calls, none of which resolve to a method, all of which return a string that the next call up
treats as content. HTML.head with no arguments joins an empty array, so it produces
<head></head>.
The two tag names Kernel already owns
A hook at the end of the lookup path only fires when the lookup fails. Anything already defined
earlier on the chain wins, silently, and HTML is an instance of Module, whose chain is not
empty:
Module.ancestors # => [Module, Object, Kernel, BasicObject]
Intersect the HTML element names with everything reachable on that module object and exactly two come back:
(HTML.methods + HTML.private_methods) & %i[html head body div p ul li select a span]
# => [:p, :select]
Kernel defines both, and this is where it gets interesting: they are private.
Kernel.private_instance_methods & %i[p select] # => [:p, :select]
Kernel.public_instance_methods & %i[p select] # => []
A private method called with an explicit receiver is not callable, and Ruby routes that failure
through method_missing too. So the hook still fires, and both tags work:
HTML.p("x") # => "<p>x</p>"
HTML.select(HTML.a("x")) # => "<select><a>x</a></select>"
Without the hook, the same call reports the real reason:
module Plain; end
Plain.p("x") # NoMethodError: private method 'p' called for module Plain
The place it does bite is inside the module, where there is no explicit receiver. A bare p there
is Kernel#p, it prints to stdout and returns its argument, and no hook is involved:
module Bare
def self.render = p("hello")
end
Bare.render # prints hello, returns "hello"
Part 2 moves the DSL into blocks, where calls are written without a receiver. This is the detail to remember when it does.
What method_missing alone does not do
respond_to? still says no. method_missing answers calls; it does not make the object admit
it answers them. Everything that introspects before calling, respond_to? included, is looking at
the ancestor chain and finds nothing:
HTML.respond_to?(:div) # => false
HTML.method(:div) # NameError: undefined method 'div' for class '#<Class:HTML>'
The fix is respond_to_missing?, the companion hook, which Ruby consults for exactly these
questions. Pairing the two also lets the generator declare which names it actually handles, so a
typo still raises instead of quietly becoming a tag:
module HTML
TAGS = %i[html head body div p ul li select a span].freeze
def self.method_missing(tag, *args, &block)
return super unless TAGS.include?(tag)
"<#{tag}>#{args.join}</#{tag}>"
end
def self.respond_to_missing?(tag, include_private = false)
TAGS.include?(tag) || super
end
end
HTML.respond_to?(:div) # => true
HTML.method(:div).call("x") # => "<div>x</div>"
HTML.nope("x") # NoMethodError: undefined method 'nope' for module HTML
super is what makes the last line work. Falling through to BasicObject#method_missing is how you
get the standard error back for names you do not handle, instead of inventing a <nope> tag.
Nothing is escaped. Content is interpolated raw, so the generator will happily emit whatever it is given:
HTML.li("<script>alert(1)</script>")
# => "<li><script>alert(1)</script></li>"
For a generator that only ever sees literals in your own source, that is fine. The moment any
content comes from a user, it goes through CGI.escapeHTML first, which is what Rails does for you
in ERB and what this DSL does not do yet.
The BasicObject alternative, and its eight methods
The video names the other way to do this: a class inheriting from BasicObject, with
method_missing as an instance method.
class HTMLGenerator < BasicObject
def method_missing(tag, *args, &block)
"<#{tag}>#{args.join}</#{tag}>"
end
end
g = HTMLGenerator.new
g.html(g.head, g.body(g.ul(g.li("first li"), g.li("second li"))))
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"
It is a better container in one specific way: the chain it has to get past is far shorter, so there is almost nothing left to shadow a tag name.
BasicObject.instance_methods
# => [:!, :equal?, :__send__, :==, :!=, :__id__, :instance_exec, :instance_eval]
BasicObject.instance_methods.size # => 8
Object.instance_methods.size # => 51
Eight methods against fifty one, and p and select are not among the eight. This is the classic
reason BasicObject exists, and it is what gems building DSLs reach for.
The video keeps the module for part 1 anyway, and says why: the next steps want methods that live in
Kernel, and a BasicObject subclass does not have them. Blank slate cuts both ways, so you take
it when the collisions cost more than the conveniences.
What part 2 changes: blocks, instance_eval, and bare p
Part 1's DSL passes everything as arguments, which works and is not how good Ruby DSLs read. Ruby's answer is blocks, and turning
HTML.body(HTML.ul(HTML.li("first li")))
into a block form is what part 2 is about. That is where instance_eval comes in, where the
&block parameter the hook has been ignoring starts being used, and where a bare p inside a block
becomes the problem this article measured in advance.
Related quizzes
Three short ones on the machinery this article walks through:
instance_of?ignores inheritance: the difference between an object's exact class and its ancestor chain.aliasvsalias_methodacross inheritance: what resolves lexically and what resolves onselfat runtime.- Shadowing
Net::HTTPwith your own module: what a constant already holding a class does tomodule Something.
Comments
No comments yet. Be the first.