LaunchKit
← All posts
Video · 12 min read · by Mehdi Farsi · 18 views

An HTML generator in one hook, and the Ruby method lookup path

Watch on YouTube

Text version

The goal of this mini project is the best DSL we can get for generating HTML. Here is the result we are aiming for:

HTML.html(
  HTML.head,
  HTML.body(
    HTML.ul(
      HTML.li("first li"),
      HTML.li("second li")
    )
  )
)
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"

By the end of this part, the entire generator is one method, and not one of html, head, body, ul or li is defined anywhere. Getting there means walking the whole Ruby object model, because the trick that writes those methods for us is a hook at the very end of the method lookup path.

The draft, one method per tag

Take the approach the video calls the DHH one: the first version of a piece of code is always a draft, so start modestly and get a decent result.

module HTML
  def self.html(*content) = "<html>#{content.join}</html>"
  def self.head(*content) = "<head>#{content.join}</head>"
  def self.body(*content) = "<body>#{content.join}</body>"
  def self.ul(*content)   = "<ul>#{content.join}</ul>"
  def self.li(*content)   = "<li>#{content.join}</li>"
end

HTML.html(HTML.head, HTML.body(HTML.ul(HTML.li("first li"), HTML.li("second li"))))
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"

That is the target output, on the target DSL, in seven lines. It also has one obvious problem: the HTML standard defines well over a hundred elements, and every one of those methods is the same method with a different name in it. Nobody is writing that by hand.

So the real question of part 1 is not "how do I generate HTML". It is "how do I get Ruby to define those hundred methods for me".

Why a module and not a class

The first decision is what container the generator lives in. It is a module, and the reason is worth being precise about.

A module in Ruby is a data structure that cannot be instantiated. It is a container that holds methods, constants, other modules and classes. Modules hold behavior only. Classes hold behavior and state.

The three usual reasons to reach for one:

  • As a mixin, mixed into another module or class with include, extend or prepend. Each keyword puts the module at a different place in the ancestor chain.
  • As a namespace, to prevent name clashing and to make the code read better.
  • As a standalone entity, self-sufficient, holding module functions and nothing else. Math is the one everybody has used.

The generator is the third kind. It produces HTML on demand: there is no state to carry from one call to the next, no object whose life cycle matters, nothing to instantiate. A module of module functions is the exact shape of the problem.

That is also why the hook is defined on self. Every call in the DSL is HTML.something, a method call on the module object itself, not on an instance of anything.

Classes are objects too

To understand the hook we need the ancestor chain, and to understand the ancestor chain we need the object model. In Ruby, classes are first class objects: they can appear in an expression, be assigned to a variable, and be passed as an argument.

class OtherClass; end

OtherClass.class        # => Class
OtherClass.superclass   # => Object

So OtherClass is an instance of the Class class, stored in a constant. What you call "a class" is an object like any other, and the class OtherClass; end syntax is sugar for creating one and assigning it to a constant.

Class itself has a parent:

Class.superclass    # => Module
Module.superclass   # => Object
Object.superclass   # => BasicObject

A class is a specialization of a module. Everything a module can do, a class can do, plus one thing: it can be instantiated.

OtherClass.new        # => #<OtherClass>

OtherModule = Module.new
OtherModule.new       # NoMethodError: undefined method 'new' for an instance of Module

That is the whole difference between the two, and it is why the choice in the previous section was a real choice rather than a style preference.

Three names sit at the root of all of this:

  • BasicObject is the top parent of every class. It holds the bare minimum: object creation and object comparison.
  • Kernel is a module included in Object. It holds the bulk of the object manipulation logic, and it is where puts, p, send and format actually live.
  • Object inherits from BasicObject and includes Kernel. Because Kernel carries most of the methods, Object is used mostly as the interface its name gives to all of its children.
Object.superclass          # => BasicObject
Object.include?(Kernel)    # => true
method(:puts).owner        # => Kernel

That last line is the one to keep. puts is not defined on Object. It is defined on Kernel, and you can call it anywhere because Object includes Kernel and everything is an Object.

The main object

When a program starts, Ruby creates the main object, an instance of Object. It is the top level context of any program, most likely a nod to C, where main is the entry point of every program.

self          # => main
self.class    # => Object

So a bare puts "hi" at the top of a file is a method call on main, resolved through Object to Kernel. Nothing is special about the top level; it is just an object you did not have to create.

A method call is a message

What you call a method call is, behind the scenes, a message. A message has a name, usually a symbol, and an optional payload, the argument list. It needs a receiver: an object that responds to that message through a message handler. The sender is always the calling context, which is the main object when you are outside any class.

You can send one explicitly:

"a b c".send(:split, " ")   # => ["a", "b", "c"]

Receiver: the string. Message name: :split. Payload: " ". The dot syntax you write every day is the same thing with nicer punctuation:

"a b c".split(" ")   # => ["a", "b", "c"]

This matters here because the hook we are about to define receives exactly those pieces: the message name, the payload, and the block.

The ancestor chain

The ancestor chain is the class hierarchy of an object, in order. It contains the class itself, the modules it includes, its parent class, that parent's included modules, and so on up.

Array.ancestors        # => [Array, Enumerable, Object, Kernel, BasicObject]

class OtherClass; end
OtherClass.ancestors          # => [OtherClass, Object, Kernel, BasicObject]
OtherClass.included_modules   # => [Kernel]

Array includes Enumerable, inherits from Object, which includes Kernel and inherits from BasicObject. Reading that array top to bottom is reading the order in which Ruby will look for a method.

The method lookup path

When you call a method, Ruby first checks whether it exists in the context of self. If it does not, it walks up the ancestor chain until it finds one.

class OtherClass; end

a = OtherClass.new
b = OtherClass.new

a.equal?(b)   # => false

equal? is defined on BasicObject. The lookup tries OtherClass, then Object, then Kernel, then BasicObject, where it finally finds it. That mechanism is the method lookup path.

So what happens when the chain runs out? The responsibility is handed to the method_missing hook, which is itself looked up the same way, along the same chain. BasicObject ships an implementation at the end of it, and that implementation is the one that raises:

module Plain; end

Plain.nope   # NoMethodError: undefined method 'nope' for module Plain

The familiar NoMethodError is not the absence of a method. It is the return value of the last method_missing on the chain. Override it earlier on that chain and the error never happens.

The whole generator

module HTML
  def self.method_missing(tag, *args, &block)
    "<#{tag}>#{args.join}</#{tag}>"
  end
end

HTML.html(HTML.head, HTML.body(HTML.ul(HTML.li("first li"), HTML.li("second li"))))
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"

That is it. Every tag in the DSL, from one method.

method_missing is defined on self because the DSL calls module methods. The first parameter is the method id, a symbol naming the call Ruby could not resolve: :html, :head, :body. It is already the tag name, which is why no mapping table is needed. Then the argument list, and a block that part 1 ignores.

args.join is what makes nesting work, and it is worth being explicit about why. Ruby evaluates arguments before it evaluates the call, so the innermost tags are generated first and the outer call receives strings. Adding a trace line to the hook shows the order:

method_missing(:head, [])
method_missing(:li, ["first li"])
method_missing(:li, ["second li"])
method_missing(:ul, ["<li>first li</li>", "<li>second li</li>"])
method_missing(:body, ["<ul><li>first li</li><li>second li</li></ul>"])
method_missing(:html, ["<head></head>", "<body><ul>...</ul></body>"])

Six calls, none of which resolve to a method, all of which return a string that the next call up treats as content. HTML.head with no arguments joins an empty array, so it produces <head></head>.

The two tag names Kernel already owns

A hook at the end of the lookup path only fires when the lookup fails. Anything already defined earlier on the chain wins, silently, and HTML is an instance of Module, whose chain is not empty:

Module.ancestors   # => [Module, Object, Kernel, BasicObject]

Intersect the HTML element names with everything reachable on that module object and exactly two come back:

(HTML.methods + HTML.private_methods) & %i[html head body div p ul li select a span]
# => [:p, :select]

Kernel defines both, and this is where it gets interesting: they are private.

Kernel.private_instance_methods & %i[p select]   # => [:p, :select]
Kernel.public_instance_methods  & %i[p select]   # => []

A private method called with an explicit receiver is not callable, and Ruby routes that failure through method_missing too. So the hook still fires, and both tags work:

HTML.p("x")               # => "<p>x</p>"
HTML.select(HTML.a("x"))  # => "<select><a>x</a></select>"

Without the hook, the same call reports the real reason:

module Plain; end
Plain.p("x")   # NoMethodError: private method 'p' called for module Plain

The place it does bite is inside the module, where there is no explicit receiver. A bare p there is Kernel#p, it prints to stdout and returns its argument, and no hook is involved:

module Bare
  def self.render = p("hello")
end

Bare.render   # prints hello, returns "hello"

Part 2 moves the DSL into blocks, where calls are written without a receiver. This is the detail to remember when it does.

What method_missing alone does not do

respond_to? still says no. method_missing answers calls; it does not make the object admit it answers them. Everything that introspects before calling, respond_to? included, is looking at the ancestor chain and finds nothing:

HTML.respond_to?(:div)   # => false
HTML.method(:div)        # NameError: undefined method 'div' for class '#<Class:HTML>'

The fix is respond_to_missing?, the companion hook, which Ruby consults for exactly these questions. Pairing the two also lets the generator declare which names it actually handles, so a typo still raises instead of quietly becoming a tag:

module HTML
  TAGS = %i[html head body div p ul li select a span].freeze

  def self.method_missing(tag, *args, &block)
    return super unless TAGS.include?(tag)

    "<#{tag}>#{args.join}</#{tag}>"
  end

  def self.respond_to_missing?(tag, include_private = false)
    TAGS.include?(tag) || super
  end
end

HTML.respond_to?(:div)     # => true
HTML.method(:div).call("x") # => "<div>x</div>"
HTML.nope("x")             # NoMethodError: undefined method 'nope' for module HTML

super is what makes the last line work. Falling through to BasicObject#method_missing is how you get the standard error back for names you do not handle, instead of inventing a <nope> tag.

Nothing is escaped. Content is interpolated raw, so the generator will happily emit whatever it is given:

HTML.li("<script>alert(1)</script>")
# => "<li><script>alert(1)</script></li>"

For a generator that only ever sees literals in your own source, that is fine. The moment any content comes from a user, it goes through CGI.escapeHTML first, which is what Rails does for you in ERB and what this DSL does not do yet.

The BasicObject alternative, and its eight methods

The video names the other way to do this: a class inheriting from BasicObject, with method_missing as an instance method.

class HTMLGenerator < BasicObject
  def method_missing(tag, *args, &block)
    "<#{tag}>#{args.join}</#{tag}>"
  end
end

g = HTMLGenerator.new
g.html(g.head, g.body(g.ul(g.li("first li"), g.li("second li"))))
# => "<html><head></head><body><ul><li>first li</li><li>second li</li></ul></body></html>"

It is a better container in one specific way: the chain it has to get past is far shorter, so there is almost nothing left to shadow a tag name.

BasicObject.instance_methods
# => [:!, :equal?, :__send__, :==, :!=, :__id__, :instance_exec, :instance_eval]

BasicObject.instance_methods.size   # => 8
Object.instance_methods.size        # => 51

Eight methods against fifty one, and p and select are not among the eight. This is the classic reason BasicObject exists, and it is what gems building DSLs reach for.

The video keeps the module for part 1 anyway, and says why: the next steps want methods that live in Kernel, and a BasicObject subclass does not have them. Blank slate cuts both ways, so you take it when the collisions cost more than the conveniences.

What part 2 changes: blocks, instance_eval, and bare p

Part 1's DSL passes everything as arguments, which works and is not how good Ruby DSLs read. Ruby's answer is blocks, and turning

HTML.body(HTML.ul(HTML.li("first li")))

into a block form is what part 2 is about. That is where instance_eval comes in, where the &block parameter the hook has been ignoring starts being used, and where a bare p inside a block becomes the problem this article measured in advance.

Three short ones on the machinery this article walks through:

#ruby #video #metaprogramming

Comments

No comments yet. Be the first.

Only used to confirm and publish your comment. Never shown publicly, never shared.

Markdown: **bold**, `code`, ```fenced blocks```, > quotes, [links](url). HTML and images are not rendered.