Programming Ruby 1.9 & 2.0: The Pragmatic Programmers’ Guide (2013)

Part 3. Ruby Crystallized

Chapter 24. Metaprogramming

The Jacquard loom, invented more than 200 years ago, was the first device controlled using punched cards—rows of holes in each card were used to control the pattern woven into the cloth. But imagine if instead of churning out fabric, the loom could punch more cards, and those cards could be fed back into the mechanism. The machine could be used to create new programming that it could then execute. And that would be metaprogramming—writing code that writes code.

Programming is all about building layers of abstractions. As you solve problems, you’re building bridges from the unrelenting and mechanical world of silicon to the more ambiguous and fluid world we inhabit. Some programming languages—such as C—are close to the machine. The distance from C code to the application domain can be large. Other languages—Ruby, perhaps—provide higher-level abstractions and hence let you start coding closer to the target domain. For this reason, most people consider a higher-level language to be a better starting place for application development (although they’ll argue about the choice of language).

But when you metaprogram, you are no longer limited to the set of abstractions built in to your programming language. Instead, you create new abstractions that are integrated into the host language. In effect, you’re creating a new, domain-specific programming language—one that lets you express the concepts you need to solve your particular problem.

Ruby makes metaprogramming easy. As a result, most advanced Ruby programmers will use metaprogramming techniques to simplify their code. This chapter shows how they do it. It isn’t intended to be an exhaustive survey of metaprogramming techniques. Instead, we’ll look at the underlying Ruby principles that make metaprogramming possible. From there you’ll be able to invent your own metaprogramming idioms.

24.1 Objects and Classes

Classes and objects are obviously central to Ruby, but at first sight they can be a little confusing. It seems like there are a lot of concepts: classes, objects, class objects, instance methods, class methods, singleton classes, and virtual classes. In reality, however, Ruby has just a single underlying class and object structure.

A Ruby object has three components: a set of flags, some instance variables, and an associated class. A Ruby class is itself an object of class Class. It contains all the things an object has plus a set of method definitions and a reference to a superclass (which is itself another class).

And, basically, that’s it. From here, you could work out the details of metaprogramming for yourself. But, as always, the devil lurks in the details, so let’s dig a little deeper.

self and Method Calling

Ruby has the concept of the current object . This current object is referenced by the built-in, read-only variable self. self has two significant roles in a running Ruby program.

First, self controls how Ruby finds instance variables. We already said that every object carries around a set of instance variables. When you access an instance variable, Ruby looks for it in the object referenced by self.

Second, self plays a vital role in method calling. In Ruby, each method call is made on some object. This object is called the receiver of the call. When you make a call such as items.size, the object referenced by the variable items is the receiver and size is the method to invoke.

If you make a method call such as puts "hi", there’s no explicit receiver. In this case, Ruby uses the current object, self, as the receiver. It goes to self’s class and looks up the method (in this case, puts ). If it can’t find the method in the class, it looks in the class’s superclass and then in that class’s superclass, stopping when it runs out of superclasses (which will happen after it has looked in BasicObject).[106]

When you make a method call with an explicit receiver (for example, invoking items.size), the process is surprisingly similar. The only change—but it’s a vitally important one—is the fact that self is changed for the duration of the call. Before starting the method lookup process, Ruby sets self to the receiver (the object referenced by items in this case). Then, after the call returns, Ruby restores the value that self had before the call.

Let’s see how this works in practice. Here’s a simple program:

Produces:

The call to Test.new on the second-to-last line creates a new object of class Test, assigning that object to the variable t. Then, on the next line, we call the method t.one . To execute this call, Ruby sets self to t and then looks in t’s class for the method one . Ruby finds the method defined on line 2 and calls it.

Inside the method, we set the instance variable @var to 99. This instance variable will be associated with the current object. What is that object? Well, the call to t.one set self to t, so within the one method, self will be that particular instance of class Test.

On the next line, one calls two . Because there’s no explicit receiver, self is not changed. When Ruby looks for the method two , it looks in Test, the class of t.

The method two references an instance variable @var. Again, Ruby looks for this variable in the current object and finds the same variable that was set by the method one .

The call to puts at the end of two works the same way. Again, because there’s no explicit receiver, self will be unchanged. Ruby looks for the puts method in the class of the current object but can’t find it. It then looks in Test’s superclass, class Object. Again, it doesn’t find puts . However, Object mixes in the module Kernel. We’ll talk more about this later; for now we can say that mixed-in modules act as if they were superclasses. The Kernel module does define puts , so the method is found and executed.

After two and one return, Ruby resets self to the value it had before the original call to t.one .

This explanation may seem labored, but understanding it is vital to mastering metaprogramming in Ruby.

self and Class Definitions

We’ve seen that calling a method with an explicit receiver changes self. Perhaps surprisingly, self is also changed by a class definition. This is a consequence of the fact that class definitions are actually executable code in Ruby—if we can execute code, we need to have a current object. A simple test shows what this object is:

Produces:

Inside a class definition, self is set to the class object of the class being defined. This means that instance variables set in a class definition will be available to class methods (because self will be the same when the variables are defined and when the methods execute):

The fact that self is set to the class during a class definition turns out to be a dramatically elegant decision, but to see why, we’ll first need to have a look at singletons.

24.2 Singletons

Ruby lets you define methods that are specific to a particular object. These are called singleton methods . Let’s start with a simple string object:

Produces:

This results in the following object structure.

The animal variable points to an object containing (among other things) the value of the string ("cat") and a pointer to the object’s class, String.

When we call animal.upcase, Ruby goes to the object referenced by the animal variable and then looks up the method upcase in the class object referenced from the animal object. Our animal is a string and so has the methods of class String available.

Now let’s make it more interesting. We’ll define a singleton method on the string referenced by animal:

Produces:

We’ve already seen how the call to animal.speak works when we looked at how methods are invoked. Ruby sets self to the string object "cat" referenced by animal and then looks for a method speak in that object’s class. Surprisingly, it finds it. It’s initially surprising because the class of "cat" is String, and String doesn’t have a speak method. So, does Ruby have some kind of special-case magic for these methods that are defined on individual objects?

Thankfully, the answer is “no.” Ruby’s object model is remarkably consistent. When we defined the singleton method for the "cat" object, Ruby created a new anonymous class and defined the speak method in that class. This anonymous class is sometimes called a singleton class and other times an eigenclass . I prefer the former, because it ties in to the idea of singleton methods.

Ruby makes this singleton class the class of the "cat" object and makes String (which was the original class of "cat") the superclass of the singleton class. The picture looks like this:

Now let’s follow the call to animal.speak. Ruby goes to the object referenced by animal and then looks in its class for the method speak . The class of the animal object is the newly created singleton class, and it contains the method we need.

What happens if we instead call animal.upcase? The processing starts the same way: Ruby looks for the method upcase in the singleton class but fails to find it there. It then follows the normal processing rules and starts looking up the chain of superclasses. The superclass of the singleton is String, and Ruby finds the upcase method there. Notice that there is no special-case processing here—Ruby method calls always work the same way.

Singletons and Classes

Earlier, we said that inside a class definition, self is set to the class object being defined. It turns out that this is the basis for one of the more elegant aspects of Ruby’s object model.

Recall that we can define class methods in Ruby using either of the forms def self.xxx or def ClassName.xxx:

Produces:

Now we know why the two forms are identical: inside the class definition, self is set to Dave.

But now that we’ve looked at singleton methods, we also know that, in reality, there is no such thing as class methods in Ruby. Both of the previous definitions define singleton methods on the class object. As with all other singleton methods, we can then call them via the object (in this case, the class Dave).

Before we created the two singleton methods in class Dave, the class pointer in the class object pointed to class Class. (That’s a confusing sentence. Another way of saying it is “Dave is a class, so the class of Dave is class Class,” but that’s pretty confusing, too.) The situation looks like this:

The object diagram for class Dave after the methods are defined looks like this:

Do you see how the singleton class is created, just as it was for the animal example? The class is inserted as the class of Dave, and the original class of Dave is made this new class’s parent.

We can now tie together the two uses of self, the current object. We talked about how instance variables are looked up in self, and we talked about how singleton methods defined on self become class methods. Let’s use these facts to access instance variables for class objects:

Produces:

Newcomers to Ruby commonly make the mistake of setting instance variables inline in the class definition (as we did with @var in the previous code) and then attempting to access these variables from instance methods. As the code illustrates, this won’t work, because instance variables defined in the class body are associated with the class object, not with instances of the class.

Another Way to Access the Singleton Class

We’ve seen how you can create methods in an object’s singleton class by adding the object reference to the method definition using something like def animal.speak.

You can do the same using Ruby’s class << an_object notation:

Produces:

Inside this kind of class definition, self is set to the singleton class for the given object (animal in this case). Because class definitions return the value of the last statement executed in the class body, we can use this fact to get the singleton class object:

Produces:

Note the notation that Ruby uses to denote a singleton class: #<Class:#<String:...>>.

Ruby goes to some trouble to stop you from using singleton classes outside the context of their original object. For example, you can’t create a new instance of a singleton class:

Produces:

Let’s tie together what we know about instance variables, self, and singleton classes. We wrote class-level accessor methods to let us get and set the value of an instance variable defined in a class object. But Ruby already has attr_accessor , which defines getter and setter methods. Normally, though, these are defined as instance methods and hence will access values stored in instances of a class. To make them work with class-level instance variables, we have to invoke attr_accessor in the singleton class:

Produces:

24.3 Inheritance and Visibility

There’s a wrinkle to when it comes to method definition and class inheritance, but it’s fairly obscure. Within a class definition, you can change the visibility of a method in an ancestor class. For example, you can do something like this:

In this example, you would be able to invoke a_method in instances of class Derived1 but not via instances of Base or Derived2.

So, how does Ruby pull off this feat of having one method with two different visibilities? Simply put, it cheats.

If a subclass changes the visibility of a method in a parent, Ruby effectively inserts a hidden proxy method in the subclass that invokes the original method using super. It then sets the visibility of that proxy to whatever you requested. This means that the following code:

is effectively the same as this:

The call to super can access the parent’s method regardless of its visibility, so the rewrite allows the subclass to override its parent’s visibility rules. Pretty scary, eh?

24.4 Modules and Mixins

You know that when you include a module into a Ruby class, the instance methods in that module become available as instance methods of the class.

Produces:

Ruby implements include very simply: the module that you include is effectively added as a superclass of the class being defined. It’s as if the module is the parent of the class that it is mixed in to. And that would be the end of the description except for one small wrinkle. Because the module is injected into the chain of superclasses, it must itself hold a link to the original parent class. If it didn’t, there’d be no way of traversing the superclass chain to look up methods. However, you can mix the same module into many different classes, and those classes could potentially have totally different superclass chains. If there were just one module object that we mixed in to all these classes, there’d be no way of keeping track of the different superclasses for each.

To get around this, Ruby uses a clever trick. When you include a module in class Example, Ruby constructs a new class object, makes it the superclass of Example, and then sets the superclass of the new class to be the original superclass of Example. It then references the module’s methods from this new class object in such a way that when you look a method up in this class, it actually looks it up in the module:

A nice side effect of this arrangement is that if you change a module after including it in a class, those changes are reflected in the class (and the class’s objects). In this way, modules behave just like classes.

Produces:

If a module itself includes other modules, a chain of proxy classes will be added to any class that includes that module, one proxy for each module that is directly or indirectly included.

Finally, Ruby will include a module only once in an inheritance chain—including a module that is already included by one of your superclasses has no effect.

prepend

Ruby 2 introduced the prepend method. Logically, this behaves just like include , but the methods in the prepended module take precedence over those in the host class. Ruby pulls off this magic by inserting a dummy class in place of the original host class[107] and then inserting the prepended module between the two.^«2.0»

If a method inside a prepended module has the same name as one in the original class, it will be invoked instead of the original. The prepended method can then call the original using super .

Produces:

However, there is a problem with this—the change we just made to class Object is global. We’ll see how to fix that shortly when we look at refinements.

extend

The include method effectively adds a module as a superclass of self. It is used inside a class definition to make the instance methods in the module available to instances of the class.

However, it is sometimes useful to add the instance methods to a particular object. You do this using Object#extend. Here’s an example:

Stop for a second to think about how this might be implemented.

When Ruby executes obj.tickle in this code example, it does the usual trick of looking in the class of obj for a method called tickle . For extend to work, it has to add the instance methods in the Humor module into the superclass chain for the class of obj. So, just as with singleton method definitions, Ruby creates a singleton class for obj and then includes the module Humor in that class. In fact, just to prove that this is all that happens, here’s the C implementation of extend in the current Ruby 1.9 interpreter:

There is an interesting trick with extend . If you use it within a class definition, the module’s methods become class methods. This is because calling extend is equivalent to self.extend, so the methods are added to self, which in a class definition is the class itself.

Here’s an example of adding a module’s methods at the class level:

Later, we’ll see how to use extend to add macro-style methods to a class.

Refinements

We previously looked at applying a change to a builtin class by defining the altered version of a method in a module, and then prepending that module in the class. When we subsequently call the method on instances of the class, it finds the version in the module first.^«2.0»

This technique is time-honored—frameworks such as Ruby on Rails rely on it. But it comes with a price—any changes we make like this are global. They apply not just to the code we wrote for our application, but also to the code in all the libraries and gems we use, too. It is possible that a change that made our code easier to write breaks someone else’s library code that we rely on.[108]

Ruby 2.0 is experimenting with a new way of dealing with this problem. The technique is called refinements.

Now, before going any further, here is the mandatory warning. Refinements are not yet completely worked out. In fact, a fair amount of refinement functionality was removed just a few weeks before Ruby 2.0 was released. So what we’re documenting here may well become outdated as new releases of Ruby change the spec. If you are writing code that uses refinements, you’ll want to keep track of these changes.

A refinement is a way of packaging a set of changes to one or more classes. These refinements are defined within a module.

If a Ruby source file then elects to use this module of refinements, the change will apply to the source in that module past the point where the refinement is used. However code outside this file is not affected.

Let’s make this concrete. Here’s our vanity version of puts rewritten using refinements.

Produces:

The refinement is contained in the module VanityPuts. The refine block takes a class and a block. Within the block are the methods that you would like to update in that class. At this point, no change is made—you have defined a method, but haven’t yet told Ruby to use it.

That’s what the using clause does. You give it a module containing one or more refinements, and it marks the refined objects to say “for the rest of this source file, when you make a call to an instance of Object, first check to see if the method can be found in the refinement. If so, invoke it, otherwise invoke the original.

Let’s step it up a notch. We’ll define three source files. Here’s one that contains a refinement definition:

And here’s a file that uses this refinement.

Finally, let’s run them from a third file:

Produces:

Notice how the puts calls in the main program are unadorned, but the calls in the file that uses the refinement has the vanity prefix.

Refinements—use and scoping

You can define refinements in any module. A refinement may only refer to a class (not a module).

The using call that activates a refinement module can only occur at the top-level scope or in a string that is evaluated. using may not be called inside a class or module definition.[109]

The basic scoping rule is simple. A refinement is activated in a source file by calling using . For the rest of that source file, the methods that are defined in that refinement are active.

Designing with Refinements

Refinements are too new for the community to have come to any kind of consensus on the best way to use them. However, it seems likey that there are at least two basic use cases.

The first is the case where a developer wants to make changes to a third party class for their own use. For example, a tool such as Rake, which issues a lot of calls to run external programs using the system method might want to modify the built-in version of system so that it logs errors differently. However, it does not want that logging to apply to other calls to system that are not part of Rake. In this case, the code will use the refinement locally within its own source files. It will be an implementation detail, hidden from users of the code.

The second use case is where a library writer offers the refinement as part of their external interface. For example, the Rails Active Support code defines methods such as hours, minutes, and ago on numbers, allowing you to write 3.days.ago. Right now, those changes to numbers are global. But, using refinements, the Rails team could code the new methods, but not add them in to any system classes. Instead, their API would document how to add them for yourself into just those source files that uses them. They might tell you to to write

in any source file that needs to use them.

Obviously there are many more use cases. And the two we’ve mentioned are not mutually exclusive—the Rails framework, for example, is likely to want to use these duration-related methods itself, as well as making them available via a documented refinement. So this is an interesting time to work with Ruby—as a community we’ll be discussing how best to use refinements for many years.

24.5 Metaprogramming Class-Level Macros

If you’ve used Ruby for any time at all, you’re likely to have used attr_accessor , the method that defines reader and writer methods for instance variables:

If you’ve written a Ruby on Rails application, you’ve probably used has_many :

These are both examples of class-level methods that generate code behind the scenes. Because of the way they expand into something bigger, folks sometimes call these kinds of methods macros .

Let’s create a trivial example and then build it up into something realistic. We’ll start by implementing a simple method that adds logging capabilities to instances of a class. We previously did this using a module—this time we’ll do it using a class-level method. Here’s the first iteration:

Produces:

Clearly, this is a silly piece of code. But bear with me—it’ll get better. And we can still learn some stuff from it. First, notice that add_logging is a class method—it is defined in the class object’s singleton class. That means we can call it later in the class definition without an explicit receiver, because self is set to the class object inside a class definition.

Then, notice that the add_logging method contains a nested method definition. This inner definition will get executed only when we call the add_logging method. The result is that log will be defined as an instance method of class Example.

Let’s take one more step. We can define the add_logging method in one class and then use it in a subclass. This works because the singleton class hierarchy parallels the regular class hierarchy. As a result, class methods in a parent class are also available in the child class, as the following example shows.

Produces:

Think back to the two examples at the start of this section. Both work this way. attr_accessor is an instance method defined in class Module and so is available in all module and class definitions. has_many is a class method defined in the Base class within the Rails ActiveRecord module and so is available to all classes that subclass ActiveRecord::Base.

This example is still not particularly compelling; it would still be easier to add the log method directly as an instance method of our Logger class. But what happens if we want to construct a different version of the log method for each class that uses it? For example, let’s add the capability to add a short class-specific identifying string to the start of each log message. We want to be able to say something like this:

To do this, let’s define the log method on the fly. We can no longer use a straightforward def...end-style definition. Instead, we’ll use one of the cornerstones of metaprogramming, define_method . This takes the name of a method and a block, defining a method with the given name and with the block as the method body. Any arguments in the block definition become parameters to the method being defined.

Produces:

There’s an important subtlety in this code. The body of the log method contains this line:

The value now is a local variable, and msg is the parameter to the block. But id_string is the parameter to the enclosing add_logging method. It’s accessible inside the block because block definitions create closures, allowing the context in which the block is defined to be carried forward and used when the block is used. In this case, we’re taking a value from a class-level method and using it in an instance method we’re defining. This is a common pattern when creating these kinds of class-level macros.

As well as passing parameters from the class method into the body of the method being defined, we can also use the parameter to determine the name of the method or methods to create. Here’s an example that creates a new kind of attr_accessor that logs all assignments to a given instance variable:

Produces:

Again, we use the fact that the block defining the method body is a closure, accessing the name of the attribute in the log message string. Notice we also make use of the fact that attr_reader is simply a class method—we can call it inside our class method to define the reader method for our attribute. Note another common bit of metaprogramming—we use instance_variable_set to set the value of an instance variable (duh). There’s a corresponding _get method that fetches the value of a named instance variable.

Class Methods and Modules

You can define class methods in one class and then use them in subclasses of that class. But it is often inappropriate to use subclassing, either because we already have to subclass some other class or because our design aesthetic rebels against making Song a subclass of Logger.

In these cases, you can use a module to hold your metaprogramming implementation. As we’ve seen, using extend inside a class definition will add the methods in a module as class methods to the class being defined:

Produces:

Things get a little trickier if you want to add both class methods and instance methods into the class being defined. Here’s one technique, used extensively in the implementation of Rails. It makes use of a Ruby hook method, included , which is called automatically by Ruby when you include a module into a class. It is passed the class object of the class being defined.

Produces:

Notice how the included callback is used to extend the host class with the methods defined in the inner module ClassMethods.

Now, as an exercise, try executing the previous example in your head. For each line of code, work out the value of self. Master this, and you’ve pretty much mastered this style of metaprogramming in Ruby.

24.6 Two Other Forms of Class Definition

Just in case you thought we’d exhausted the ways of defining Ruby classes, let’s look at two other options.

Subclassing Expressions

The first form is really nothing new—it’s simply a generalization of the regular class definition syntax. You know that you can write this:

What you might not know is that the thing to the right of the < needn’t be just a class name; it can be any expression that returns a class object. In this code example, we have the constant Parent. A constant is a simple form of expression, and in this case the constant Parent holds the class object of the first class we defined.

Ruby comes with a class called Struct, which allows you to define classes that contain just data attributes. For example, you could write this:

Produces:

The return value from Struct.new(...) is a class object. By assigning it to the constant Person, we can thereafter use Person as if it were any other class.

But say we wanted to change the to_s method of our structure.

We could do it by opening up the class and writing the following method.

However, we can do this more elegantly (although at the cost of an additional class object) by writing this:

Produces:

Creating Singleton Classes

Let’s look at some Ruby code:

When we call Example.new, we’re invoking the method new on the class object Example. This is a regular method call—Ruby looks for the method new in the class of the object (and the class of Example is Class) and invokes it. So we can also invoke Class#new directly:

Produces:

If you pass Class.new a block, that block is used as the body of the class:

Produces:

By default, these classes will be direct descendents of Object. You can give them a different parent by passing the parent’s class as a parameter:

Produces:

How Classes Get Their Names

You may have noticed that the classes created by Class.new have no name. However, if you assign the class object for a class with no name to a constant, Ruby automatically names the class after the constant:

Produces:

We can use these dynamically constructed classes to extend Ruby in interesting ways. For example, here’s a simple reimplementation of the Ruby Struct class:

Produces:

24.7 instance_eval and class_eval

The methods Object#instance_eval, Module#class_eval, and Module#module_eval let you set self to be some arbitrary object, evaluate the code in a block with, and then reset self:

Produces:

Both forms also take a string, but this is considered a little dangerous.

First, it is slow—calling eval effectively compiles the code in the string before executing it. But, even worse, eval can be dangerous. If there’s any chance that external data—stuff that comes from outside your application—can wind up inside the parameter to eval , then you have a security hole, because that external data may end up containing arbitrary code that your application will blindly execute.

That said, here’s an example using a string parameter:

Produces:

class_eval and instance_eval both set self for the duration of the block. However, they differ in the way they set up the environment for method definition. class_eval sets things up as if you were in the body of a class definition, so method definitions will define instance methods:

Produces:

In contrast, calling instance_eval on a class acts as if you were working inside the singleton class of self. Therefore, any methods you define will become class methods.

Produces:

It might be helpful to remember that, when defining methods, class_eval and instance_eval have precisely the wrong names: class_eval defines instance methods, and instance_eval defines class methods. Go figure.

Ruby has variants of these methods. Object#instance_exec, Module#class_exec, and Module#module_exec behave identically to their _eval counterparts but take only a block (that is, they do not take a string). Any arguments given to the methods are passed in as block parameters. This is an important feature. Previously it was impossible to pass an instance variable into a block given to one of the _eval methods—because self is changed by the call, these variables go out of scope. With the _exec form, you can now pass them in:

Produces:

instance_eval and Constants

Ruby 1.9 changed the way Ruby looks up constants when executing a block using instance_eval and class_eval . Ruby 1.9.2 then reverted the change. In Ruby 1.8 and Ruby 1.9.2, constants are looked up in the lexical scope in which they were referenced. In Ruby 1.9.0, they are looked up in the scope in which instance_eval is called. This (artificial) example shows the behavior at the time I last built this book—it may well have changed again by the time you run it....

In Ruby 1.9.0, this same code would evaluate to "Defined in One".

instance_eval and Domain-Specific Languages

It turns out that instance_eval has a pivotal role to play in a certain type of domain-specific language (DSL). For example, we might be writing a simple DSL for turtle graphics.[110] To draw a set of three 5x5 squares, we might write the following:[111]

Clearly, pen_down , forward , left , and pen_up can be implemented as Ruby methods. However, to call them without a receiver like this, either we have to be within a class that defines them (or is a child of such a class) or we have to make the methods global. instance_eval to the rescue. We can define a class Turtle that defines the various methods we need as instance methods. We’ll also define a walk method, which will execute our turtle DSL, and a draw method to draw the resulting picture:

If we implement walk correctly, we can then write this:

So, what is the correct implementation of walk ? Well, we clearly have to use instance_eval , because we want the DSL commands in the block to call the methods in the turtle object. We also have to arrange to pass the block given to the walk method to be evaluated by that instance_eval call. Our implementation looks like this:

Notice how we captured the block into a variable and then expanded that variable back into a block in the call to instance_eval .

See Section 24.11, The Turtle Graphics Program for a complete listing of the turtle program.

Is this a good use of instance_eval ? It depends on the circumstances. The benefit is that the code inside the block looks simple—you don’t have to make the receiver explicit:

There’s a drawback, though. Inside the block, scope isn’t what you think it is, so this code wouldn’t work:

Instance variables are looked up in self, and self in the block isn’t the same as self in the code that sets the instance variable @size. Because of this, most people are moving away from this style of instance_evaled block.

24.8 Hook Methods

In Class Methods and Modules, we defined a method called included in our GeneralLogger module. When this module was included in a class, Ruby automatically invoked this included method, allowing our module to add class methods to the host class.

included is an example of a hook method (sometimes called a callback ). A hook method is a method that you write but that Ruby calls from within the interpreter when some particular event occurs. The interpreter looks for these methods by name—if you define a method in the right context with an appropriate name, Ruby will call it when the corresponding event happens.

The methods that can be invoked from within the interpreter are:

Method-related hooks

method_added , method_missing , method_removed , method_undefined , singleton_method_added , singleton_method_removed , singleton_method_undefined

Class and module-related hooks

append_features , const_missing , extend_object , extended , included , inherited , initialize_clone , initialize_copy , initialize_dup

Object marshaling hooks

marshal_dump , marshal_load

Coercion hooks

coerce , induced_from , to_xxx

We won’t discuss all of them in this chapter—instead, we’ll show just a few examples of use. The reference section of this book describes the individual methods, and Chapter 23, Duck Typing discusses the coercion methods in more detail.

The inherited Hook

If a class defines a class method called inherited , Ruby will call it whenever that class is subclassed (that is, whenever any class inherits from the original).

This hook is often used in situations where a base class needs to keep track of its children. For example, an online store might offer a variety of shipping options. Each might be represented by a separate class, and each of these classes could be a subclass of a single Shipping class. This parent class could keep track of all the various shipping options by recording every class that subclasses it. When it comes time to display the shipping options to the user, the application could call the base class, asking it for a list of its children:

Produces:

Command interpreters often use this pattern: the base class keeps a track of available commands, each of which is implemented in a subclass.

The method_missing Hook

Earlier, we saw how Ruby executes a method call by looking for the method, first in the object’s class, then in its superclass, then in that class’s superclass, and so on. If the method call has an explicit receiver, then private methods are skipped in this search. If the method is not found by the time we run out of superclasses (because BasicObject has no superclass), then Ruby tries to invoke the hook method method_missing on the original object. Again, the same process is followed—Ruby first looks in the object’s class, then in its superclass, and so on. However, Ruby predefines its own version of method_missing in class BasicObject, so typically the search stops there. The built-in method_missing basically raises an exception (either a NoMethodError or a NameError depending on the circumstances).

The key here is that method_missing is simply a Ruby method. We can override it in our own classes to handle calls to otherwise undefined methods in an application-specific way.

method_missing has a simple signature, but many people get it wrong:

The name argument receives the name of the method that couldn’t be found. It is passed as a symbol. The args argument is an array of the arguments that were passed in the original call. And the oft-forgotten block argument will receive any block passed to the original method.

Produces:

Before we get too deep into the details, I’ll offer a tip about etiquette. There are two main ways that people use method_missing . The first intercepts every use of an undefined method and handles it. The second is more subtle; it intercepts all calls but handles only some of them. In the latter case, it is important to forward on the call to a superclass if you decide not to handle it in your method_missing implementation:

If you fail to pass on calls that you don’t handle, your application will silently ignore calls to unknown methods in your class.

Let’s show a couple of uses of method_missing .

method_missing to Simulate Accessors

The OpenStruct class is distributed with Ruby. It allows you to write objects with attributes that are created dynamically by assignment. (We describe it in more detail in the library documentation.) For example, you could write this:

Produces:

Let’s use method_missing to write our own version of OpenStruct:

Produces:

Notice how we base our class on BasicObject, a class introduced in Ruby 1.9. BasicObject is the root of Ruby’s object hierarchy and contains only a minimal number of methods:

Produces:

This is good, because it means that our MyOpenStruct class will be able to have attributes such as display or class . If instead we’d based MyOpenStruct on class Object, then these names, along with forty-nine others, would have been predefined and hence wouldn’t trigger method_missing .

Notice also another common pattern inside method_missing . The first time we reference or assign to an attribute of our object, we access or update the @values hash appropriately. But we also define the method that the caller was trying to access. This means that the next time this attribute is used, it will use the method and not invoke method_missing . This may or may not be worth the trouble, depending on the access patterns to your object.

Also notice how we had to jump through some hoops to define the method. We want to define the method only for the current object. This means we have to put the method into the object’s singleton class. We can do that using instance_exec and define_method . But that means we have to use the class << self trick to get the object’s singleton class. Through an interesting implementation subtlety, define_method will always define an instance method, independent of whether it is invoked via instance_exec or class_exec .

However, this code reveals a dark underbelly of using method_missing and BasicObject:

Produces:

The dup method is not defined by BasicObject; it appears in class Object. So when we called dup , it was picked up by our method_missing handler, and we just returned nil (because we don’t have yet have an attribute called dup). We could fix this so that it at least reports an error:

This class now reports an error if we call dup (or any other method) on it. However, we still can’t dup or clone it (or inspect, convert to a string, and so on). Although BasicObject seems like a natural fit for method_missing , you may find it to be more trouble than it’s worth.

method_missing as a Filter

As the previous example showed, method_missing has some drawbacks if you use it to intercept all calls. It is probably better to use it to recognize certain patterns of call, passing on those it doesn’t recognize to its parent class to handle.

An example of this is the dynamic finder facility in the Ruby on Rails ActiveRecord module. ActiveRecord is the object-relational library in Rails—it allows you to access relational databases as if they were object stores. One particular feature allows you to find rows that match the criteria of having given values in certain columns. For example, if an Active Record class called Book were mapping a relational table called books and the books table included columns called title and author, you could write this:

Active Record does not predefine all these potential finder methods. Instead, it uses our old friend method_missing . Inside that method, it looks for calls to undefined methods that match the pattern /^find_(all_)?by_(.*)/.[112] If the method being invoked does not match this pattern or if the fields in the method name don’t correspond to columns in the database table, Active Record calls super so that a genuine method_missing report will be generated.

24.9 One Last Example

Let’s bring together all the metaprogramming topics we’ve discussed in a final example by writing a module that allows us to trace the execution of methods in any class that mixes the module in. This would let us write the following:

Produces:

We can see immediately that there’s a subtlety here. When we mix the TraceCalls module into a class, it has to add tracing to any existing instance methods in that class. It also has to arrange to add tracing to any methods we subsequently add.

Let’s start with the full listing of the TraceCalls module:

When we include this module in a class, the included hook method gets invoked. It first uses the instance_methods reflection method to find all the existing instance methods in the host class (the false parameter limits the list to methods in the class itself, and not in its superclasses). For each existing method, the module calls a helper method, wrap , to add some tracing code to it. We’ll talk about wrap shortly.

Next, the included method uses another hook, method_added . This is called by Ruby whenever a method is defined in the receiver. Note that we define this method in the class passed to the included method. This means that the method will be called when methods are added to this host class and not to the module. This is what allows us to include TraceCalls at the top of a class and then add methods to that class—all those method definitions will be handled by method_added .

Now look at the code inside the method_added method. We have to deal with a potential problem here. As you’ll see when we look at the wrap method, we add tracing to a method by creating a new version of the method that calls the old. Inside method_added , we call the wrap function to add this tracing. But inside wrap , we’ll define a new method to handle this wrapping, and that definition will invoke method_added again, and then we’d call wrap again, and so on, until the stack gets exhausted. To prevent this, we use an instance variable and do the wrapping only if we’re not already doing it.

The wrap method takes a class object and the name of a method to wrap. It finds the original definition of that method (using instance_method ) and saves it. It then redefines this method. This new method outputs some tracing and then calls the original, passing in the parameters and block from the wrapper.[113] Note how we call the method by binding the method object to the current instance and then invoking that bound method.

The key to understanding this code, and most metaprogramming code, is to follow the basic principles we worked out at the start of this chapter—how self changes as methods are called and classes are defined and how methods are called by looking for them in the class of the receiver. If you get stuck, do what we do and draw little boxes and arrows. We find it useful to stick with the convention used in this chapter: class links go to the right, and superclass links go up. Given an object, a method call is then a question of finding the receiver object, going right once, and then following the superclass chain up as far as you need to go.

24.10 Top-Level Execution Environment

Finally, there’s one small detail we have to cover to complete the metaprogramming environment. Many times in this book we’ve claimed that everything in Ruby is an object. However, we’ve used one thing time and time again that appears to contradict this—the top-level Ruby execution environment:

Not an object in sight. We may as well be writing some variant of Fortran or Basic. But dig deeper, and you’ll come across objects and classes lurking in even the simplest code.

We know that the literal "Hello, World" generates a Ruby String, so that’s one object. We also know that the bare method call to puts is effectively the same as self.puts. But what is self?

At the top level, we’re executing code in the context of some predefined object. When we define methods, we’re actually creating (private) instance methods for class Object. This is fairly subtle; as they are in class Object, these methods are available everywhere. And because we’re in the context of Object, we can use all of Object’s methods (including those mixed in from Kernel) in function form. This explains why we can call Kernel methods such as puts at the top level (and indeed throughout Ruby); it’s because these methods are part of every object. Top-level instance variables also belong to this top-level object.

Metaprogramming is one of Ruby’s sharpest tools. Don’t be afraid to use it to raise up the level at which you program. But, at the same time, use it only when necessary—overly metaprogrammed applications can become pretty obscure pretty quickly.

24.11 The Turtle Graphics Program

Produces:

Footnotes