Metaprogramming Ruby 2: Program Like the Ruby Pros (2014)

Part 3. Appendixes

Appendix 2. Domain-Specific Languages

Domain-specific languages are a popular topic these days. They overlap somewhat with metaprogramming, so you’ll probably want to know a thing or two about them.

The Case for Domain-Specific Languages

Are you old enough to remember Zork? It was one of the first “text adventures”: text-based computer games that were popular in the early 1980s. Here are the first few lines from a game of Zork:

<=	West of house
	You are standing in an open field west of a
	white house, with a boarded front door.
	You see a small mailbox here.
=>	open mailbox
<=	Opening the small mailbox reveals a leaflet.
=>	take leaflet
<=	Taken.

Suppose you have to write a text adventure as your next job. What language would you write it in?

You’d probably pick a language that’s good at manipulating strings and supports object-oriented programming. But whatever language you chose, you’d still have a gap between that language and the problem you’re trying to solve. This probably happens in your daily programming job as well. For example, many large Java applications deal with money, but Money is not a standard Java type. That means each application has to reinvent money, usually as a class.

In the case of our adventure game, you have to deal with entities such as rooms and items. No general-purpose language supports these entities directly. How would you like a language that’s specifically targeted to text adventures? Given such a language, you could write code like this:

	me: Actor
	location = westOfHouse
	;

	westOfHouse : Room 'West of house'
	"You are standing in an open field west of
	a white house, with a boarded front door."
	;

	+ mailbox : OpenableContainer 'mailbox' 'small mailbox';

	++ leaflet : Thing 'leaflet' 'leaflet';

This is not a mocked-up example—it’s real code. It’s written in a language called TADS, specifically designed for creating “interactive fiction” (today’s fancier name for text adventures). TADS is an example of a domain-specific language (DSL), a language that focuses on a specific problem domain.

The opposite of a DSL is a general-purpose language (GPL), such as C++ or Ruby. You can use a GPL to tackle a wide variety of problems, even if it might be more suited to some problems than others. Whenever you write a program, it’s up to you to choose between a flexible GPL and a focused DSL.

Let’s assume that you decide to go down the DSL route. How would you proceed then?

Using DSLs

If you want a DSL for your own specific problem, you might get lucky. There are hundreds of DSLs around, focusing on a wide range of domains. The UNIX shell is a DSL for gluing command-line utilities together. Microsoft’s VBA was designed to extend Excel and other Microsoft Office applications. The make language is a DSL focused on building C programs, and Ant is an XML-based equivalent of make for Java programs. Some of these languages are limited in scope, while others are flexible enough to cross the line into GPL-dom.

What if you can’t find a ready-made DSL that fits the domain you’re working in? In that case, you can write your own DSL and then use that DSL to write your program. You could say that this process—writing a DSL and then using it—is another take on metaprogramming. It can be a slippery path, though. You’ll probably need to define a grammar for your language with a system such as ANTLR or Yacc, which are themselves DSLs for writing language parsers. As the scope of your problem expands, your humble little language can grow into a GPL before you even realize it. At that point, your leisurely foray into language writing will have escalated into an exhausting marathon.

To avoid these difficulties, you can pick a different route. Rather than writing a full-fledged DSL, you can bend a GPL into something resembling a DSL for your specific problem. The next section shows you how.

Internal and External DSLs

Let’s see an example of a DSL that’s actually a GPL in disguise. Here’s a snippet of Ruby code that uses the Markaby gem to generate HTML:

dsl/markaby_example.rb
	require 'markaby'

	html = Markaby::Builder.new do
	head { title "My wonderful home page" }
	body do
	h1 "Welcome to my home page!"
	b "My hobbies:"
	ul do
	li "Juggling"
	li "Knitting"
	li "Metaprogramming"
	end
	end
	end

This code is plain old Ruby, but it looks like a specific language for HTML generation. You can call Markaby an internal DSL, because it lives within a larger, general-purpose language. By contrast, languages that have their own parser, such as make, are often called external DSLs. One example of an external DSL is the Ant build language. Even though the Ant interpreter is written in Java, the Ant language is completely different from Java.

Let’s leave the GPL vs. DSL match behind us and assume that you want to use a DSL. Which DSL should you prefer? An internal DSL or an external DSL?

One advantage of an internal DSL is that you can easily fall back on the underlying GPL whenever you need to do so. However, the syntax of your internal DSL will be constrained by the syntax of the GPL behind it. This is a big problem with some languages. For example, you can write an internal DSL in Java, but the result is probably still going to look pretty much like Java. But with Ruby, you can write an internal DSL that looks more like an ad hoc language tailored to the problem at hand. Thanks to Ruby’s flexible, uncluttered syntax, the Markaby example shown earlier barely looks like Ruby at all.

That’s why Ruby programmers tend to use Ruby where Java programmers would use an external language or an XML file. It’s easier to adapt Ruby to your own needs than it is to adapt Java. As an example, consider build languages. The standard build languages for Java and C (Ant and make, respectively) are external DSLs, while the standard build language for Ruby (Rake) is just a Ruby library—an internal DSL.

DSLs and Metaprogramming

At the beginning of this book, we defined metaprogramming as “writing code that writes code” (or, if you want to be more precise, “writing code that manipulates the language constructs at runtime”). Now that you know about DSLs, you have another definition of metaprogramming: “designing a domain-specific language and then using that DSL to write your program.”

This is a book about the first definition, not a book about DSLs. To write a DSL, you have to deal with a number of challenges that are outside the scope of this book. You have to understand your domain, care about your language’s user-friendliness, and carefully evaluate the constraints and tradeoffs of your grammar. While writing this book, I opted to keep this particular can of worms shut.

Still, metaprogramming and DSLs have a close relationship in the Ruby world. To build an internal DSL, you must bend the language itself, and doing so requires many of the techniques described in this book. Put another way, metaprogramming provides the bricks that you need to build DSLs. If you’re interested in internal Ruby DSLs, this book contains information that’s important for you.