Functional Thinking (2014)
Chapter 7. Practical Thinking
Many of the examples I’ve featured thus far are abstract, because I’m illustrating the different thinking mode required for functional programming. But at some point you have to take these lessons back to the real world.
In this chapter, I cover some real-world occurrences of functional thinking, starting with Java 8 and moving to functional architectures and web frameworks.
Java 8
I’ve shown the lambda syntax for Java 8 in numerous examples throughout the book. Rather than merely bolt higher-order functions onto the language, the Java 8 engineers made clever choices to enable older interfaces to take advantage of functional features.
In Chapter 2, I showed an example of the “company process,” along with the Java 8 solution, repeated here in Example 7-1.
Example 7-1. Company process in Java 8
public String cleanNames(List<String> names) {
if (names == null) return "";
return names
.stream()
.filter(name -> name.length() > 1)
.map(name -> capitalize(name))
.collect(Collectors.joining(","));
}
private String capitalize(String e) {
return e.substring(0, 1).toUpperCase() + e.substring(1, e.length());
}
With streams in Java 8, you can chain and compose functions together until you call a function that generates output (called a terminal operation), such as collect() or forEach(). The filter() method in Example 7-1 is the same filter method I’ve shown throughout the book. Thefilter() method accepts a higher-order function that returns a Boolean value, which is used as the filtering criterion: true indicates inclusion in the filtered collection, and false indicates absence from the collection.
However, Java has added some syntactic sugar to the language in addition to the functional features. For example, I simplify the lambda block in Example 7-1 from filter((name) → name.length() > 1) to the shorter filter(name → name.length() > 1). For single parameters, the superfluous set of parentheses is optional. The return types also sometimes “hide” with the stream.
The filter() method accepts a type of Predicate<T>, which is a method that returns a Boolean value. You can explicitly create predicate instances if you like, as shown in Example 7-2.
Example 7-2. Creating a predicate manually
Predicate<String> p = (name) -> name.startsWith("Mr");
List<String> l = List.of("Mr Rogers", "Ms Robinson", "Mr Ed");
l.stream().filter(p).forEach(i -> System.out.println(i));
In Example 7-2, I create a predicate by assigning the filtering lambda block to it. When I call the filter() method on the third line, I pass the predicate as the expected parameter.
In Example 7-1, the map() method works as expected, applying the capitalize() method to each element in the collection. Finally, I call the collect() method, which is a terminal operation—a method that generates values from the stream. collect() performs the familiar reduce operation: combining elements to produce a (typically) smaller result, sometimes a single value (for example, a sum operation). Java 8 has a reduce() method, but collect() is preferred in this case because it works efficiently with mutable containers such as a StringBuilder.
By adding awareness of functional constructs such as map and reduce to existing classes and collections, Java faces the issue of efficient collection updates. For example, a reduce operation is much less useful if you can’t use it on typical Java collections such as ArrayList. Many of the collection libraries in Scala and Clojure are immutable by default, which enables the runtime to generate efficient operations. Java 8 cannot force developers to change collections, and many of the existing collection classes in Java are mutable. Therefore, Java 8 includes methods that perform mutable reduction for collections such as ArrayList and StringBuilder that update existing elements rather than replace the result each time. Although reduce() will work in Example 7-1, collect() works more efficiently for the collection returned in this instance. Understanding the difference between the two is the overhead with adding sophisticated capabilities to existing languages.
Functional Interfaces
A common Java idiom is an interface with a single method—called a SAM (single abstract method) interface—such as Runnable or Callable. In many cases, SAMs are used primarily as a transport mechanism for portable code. Now, portable code is best implemented in Java 8 with a lambda block. A clever mechanism called functional interfaces enables lambdas and SAMs to interact in useful ways. A functional interface is one that includes a single abstract method (and can include several default methods). Functional interfaces augment existing SAM interfaces to allow substitution of traditional anonymous class instances with a lambda block. For example, the Runnable interface can now be flagged with the @FunctionalInterface annotation. This optional annotation tells the compiler to verify that Runnable is an interface (and not a class or enum) and that the annotated type meets the requirements of a functional interface.
As an example of the substitutability of lambda blocks, I can create a new thread in Java 8 by passing a lambda block in lieu of a Runnable anonymous inner class:
new Thread(() -> System.out.println("Inside thread")).start();
Functional interfaces can seamlessly integrate with lambda blocks in myriad useful places. Functional interfaces are a notable innovation because they work well with established Java idioms.
With Java 8, you can also declare default methods on interfaces. A default method is a public nonabstract, nonstatic method (with a body), declared in an interface type and marked with the default keyword. Each default method is automatically added to classes that implement the interface—a convenient way to decorate classes with default functionality. For example, the Comparator interface now includes more than a dozen default methods. If I create a comparator by using a lambda block, I can trivially create the reverse comparator, as shown in Example 7-3.
Example 7-3. The Comparator class’s default methods
List<Integer> n = List.of(1, 4, 45, 12, 5, 6, 9, 101);
Comparator<Integer> c1 = (x, y) -> x - y;
Comparator<Integer> c2 = c1.reversed();
System.out.println("Smallest = " + n.stream().min(c1).get());
System.out.println("Largest = " + n.stream().min(c2).get());
In Example 7-3, I create a Comparator instance wrapped around a lambda block. Then, I can create a reverse comparator by calling the reversed() default method. The ability to attach default methods to interfaces mimics a common use of mixins and is a nice addition to the Java language.
The mixin concept is common across several languages. It originated with the Flavors language. The concept was inspired by an ice cream shop near the office where language development occurred. The ice cream parlor offered plain flavors of ice cream with any additional “mixins” (crumbled candy bars, sprinkles, nuts, etc.) that customers wanted.
Some early object-oriented languages defined attributes and methods of class together in a single block of code, whereupon the class definition was complete. In other languages, developers can define the attributes in one place but defer the method definitions until later and “mix” them into the class at the appropriate time. As object-oriented languages evolved, so did the details of how mixins work with modern languages.
In Ruby, Groovy, and similar languages, mixins augment existing class hierarchies as a cross between an interface and parent class. Like interfaces, mixins both act as types for instanceof checks and follow the same extension rule. You can apply an unlimited number of mixins to a class. Unlike interfaces, mixins not only specify the method signatures but also can implement the signatures’ behavior. Default methods in Java 8 provide mixins for Java, which allows the JDK to dispense with hacks such as the Arrays and Collections classes, which were nothing but static methods with no homes.
Optional
Notice the trailing call to get() in the terminal calls in Example 7-3. Calls to built-in methods such as min() return an Optional rather than a value in Java 8. This behavior mimics the behavior I discussed in Chapter 5. Optional prevents method returns from conflating null as an error with null as a legitimate value. Terminal operations in Java 8 can use the ifPresent() method to execute a code block only if a legitimate result exists. For example, this code prints the result only if a value is present:
n.stream()
.min((x, y) -> x - y)
.ifPresent(z -> System.out.println("smallest is " + z));
An orElse() method also exists that I can use if I want to take additional action. Browsing the Comparator interface in Java 8 is an illuminating look at how much power default methods add.
Java 8 Streams
Many functional languages and frameworks (such as Functional Java) contain a stream abstraction, each with subtle differences. Java 8 added streams and supports a nice representative subset of features.
The stream abstraction in Java 8 makes many advanced functional features possible. Streams act in many ways like collections, but with key differences:
§ Streams do not store values, but rather act as a pipeline from an input source to a destination through a terminal operation.
§ Streams are designed to be functional rather than stateful. For example, the filter() operation returns a stream of filtered values without modifying the underlying collection.
§ Stream operations try to be as lazy as possible (see Chapter 4).
§ Streams can be unbounded (or infinite). For example, you can construct a stream that returns all numbers and use methods such as limit() and findFirst() to gather subsets.
§ Like Iterator instances, streams are consumed upon use and must be regenerated before subsequent reuse.
Stream operations are either intermediate or terminal operations. Intermediate operations return a new stream and are always lazy. For example, using a filter() operation on a stream doesn’t actually filter the stream but rather creates a stream that will only return the filtered values when traversed by a terminal operation. Terminal operations traverse the stream, producing values or side effects (if you write functions that produce side effects, which is discouraged).
Functional Infrastructure
Most of the examples in this book feature advantages of functional programming on a small scale: replacing the Command design pattern with closures, using memoization, and so on. But what about the big pieces that developers must deal with every day, such as databases and software architecture?
Changing from anonymous inner classes to lambda blocks is easy because Java engineers built good solution mechanisms, and they made it easy to change your application piecemeal to use functional constructs. Unfortunately, it’s much harder to incrementally change your software architecture or the fundamental way you deal with data. In the next few sections, I explore how functional programming impacts the practical world.
Architecture
Functional architectures embrace immutability at the core level, leveraging it as much as possible. Embracing immutability is high on the list of ways to think like a functional programmer. Although building immutable objects in Java requires a bit more up-front complexity, the downstream simplification forced by this abstraction easily offsets the effort.
Immutable classes make a host of typically worrisome things in Java go away. One of the benefits of switching to a functional mindset is the realization that tests exist to check that changes occur successfully in code. In other words, testing’s real purpose is to validate mutation—and the more mutation you have, the more testing is required to make sure you get it right.
NOTE
There is a direct correlation between mutable state and tests: more of the first requires more of the latter.
If you isolate the places where changes occur by severely restricting mutation, you create a much smaller space for errors to occur and have fewer places to test. Because changes only occur upon construction, immutable classes make it trivial to write unit tests. You do not need a copy constructor, and you need never sweat the gory details of implementing a clone() method. Immutable objects make good candidates for use as keys in either maps or sets; keys in dictionary collections in Java cannot change value while being used as a key, so immutable objects make great keys.
Immutable objects are also automatically thread-safe and have no synchronization issues. They can also never exist in unknown or undesirable state because of an exception. Because all initialization occurs at construction time, which is atomic in Java, any exception occurs before you have an object instance. Joshua Bloch calls this failure atomicity: success or failure based on mutability is forever resolved once the object is constructed.
To make a Java class immutable, you must:
Make all fields final.
When you define fields as final in Java, you must either initialize them at declaration time or in the constructor. Don’t panic if your IDE complains that you don’t initialize them at the declaration site. It’ll realize that you’ve come back to your senses when you write the appropriate code in the constructor.
Make the class final so that it cannot be overridden.
If the class can be overridden, its methods’ behaviors can be overridden as well, so your safest bet is to disallow subclassing. Notice that this is the strategy used by Java’s String class.
Do not provide a no-argument constructor.
If you have an immutable object, you must set whatever state it will contain in the constructor. If you have no state to set, why do you have an object? Static methods on a stateless class would work just as well. Thus, you should never have a no-argument constructor for an immutable class. If you’re using a framework that requires this for some reason, see if you can satisfy it by providing a private no-argument constructor (which is visible via reflection).
Notice that the lack of a no-argument constructor violates the JavaBeans standard, which insists on a default constructor. But JavaBeans cannot be immutable anyway, because of the way their setXXX methods work.
Provide at least one constructor.
If you haven’t provided a no-argument one, this is your last chance to add some state to the object!
Do not provide any mutating methods other than the constructor.
Not only must you avoid typical JavaBeans-inspired setXXX methods, but you must also be careful not to return mutable object references. The fact that the object reference is final doesn’t mean that you can’t change what it points to. Thus, you need to make sure that you defensively copy any object references that you return from getXXX methods.
Groovy has syntactic sugar to handle the gory details of immutablility for you, as shown in Example 7-4.
Example 7-4. An immutable Client class
@Immutable
classClient {
String name, city, state, zip
String[] streets
}
By virtue of using the @Immutable annotation, this class has the following characteristics:
§ It is final.
§ Properties automatically have private backing fields with get methods synthesized.
§ Any attempts to update properties result in a ReadOnlyPropertyException.
§ Groovy creates both ordinal and map-based constructors.
§ Collection classes are wrapped in appropriate wrappers, and arrays (and other cloneable objects) are cloned.
§ Default equals(), hashcode(), and toString() methods are automatically generated.
The @Immutable annotation nicely illustrates my recurring theme: ceding implementation details to the runtime.
Tools such as object-relational mappers unfortunately assume mutable objects in many cases, and either work inefficiently or not at all with immutable objects. Thus, to see pervasive changes toward the functional paradigm will take major changes in many parts of existing systems.
Using a functional language such as Scala or Clojure and their frameworks makes it easier to build systems that embrace functional concepts at a deep level.
Some architectures exist that embrace functional ideals using existing infrastructure, such as Command-Query Responsibility Segregation (CQRS).
CQRS
Greg Young introduced the concept of CQRS and Martin Fowler wrote an influential description of the concept, which embodies many function aspects.
Traditional application architecture complects reading and writing data, typically to a relational database. Developers have expended vast amounts of time and effort to solve the object-relational mapping, with moderate success. Traditional application architecture resembles Figure 7-1.
Figure 7-1. Traditional application architecture
The model part of the application handles work such as business rules and validations. Typically, model objects also coordinate persistence, either internally or via another logical tier. Developers must consider mixed read and write implications throughout the model, adding complexity.
CQRS simplifies parts of your architecture by separating the read and command parts of the architecture. In the CQRS world illustrated in Figure 7-2, one part of the model (and perhaps specialized controllers as well) deal with database updates, while other parts of the model handle presentation and reporting.
Figure 7-2. CQRS architecture
The logic on the query side is typically much simpler because developers can assume immutability; updates go through another mechanism. Separate models can imply separate logical processes, perhaps even running on different machines. For example, one set of servers is responsible for display, and the user is routed to another subnet to make changes or additions.
Architecture is always about trade-offs. While making some things easier, CQRS complicates others. For example, if you have a monolithic database, transactions are easy. In CQRS, you will likely need to move to an eventual consistency model rather than transactional.
EVENTUAL CONSISTENCY
The eventual consistency model of distributed computing doesn’t require hard time limits on changes to a model but guarantees that, if no new updates occur, the model will eventually be consistent.
Transactions rely on ACID (Atomic, Consistent, Isolated, Durable), whereas eventual consistency uses BASE (Basically Available, Soft state, Eventual consistency).
Moving away from transactions is often forced as applications must scale. CQRS is a natural fit for architectural patterns such as event sourcing, in which you capture all the application state changes as a stream of events. Separating read from mutate allows for simpler logic; for example, everything on the read side can be immutable.
Web Frameworks
No discussion of new languages or paradigms is complete until the subject of web frameworks is addressed. Web programming is a terrific fit for functional programming: one can view the entire Web as a series of transformations from request to response.
Every functional language has a variety of web frameworks, with varying degrees of popularity. They all have most of these characteristics in common:
Routing frameworks
Most modern web application frameworks, including functional web frameworks, utilize routing libraries to decouple routing details from application functionality. Often, routing information is kept in a standard data structure (e.g., a Vector inside another Vector) that libraries know how to traverse.
Functions as targets
Many functional web applications use functions as the target for routing destinations. It’s easy to think about a web request as a function that accepts a Request and returns a Response; some portion of functional web frameworks represents syntactic sugar around this plumbing.
DSLs
Martin Fowler defines DSLs as computer programming languages of limited expressiveness, focused on a narrow problem domain. A popular type of DSL is an internal DSL, a new pseudolanguage written atop another language, using stylized syntactic sugar of the host language. TheRuby on Rails web framework and C#’s LINQ are good examples of this approach.
Most modern web frameworks use DSLs in several locations, including routing, embedded elements in HTML, database interaction, and so on. Use of DSLs is a common approach across a wide variety of languages, but functional languages in particlar favor declarative code, often the goal of DSLs.
Tight integration with build tools
Rather than being tethered to an IDE, most functional web frameworks have tight integration with command-line build tools, using them for everything from generating new project structure to running tests. IDEs and editors alike can automate around the existing tool chains, but the coupling between build tools and artifact production is tight, making it hard to rebuild.
As with all general-purpose languages, some parts of web development with functional languages will be easier and some will be harder. In general, coding with more immutable values cuts down on your testing burden: less state transition to verify via tests. Once you can establish conventions (such as immutability) throughout your architecture, it becomes easier for each piece to cooperate.
Databases
What is the motivation for relational databases doing destructive updates? In other words, every time you do a database update, you destroy the old value when you replace it with the new. Why are databases designed this way? To maximize storage space, so that your data won’t continue to grow. This architectural decision was embedded into database design decades ago, yet the landscape has changed. Resources (particularly virtual ones) are cheap now—Amazon will rent them to you for pennies! Yet developers still endure the pains imposed by the architectural constraints of yesteryear.
The Clojure community has been slowly building tools to support functional architectures from the browser down to the persistence tier. Datomic, the Clojure community’s entry into the commercial NoSQL database world, is particularly interesting because it is an example of how far designers can push functional concepts when they suffuse the entire architecture.
Datomic is an immutable database that timestamps each fact as it is entered. As the Datomic documentation observes, when President Obama took office, it didn’t mean that President Bush had never been president. By storing values rather than data, Datomic uses space quite efficiently. Once a value has been entered, all other instances of that value can point to the original (which can never change because it is immutable), making space usage efficient. If you need to update what your application sees at some point in the future, you can point to another value. Datomic has added the concept of time to information, allowing facts to always remain in their proper context.
Several interesting implications fall out of this design:
Permanent recording of all schema and data changes
Everything (including schema manipulation) is retained, making it trivial to migrate to an earlier version of your database. An entire category of tools (database migration tools) exists to solve this problem for relational databases.
Separation of reads and writes
Datomic’s architecture naturally separates read and write operations, meaning that updates are never delayed by queries. Thus, Datomic is architecturally a CQRS system internally.
Immutability and timestamps for event-driven architectures
Event-driven architectures rely on a stream of events to capture application state change. A database that captures and timestamps each piece of information is ideally suited, allowing rewind and playback as a feature of the database.
Datomic is just one example of the kinds of tools that developers will be able to create once they can cast off previous constraints and start building tools and frameworks with a deep paradigm change toward functional programming.