Introducing CQRS - Supporting architectures - Microsoft .NET Architecting Applications for the Enterprise, Second Edition (2014)

Microsoft .NET Architecting Applications for the Enterprise, Second Edition (2014)

Part III: Supporting architectures

Chapter 10. Introducing CQRS

Beware of false knowledge; it is more dangerous than ignorance.

—George Bernard Shaw

As discussed in Chapter 5, “Discovering the domain architecture,” there are two distinct but interoperating parts in Domain-Driven Design (DDD). The analytical part is about discovering the top-level architecture, using the ubiquitous language to dig out bounded contexts and their relationships. The strategic part is about giving each bounded context the most appropriate architecture. A decade ago, the standard architecture for a bounded context was a layered architecture, with a domain layer made of an object-oriented, all-encompassing model and domain services. The effort of developers was then to crunch knowledge about the domain and render it through a web of interconnected objects with state and behavior. The model was unique and intended to fully describe the entire business domain.

It didn’t look, in the beginning, like a thing that’s far easier to say than do.

Some projects that embraced DDD eventually worked; other projects failed. Success stories can be told too, but many people still believe that DDD is hard to do even though it can possibly deliver significant benefits. The point is that, for many people, the perception that DDD holds benefits is much less concrete than the perception of the damage that might result from using DDD and failing.

Is there anything wrong with DDD?

The analytical part of DDD has little to do with code and software design. It’s all about figuring out the top-level architecture while using ad hoc tools like the ubiquitous language. This is an excellent approach to take for just about any project. In complex scenarios, it helps to understand the big picture and lay out modules and services. In simple scenarios, it boils down to having just one context and a single module to build.

The critical part of the original vision of DDD is the suggested architecture for a bounded context. First and foremost, the layered architecture with an object-oriented model and services is just one option, and simpler solutions—for example, Content Management System (CMS), Customer Relationship Management (CRM), coupled CRUD, and two-layer systems—certainly are not banned as long as they fit the needs. Second, even when the layered architecture with a domain layer appears to be the ideal solution for a bounded context, the model doesn’t have to be object-oriented, nor does it have to be an all-encompassing model for the entire context.

In this chapter, we introduce a pattern that splits the domain model in two, actually achieving much more than just separation of concerns.

Separating commands from queries

Most of the difficulties that early adopters of DDD faced were in designing a single model to take care of all aspects of the domain. Generally speaking, any actions performed on a software system belong to one of the following two categories: query or command. In this context, a query is an operation that doesn’t alter in any way the state of the system and just returns data. The command, on the other hand, does alter the state of the system and doesn’t return data, except perhaps for a status code or an acknowledgment.

The logical separation that exists between queries and commands doesn’t show up clearly if the two groups of actions are forced to use the same domain model. For this reason, a new supporting architecture emerged in the past few years called CQRS, which is short for Command/Query Responsibility Segregation.

Generalities of the CQRS pattern

Since the days of ancient Rome, Divide et Impera has been an extremely successful approach to actually getting things done. Roman ruler Julius Caesar won a number of battles fighting against the entire enemy army, but when things got more complicated, he implemented a strategy of leading his enemy into dividing forces across the battlefields so that he could fight against a smaller army.

Similarly, the CQRS pattern is based on a simple, almost commonplace, idea: queries and commands (sometimes also referred to as reads and writes) are very different things and should be treated separately. Yet, for a long time, developers—like short-sighted commanders—insisted on having the same conceptual model for both queries and commands in their systems.

Especially in complex business scenarios, a single model soon becomes unmanageable. It doesn’t just grow exponentially large and complex (and subsequently absorb time and budget), it also never ends up working the way it should.


Image Note

We wouldn’t be too surprised to find out at some point that developers insisted for years on having a single model because having two distinct models might seem like a negative statement about their ability to work out a single, all-encompassing model. Our egos grow big sometimes and obscures a proper perspective on things!


From domain model to CQRS

In a way, CQRS is a form of lateral thinking resulting from the difficulty of finding a well-conceived model for complex domains. If the Domain Model turns out to be expensive and objectively complex, are we sure we’re approaching it right? That was probably the question that led to investigating and formalizing a different pattern.

At the end of the day, CQRS uses two distinct domain layers rather than just one. The separation is obtained by grouping operations that are queries in one layer and operations that are commands in another. Each layer, then, has its own architecture and its own set of services dedicated to only queries and commands, respectively. Figure 10-1 captures the difference.

Image

FIGURE 10-1 Visual comparison between Domain Model and CQRS.

In CQRS, it is not a far-fetched idea to have the query stack based exclusively on SQL queries and completely devoid of models, an application layer, and a domain layer. Having a full domain-model implementation in the query stack is not common. In general, the query stack should be simplified to the extreme. In addition, typically a CQRS approach has a different database for each side.

Structure of the query and command domain layers

As surprising as it might sound, the simple recognition that commands and queries are two different things has a deep impact on the overall architecture of the system. In Figure 10-1, we split the domain layer into two blocks.

Are they just two smaller and simpler versions of a domain layer like we discussed in the past two chapters?

The interesting thing is that with the architecture of the system organized as two parallel branches as shown in Figure 10-1, the requirement of having a full-fledged domain model is much less strict. For one thing, you might not need a domain model at all to serve queries. Queries are now just data to be rendered in some way through the user interface. There’s no command to be arranged on queried data; as such, many of the relationships that make discovering aggregates so important in a classic domain model are unnecessary. The model for the domain layer of the query side of a CQRS system can be simply a collection of made-to-measure data-transfer objects (DTOs). Following this consideration, the domain services might become just classes that implement pieces of business logic on top of an anemic model.

Similar things can be said for the domain layer of the command side of the system. Depending on the commands you actually implement, a classic domain model might or might not be necessary. In general, there’s a greater chance you might need a domain model for the command side because here you express business logic and implement business rules. At any rate, the domain model you might have on the command side of a CQRS system is likely far simpler because it is tailor-made for the commands.

In summary, recognizing that queries and commands are different things triggers a chain reaction that sets the foundation for domain modeling, as discussed in the past two chapters. We justified domain models as the ideal way to tackle complexity in the heart of software. Along the way, we ended up facing a good deal of complexity and thought it was, for the most part, complexity that is inherent to the business domain. Instead, most of that complexity results from the Cartesian product of queries and commands. Separating commands from queries can reduce complexity by an order of magnitude.


Image Note

Just in case you were wondering, a Cartesian product is a mathematical operation that, given two or more sets, returns a new and larger set made of all ordered pairs (or tuples), where each element belongs to a different set. The cardinality of the resulting set is the product of the cardinalities of all input sets.


CQRS is not a top-level architecture

Unlike DDD, CQRS is not a comprehensive approach to the design of an enterprise-class system. CQRS is simply a pattern that guides you in architecting a specific bounded context of a possibly larger system. Performing a DDD analysis based on a ubiquitous language and aimed at identifying bounded contexts remains a recommended preliminary step.

Next, CQRS becomes a valid alternative to Domain Model, CRUD, and other supporting architectures for the implementation of a particular bounded context.

Benefits of CQRS

The list of benefits brought about by using CQRS to implement a bounded context is not particularly long. Overall, we think that there are essentially two benefits. Their impact on the solution, though, is dramatic.

Simplification of the design

As our interactions with the Domain Model taught us, most of the complexity you face in a software system is usually related to operations that change the state of the system. Commands should validate the current state and determine whether they can run. Next, commands should take care of leaving the system in a consistent state.

Finally, in a scenario in which reading and writing operations share the same representation of data, it sometimes becomes hard to prevent unwanted operations from becoming available in reading or writing. We already raised this point in the last chapter when we pointed out that it’s nearly impossible to give, say, a list of order items a single representation that fits in both the query and command scenarios. Anyway, we’ll return to this aspect in a moment with a detailed example.

We stated in an earlier note that the complexity of the Domain Model results from the Cartesian product of queries and commands. If we take the analysis one step further, we can even measure by a rule of thumb the amount of reduced complexity. Let’s call N the complexity of queries and commands. In a single domain model, where requirements and constraints of queries affect commands and vice versa, like in a Cartesian product, you have a resulting complexity of NxN. By separating queries from commands and treating them independently, all you have is N+N.

Potential for enhanced scalability

Scalability has many faces and factors; the recipe for scalability tends to be unique for each system you consider. In general, scalability defines the system’s ability to maintain the same level of performance as the number of users grows. A system with more users performs certain operations more frequently. Scalability, therefore, depends on the margins that architects have to fine-tune the system to make it perform more operations in the same unit of time.

The way to achieve scalability depends on the type of operations most commonly performed. If reads are the predominant operation, you can introduce levels of caching to drastically reduce the number of accesses to the database. If writes are enough to slow down the system at peak hours, you might want to consider switching from a classic synchronous writing model to async writes or even queues of commands.

Separating queries from commands gives you the chance to work on the scalability aspects of both parts in total isolation.


Image Note

A good example of what it means to treat reads and writes separately is given by a cloud platform such as Microsoft Azure. You deploy query and command layers as distinct web or worker roles and scale them independently, both in terms of instances and the size of each instance. Similarly, when reads are vastly predominant, you can decide to offload some pages to a distinct server with a thick layer of caching on top. The ability to act on query and command layers separately is invaluable. Note also that the investment in Microsoft Azure is focused more on websites, WebJobs, and even mobile services rather than the original web roles and worker roles. In particular, WebJobs is more of a lightweight approach than web roles and worker roles.


Pleasant side effects of CQRS

A couple of other pleasant side effects of CQRS are worth noting here. First, CQRS leads you to a deep understanding of what your application reads and what it processes. The neat separation of modules also makes it safe to make changes to each without incurring some form of regression on one or the other.

Second, thinking about queries and commands leads to reasoning in terms of tasks and a task-based user interface, which is very good for end users.

Fitting CQRS in the business layer

Honestly, we don’t think there are significant downsides to CQRS. It all depends in the end on what you mean exactly by using CQRS. So far we just defined it as a pattern that suggests you have two distinct layers: one filled with the model and services necessary for reading, and one with the model and services for commands. What a model is—whether it is an object model, a library of functions, or a collection of data-transfer objects—ultimately is an implementation detail.

With this definition in place, nearly any system can benefit from CQRS and coding it doesn’t require doing things in a different way. Neither does it mean learning new and scary things.

Noncollaborative vs. collaborative systems

The point, however, is that CQRS also induces some deeper architectural changes that maximize the return in terms of scalability and reduced complexity but that require some investment in learning and performing a preliminary analysis.

In the end, CQRS was discovered by looking for more effective ways to tackle complex systems in which multiple actors—both end users and software clients—operate on the data concurrently and sophisticated and ever-changing business rules apply. The major proponents of CQRS—Udi Dahan and Greg Young—called these systems collaborative systems.

Let’s try to formalize the landmarks of a collaborative system a bit better.

In a collaborative system, the underlying data can change at any time from the effect of the current user, concurrent users connected through various front ends, and even back-end software. In a collaborative system, users compete for the same resources, and this means that whatever data you get can be stale in the same moment that it is read or even long before it is displayed. One of the reasons for this continuous change is that the business logic is particularly complex and involves multiple modules that sometimes need to be loaded dynamically. The architect has two main options:

Image Lock the entire aggregate for the time required to complete any operation.

Image Keep the aggregate open to change at the cost of possibly showing out-of-sync data that eventually becomes consistent. (This is often referred to as eventual consistency.)

The first option is highly impractical for a collaborative system—the back end would be locked while serving a single request at nearly any time, and the throughput would be very low. The second option might be acceptable but, if the system is not properly fine-tuned, it can end up giving inaccurate results and taking too long a time to respond.

This is the scenario that led to formalizing CQRS.

CQRS to the rescue

CQRS is not simply about using different domain layers for queries and commands. It’s more about using distinct stacks for queries and commands architected by following a new set of guidelines. (See Figure 10-2.)

Image

FIGURE 10-2 The big picture of the CQRS implementation of a collaborative system.

In the command pipeline, any requests from the presentation layer become a command appended to the queue of a processor. Each command carries information and has its own handler that knows about the logic. In this way, each command is a logical unit that can thoroughly validate the state of the involved objects and intelligently decide which updates to perform and which to decline. The command handler processes the command just once. Processing the command might generate events handled by other registered components. In this way, other software can perform additional tasks. One of the common tasks is performing periodical updates of a database cache that exists for the sole purpose of the query pipeline.

When the business logic is extremely sophisticated, you can’t afford to handle commands synchronously. You can’t do that for two reasons:

Image It slows down the system.

Image The domain services involved become way too complex, perhaps convoluted and subject to regression, especially when rules change frequently.

With a CQRS architecture, the logic can be expressed through single commands that result in distinct, individual components that are much easier to evolve, replace, and fix. In addition, these commands can be queued if necessary.


Image Note

In a CQRS scenario, one-way commands that do not return any response do not conceptually exist. They should be modeled as events fired for one or more event handlers to handle.


The query pipeline is quite simple, on the other hand. All it has is a collection of repositories that query content from ad hoc caches of denormalized data. The structure of such database cache tables (most of the time, plain Microsoft SQL Server tables) closely reflects the data required by the user interface. So, for example, if a page requires the customer name while displaying the order details, you can arrange to have the ID and name readymade in a cache without having to JOIN every time. Furthermore, because the query pipeline is separated, it can offload to a dedicated server at any time.

CQRS always pays the architecture bill

Many seem to think that outside the realm of collaborative systems, the power of CQRS diminishes significantly. On the contrary, the power of CQRS really shines in collaborative systems because it lets you address complexity and competing resources in a much smoother and overall simpler way. There’s more to it than meets the eye, we think.

In our opinion, CQRS can sufficiently pay your architecture bills even in simpler scenarios, where the plain separation between query and command stacks leads to simplified design and dramatically reduces the risk of design errors. You don’t need to have super-skilled teams of developers to do CQRS. Quite the opposite: using CQRS enables nearly any team to do a good job in terms of scalability and cleanliness of the design.

Transaction script in the command stack

CQRS is a natural fit in a system dependent on collaboration. However, the benefits of command/query separation can apply to nearly all systems.

Most systems out there can be summarized as “CRUD with some business logic around.” In these cases, you can just use the Transaction Script (TS) pattern (as discussed in Chapter 7, “The mythical business layer”) in the implementation of the command stack. TS is an approach that has you partition the back end of the system—overall, business logic—in a collection of methods out of a few container classes. Each method essentially takes care of a command and provides a full implementation for it. The method, therefore, takes care of processing input data, invoking local components or services in another bounded context, and writing to the database. All these steps take place in a single “logical” transaction.

As Fowler said, the glory of TS is in its simplicity. TS is a natural fit for applications with a small amount of logic. The major benefit of TS is there’s only minor overhead for development teams in terms of learning and performance.


Image Note

CQRS suggests—or just makes it reasonable sometimes—to use distinct databases for reading and writing. When this happens, the adoption of TS in the organization of the business logic raises the problem of figuring out the ideal way to handle eventual consistency. We’ll return to this point in the next chapter about CQRS implementation.


EDMX for the read model

What’s the easiest way to build a data access layer that serves the purposes of the presentation layer with no extra whistles and bells? Once you know the connection string to the database to access, all you do is create an Entity Framework wrapper in the form of an EDMX designer file in Microsoft Visual Studio.

Running the Entity Framework designer on the specified connection string infers an object model out of the database tables and relationships. Because it comes from Entity Framework, the object model is essentially anemic. However, the C# mechanism of partial classes enables you to add behavior to classes, thus adding a taste of object orientation and domain modeling to the results.

Arranging queries—possibly just LINQ queries—on top of this object model is easy for most developers, and it’s effective and reliable. Expert developers can work very quickly with this approach, and junior developers can learn from it just as quickly.

The pragmatic architect’s perspective

Taking the point of view of an architect, you might wonder what the added value of CQRS is in relatively simple systems with only a limited amount of business logic.

You set the architecture to define the boundaries of command and query stacks. You pass each developer an amount of work that is commensurate to that developer’s actual skills. You still have distinct stacks to be optimized independently or even rewritten from scratch if necessary.

In a word, an expert architect has a far better chance to take on the project comfortably, even with only junior developers on the team.

The query stack

Let’s delve a bit deeper into the two pipelines that make up the CQRS architecture. In doing so, another key aspect that drives the adoption of CQRS in some highly collaborative systems will emerge clearly—the necessity of dealing with stale data.

The read domain model

A model that deals only with queries would be much easier to arrange than a model that has to deal with both queries and commands. For example, a prickly problem we hinted at in Chapter 8, “Introducing the domain model,” is brilliantly and definitely solved with the introduction of a read-only domain model.

Why you need distinct models

The problem was summarized as follows. The Order class has an Items property that exposes the list of ordered products. The property holds inherently enumerable content, but which actual type should you use for the Items property? The first option that probably comes to mind isIList<T>. It might work, but it’s not perfect. So let’s put ourselves in a Domain Model scenario and assume we want to have a single model for the entire domain that is used to support both queries and commands. Also, let’s say we use a plain list for the Items property:

public IList<OrderItem> Items { get; private set; }

The private setter is good, but it prevents only users of an Order from replacing it. Any code that gets an instance of Order can easily add or remove elements from it. This might or might not be a legitimate operation; it depends on the use-case. If the use-case is managing the order, exposing order items through a list is just fine. If the use-case is showing the last 10 orders, a list is potentially dangerous because no changes to the order are expected.


Image Important

The domain model is the API of the business domain. Once publicly exposed, an API can be invoked to perform any action it allows. To ensure consistency, the API should not rely on developers to use it only the right way. If Murphy (of “Murphy’s laws”) were a software engineer, he would say something like, “If a developer can call an API the wrong way, he will.”


On the other hand, if you expose the list as a plain enumeration of order items, you have no way to create an order and add items to it. In addition, individual items are still modifiable through direct access:

public IEnumerable<OrderItem> Items { get; private set; }

Things don’t change even if you use ReadOnlyCollection<T> instead of IEnumerable. A Microsoft .NET Framework read-only collection is read-only in the sense that it doesn’t allow changes to the structure of the collection. Furthermore, if the read-only collection is created as a wrapper for a regular list, changes to the underlying list do not affect the read-only wrapper. Here’s an example where order items are exposed as a read-only collection but methods still make it possible to populate the collection:

public class Order
{
private readonly IList<OrderItem> _items;
public Order()
{
_items = new List<MOrderItem>();
}
public ReadOnlyCollection<OrderItem> Items
{
get
{
return new ReadOnlyCollection<OrderItem>(_items);
}
}

public void Add(int id, int quantity)
{
_items.Add(new OrderItem(id, quantity));
}
}
public class OrderItem
{
public OrderItem(int id, int quantity)
{
Quantity = quantity;
ProductId = id;
}
public int Quantity { get; /*private*/ set; }
public int ProductId { get; /*private*/ set; }
}

However, direct access to elements in the collection is still possible—whether it is gained during a for-each loop, out of a LINQ query, or by index:

foreach (var i in order.Items)
{
i.Quantity ++;
Console.WriteLine(i);
}

To prevent changes to the data within the collection, you have to make the setter private.

This would work beautifully if it weren’t for yet another possible issue. Is it worthwhile to turn the OrderItem entity of the domain model into an immutable object?

Classes in the domain model are modified and made more and more complex because they can be used interchangeably in both query and command scenarios. Using the read-only wrapper, ultimately, is the first step toward making a read version of the Order entity.


Image Note

We are not trying to say that having Items coded as a list is dangerous; instead, we just want to point out a consistency hole and a sort of violation of the syntax rules of the ubiquitous language. The order displayed for review is not the order created out of a request. This is what CQRS is all about.


From a domain model to a read model

When your goal is simply creating a domain model for read-only operations, everything comes easier and classes are simpler overall. Let’s look at a few varying points.

The notion of aggregates becomes less central, and with it the entire notion of the domain model as explained in Chapter 8. You probably still need to understand how entities aggregate in the model, but there’s no need to make this knowledge explicit through interfaces.

The overall structure of classes is more similar to data-transfer objects, and properties tend to be much more numerous than methods. Ideally, all you have are DTOs that map one-to-one with each screen in the application. Does that mean that model becomes anemic? Well, the model is 100 percent anemic when made of just data. An Order class, for example, will no longer have an AddItem method.

Again, there’s no issue with CQRS having a 100 percent anemic read model. Methods on such classes can still be useful, but only as long as they query the object and provide a quick way for the presentation or application layer to work. For example, a method IsPending on an Orderclass can still be defined as follows:

public bool IsPending()
{
return State == OrderState.Pending;
}

This method is useful because it makes the code that uses the Order class easier to read and, more importantly, closer to the ubiquitous language.

Designing a read-model façade

The query stack might still need domain services to extract data from storage and serve it up to the application and presentation layers. In this case, domain services, and specifically repositories, should be retargeted to allow only read operations on the storage.

Restricting the database context

In the read stack, therefore, you don’t strictly need to have classic repositories with all CRUD methods and you don’t even need to expose all the power of the DbContext class, assuming you’re in an Entity Framework Code-First scenario, as described in Chapter 9, “Implementing the domain model,” and as it will be used in future chapters.

In Chapter 9, we had a class wrapping the Entity Framework DbContext and called it DomainModelFacade. The structure of the class is shown here:

public class DomainModelFacade : DbContext
{
public DomainModelFacade() : base(“naa4e-09”)
{
Products = base.Set<Product>();
Customers = base.Set<Customer>();
Orders = base.Set<Order>();
}

public DbSet<Order> Orders { get; private set; }
public DbSet<Customer> Customers { get; private set; }
public DbSet<Product> Products { get; private set; }
...
}

The DbSet class provides full access to the underlying database and can be used to set up queries and update operations via LINQ-to-Entities. The fundamental step toward a query pipeline is limiting the access to the database to queries only. Here are some changes:

public class ReadModelFacade : DbContext
{
public ReadModelFacade() : base(“naa4e-09”)
{
Products = base.Set<Product>();
Customers = base.Set<Customer>();
Orders = base.Set<Order>();
}

public IQueryable<Customer> Customers
{
get { return _customers; }
}

public IQueryable<Order> Orders
{
get { return _orders; }
}

public IQueryable<Product> Products
{
get { return _products; }
}
...
}

Collections to query from the business logic on are now exposed via IQueryable interfaces. We said that the notion of aggregates loses focus in a read model. However, queryable data in the read-model façade mostly corresponds to aggregates in a full domain model.

Adjusting repositories

With a read-model façade, any attempt to access the database starts with an IQueryable object. You can still have a set of repository classes, populate them with a bunch of FindXxx methods, and use them from domain services and the application layer.

In doing so, you’ll certainly run into simple situations such as just needing to query all orders that have not been processed two weeks after they were placed. The FindXxx method can return a collection of Order items:

IEnumerable<Order> FindPendingOrderAfter(TimeSpan timespan);

But there are also situations in which you need to get all orders whose total exceeds a threshold. In this case, you need to report order details (like ID, date of creation, state, payment details) as well as customer details (at least the name and membership status). And, above all, you need to report the total of the order. There’s no such type in the domain; you need to create it. OK, no big deal: it’s just a classic DTO type:

IEnumerable<OrderSummary> FindOrdersBeyond(decimal threshold);

All is good if the OrderSummary DTO is general enough to be used in several repository queries. If it is not, you end up with too many DTO classes that are also too similar, which ultimately also poses a problem with names. But beyond the name and quantity of DTOs, there’s another underlying issue here: the number of repository methods and their names and implementation. Readability and maintainability are at stake.

A common way out is leaving only common queries as methods in the repositories that return common DTOs and handling all other cases through predicates:

public IEnumerable<T> Find(Expression<Func<T, Boolean>> predicate)

In this case, though, you’re stuck with using type T, and it might not be easy to massage any queried data into a generic DTO within a single method.


Image Important

We decided to introduce relevant aspects of CQRS starting from a DDD perspective and then discuss issues that arise from using it, as well as what has been done to smooth out the rough spots according to the key guidelines of CQRS.

As far as repositories are concerned, the bottom line is that you don’t likely need them in the query stack. The entire data access layer can be articulated through LINQ queries on top of some Object/Relational Mapper (O/RM) classes placed directly in the application layer. Also, a full-fledged O/RM like Entity Framework sometimes might be overkill. You might want to consider a micro O/RM for the job, such as PetaPoco. (See http://www.toptensoftware.com/petapoco.)

Looking ahead to .NET, a better option probably is the upcoming Entity Framework 7, which will be a lot more lightweight and aligned with ASP.NET vNext.


Layered expression trees

Over the past 20 years of developing software, we have seen a recurring pattern: when a common-use solution gets overwhelmingly complex and less and less manageable over time, it’s probably because it doesn’t address the problem well. At that point, it might be worth investigating a different approach to the problem. The different approach we suggest here to reduce the complexity of repositories and DTOs in a read model leverages the power of LINQ and expression trees.

Realistic scenarios

Let’s focus first on a few realistic scenarios where you need to query data in many different ways that are heavily dependent on business rules:

Image Online store Given the profile of the user, the home page of the online store will present the three products that match the profile with the highest inventory level. It results in two conceptual queries: getting all products available for sale, and getting the three products with the highest inventory level that might be interesting to the user. The first query is common and belongs to some domain service. The second query is application specific and belongs to the application layer.

Image ERP Retrieve all invoices of a business unit that haven’t been paid 30 days after their due payment terms. There are three conceptual queries here: getting all invoices, getting all invoices for the business unit, and getting all invoices for the business unit that are unpaid 30 days later. The first two queries are common and belong to some domain services. The third query sounds more application specific.

Image CMS Retrieve all articles that have been published and, among them, pick those that match whatever search parameters have been specified. Again, it’s two conceptual queries: one domain-specific and one application-specific.

Why did we use the term conceptual query?

If you look at it conceptually, you see distinct queries. If you look at it from an implementation perspective, you just don’t want to have distinct queries. Use-cases often require queries that can be expressed in terms of filters applied over some large sets of data. Each filter expresses a business rule; rules can be composed and reused in different use-cases.

To get this, you have two approaches:

Image Hide all filters in a repository method, build a single super-optimized query, run it, and return results. Each result is likely a different DTO. In doing this, you’re going to have nearly one method for each scenario and new or modified methods when something changes. The problem is not facing change; the problem is minimizing the effort (and risk of regression) when change occurs. Touching the repository interface is a lot of work because it might have an impact on upper layers. If you can make changes only at the application level, it would be much easier to handle and less invasive.

Image Try LINQ and expression trees.

Let’s see what it takes to use layered expression trees (LET).

Using IQueryable as your currency

The idea behind LET is enabling the application layer to receive IQueryable<T> objects wherever possible. In this way, the required query emerges through the composition of filters and the actual projection of data is specified at the last minute, right in the application layer where data is being used to generate the view model for the presentation to render.

With this idea in mind, you don’t even need repositories in a read model, and perhaps not even as a container of common queries that return direct and immediately usable data that likely will not be filtered any more. A good example of a method you might still want to have in a separate repository class is a FindById.

You can use the public properties of the aforementioned read façade as the starting point to compose your queries. Or, if necessary, you can use ad hoc components for the same purpose. In this way, in fact, you encapsulate the read-model façade—still a point of contact with persistence technology—in such components. Here’s what the query to retrieve three products to feature on the home page might look like. This code ideally belongs to the application layer:

var queryProducts = (from p in CatalogServices.GetProductsAvailableForSale()
orderby p.UnitsInStock descending
select new ProductDescriptor
{
Id = p.Id,
Name = p.Name,
UnitPrice = p.UnitPrice,
UnitsInStock = p.UnitsInStock,
}).Take(3);

Here’s another example that uses the recommended async version of LINQ methods:

var userName = _securityService.GetUserName();
var currentEmployee = await _database
.Employees
.AsNoTracking()
.WhereEmployeeIsCurrentUser(userName)
.Select(employee =>
new CurrentEmployeeDTO
{
EmployeeId = employee.Id,
FirstName = employee.PersonalInformation.FirstName,
LastName = employee.PersonalInformation.LastName,
Email = employee.PersonalInformation.Email,
Identifier = employee.PersonalInformation.Identifier,
JobTitle = employee.JobTitle,
IsManager = employee.IsTeamManager,
TeamId = employee.TeamId,
}).SingleOrDefaultAsync();
currentEmployee.PictureUrl = Url.Link(“EmployeePicture”,
new { employeeId = currentEmployee.EmployeeId });

As you might have noticed, the first code snippet doesn’t end with a call to ToList, First, or similar methods. So it is crucial to clarify what it means to work with IQueryable objects.

The IQueryable interface allows you to define a query against a LINQ provider, such as a database. The query, however, has deferred execution and subsequently can be built in multiple steps. No database access is performed until you call an execution method such as ToList. For example, when you query all products on sale, you’re not retrieving all 200,000 records that match those criteria. When you add Take(3), you’re just refining the query. The query executes when the following code is invoked:

var featuredProducts = queryProducts.ToList();

The SQL code that hits the database has the following template:

SELECT TOP 3 ... WHERE ...

In the end, you pass the IQueryable object through the layers and each layer can add filters along the way, making the query more precise. You typically resolve the query in the application layer and get just the subset of data you need in that particular use-case.

Isn’t LET the same as an in-memory list?

No, LET is not the same as having an in-memory list and querying it via LINQ-to-Objects. If you load all products in memory and then use LINQ to extract a subset, you’re discarding tons of data you pulled out of the database.

LET still performs a database access using the best query that the underlying LINQ provider can generate. However, IQueryable works transparently on any LINQ provider. So if the aforementioned method GetProductsAvailableForSale internally uses a static list of preloaded Productinstances, the LET approach still works, except that it leverages LINQ-to-Objects instead of the LINQ dialect supported by the underlying database access layer.

Using LET is not the same as having a static list, but that doesn’t mean having a static list is a bad thing. If you see benefits in keeping, say, all products in memory, a static list is probably a good approach. LET is a better approach if the displayed data is read from some database every time.


Image Note

Crucial to CQRS is the fact that the database you query might not be the core database where commands write. It can easily be a separate database optimized for reading and built to denormalize some of the content in the core database. This approach is often referred to as the “pure CQRS approach.”


Upsides of LET

The use of LET has several benefits. The most remarkable benefit is that you need almost no DTOs. More precisely, you don’t need DTOs to carry data across layers. If you let queries reach the application layer, all you do is fetch data directly in the view model classes. On the other hand, a view model is unavoidable because you still need to pass data to the user interface in some way.


Image Note

As you might have noticed, we’re using different names—DTO and view model classes—for two software entities that can be described using the same words: classes that just carry data. ASP.NET MVC view model classes are actually DTOs, and the reason we’re using different names here is to emphasize that one of the benefits of LET is that you can forget about intermediate classes you might need in the classic Domain Model to carry data across layer. In CQRS with LET, all you need is LINQ to query data and a DTO to return data to the presentation. There are no other intermediaries—just LINQ queries and, in ASP.NET MVC, view model classes.


Another benefit is that the code you write is somehow natural. It’s really like you’re using the database directly, except that the language is much easier to learn and use than plain-old T-SQL.

Queries are DDD-friendly because their logic closely follows the ubiquitous language, and sometimes it seems that domain experts wrote the queries. Among other things, DDD-friendly queries are also helpful when a customer calls to report a bug. You look into the section of the code that produces unexpected results and read the query. You can almost read your code to the customer and quickly figure out whether the reason unexpected data is showing up on the screen is logical (you wrote the wrong query) or technical (the implementation is broken). Have a look at the following code:

var db = new ReadModelFacade();
var model = from i in db.IncomingInvoices
.ForBusinessUnit(buId)
.Expired()
orderby i.PaymentDueDate
select new SummaryViewModel.Invoice
{
Id = i.ID,
SupplierName = i.Party.Name,
PaymentDueDate = i.PaymentDueDate.Value,
TotalAmount = i.TotalPrice,
Notes = i.Notes
};

The code filters all invoices to retrieve those charged to a given business unit that haven’t been paid yet. Methods like ForBusinessUnit and Expired are (optional) extension methods on the IQueryable type. All they do is add a WHERE clause to the final query:

public static IQueryable<Invoice> ForBusinessUnit(this IQueryable<Invoice> query, int buId)
{
var invoices = from i in query
where i.BusinessUnit.OrganizationID == buId
select i;
return invoices;
}

Last but not certainly least, LET fetches all data in a single step. The resulting query might be complex, but it is not necessarily too slow for the application. Here we can’t help quoting the timeless wisdom of Donald Knuth: “Premature optimization is the root of all evil.” As Andrea repeats in every class he teaches, three things are really important in the assessment of enterprise architecture: measure, measure, and measure. We’re not here to say that LET will always outperform any other solution, but before looking for alternative solutions and better SQL code, first make sure you have concrete evidence that LET doesn’t work for you.

Downsides of LET

Overall LET is a solution you should always consider, but like anything else it is not a silver bullet. Let’s see which factors might make it less than appealing.

The first point to consider is that LET works beautifully on top of SQL Server and Entity Framework, but there’s no guarantee it can do the same when other databases and, more importantly, other LINQ providers are used.

LET sits in between the application layer and persistence in much the same way repositories do. So is LET a general abstraction mechanism? The IQueryable interface is, in effect, an abstraction layer. However, it strictly depends on the underlying LINQ provider, how it maps expression trees to SQL commands, and how it performs. We can attest that things always worked well on top of Entity Framework and SQL Server. Likewise, we experienced trouble using LET on top of the LINQ provider you find in NHibernate. Overall, the argument that LET is a leaky abstraction over persistence is acceptable in theory.

In practice, though, not all applications are really concerned about switching the data-access engine. Most applications just choose one engine and stick to that. If the engine is SQL Server and you use Entity Framework, the LET abstraction is not leaky. But we agree that if you’re building a framework that can be installed on top of your database of choice, repositories and DTOs are probably a better abstraction to use.

Finally, LET doesn’t work over tiers. Is this a problem? Tiers are expensive, and we suggest you always find a way to avoid them. Yet sometimes tiers provide more scalability. However, as far as scalability is concerned, let us reiterate a point we made in a past chapter: if scalability is your major concern, you should also consider scaling out by keeping the entire stack on a single tier and running more instances of it on a cloud host such as Microsoft Azure.


Image Note

When you use LET, testing can happen only on top of the LINQ-to-Objects provider built into the .NET Framework or any other LINQ provider that can be used to simulate the database. In any case, you’re not testing LET through the real provider. For the nature of LET, however, this is the barrier that exists between unit and integration tests.


The command stack

In a CQRS scenario, the command stack is concerned only about the performance of tasks that modify the state of the application. As usual, the application layer receives requests from the presentation and orchestrates their execution. So what’s going to be different in a CQRS scenario?

As shown in Figure 10-1, CQRS is about having distinct domain layers where the business logic—and objects required to have it implemented—is simpler to write because of the separation of concerns. This is already a benefit, but CQRS doesn’t stop here. Additionally, it lays the groundwork for some more relevant design changes.

In the rest of the chapter, we drill down into concepts that slowly emerged as people increasingly viewed query and command stacks separately. These concepts are still evolving and lead toward the event-sourcing supporting architecture we’ll discuss thoroughly in the next couple of chapters.

Getting back to presentation

A command is an action performed against the back end, such as registering a new user, processing the content of a shopping cart, or updating the profile of a customer. From a CQRS perspective, a task is monodirectional and generates a work flow that proceeds from the presentation down to the domain layer and likely ends up modifying some storage.

Tasks are triggered in two ways. One is when the user explicitly starts the task by acting on some UI elements. The other is when some autonomous services interact asynchronously with the system. As an example, you can think of how a shipping company interacts with its partners. The company might have an HTTP service that partners invoke to place requests.

The command placed updates the state of the system, but the caller might still need to receive some feedback.

Tasks triggered interactively

Imagine a web application like the I-Buy-Stuff online store we presented in Chapter 9. When the user clicks to buy the content of the shopping cart, she triggers a business process that creates an order, places a request for delivery, and processes payment—in a nutshell, it modifies the state of multiple systems, some interactively and some programmatically.

Yet the user who originally triggered the task is there expecting some form of feedback. That’s no big deal when commands and queries are in the same context—the task modifies the state and reads it back. But what about when commands and queries are in separated contexts? In this case, you have two tasks: one triggered interactively and one triggered programmatically. Here’s some code from an ASP.NET MVC application:

[HttpPost]
[ActionName(“AddTo”)]
public ActionResult AddToShoppingCart(int productId, int quantity=1)
{
// Perform the requested task using posted input data
var cart = RetrieveCurrentShoppingCart();
cart = _service.AddProductToShoppingCart(cart, productId, quantity);
SaveCurrentShoppingCart(cart);

// Query task triggered programmatically
return RedirectToAction(“AddTo”);
}

[HttpGet]
[ActionName(“AddTo”)]
public ActionResult DisplayShoppingCart()
{
var cart = RetrieveCurrentShoppingCart();
return View(“shoppingcart”, cart);
}

The method AddToShoppingCart is a command triggered interactively, as evidenced by the HttpPost attribute. It reads the current state of the shopping cart, adds the new item, and saves it back. The command Add-to-Shopping-Cart ends here, but there’s a user who still needs some feedback.

In this specific case—an ASP.NET application—you need a second command triggered programmatically that refreshes the user interface by placing a query or, like in this case, performing an action within the realm of the application layer. This is the effect of RedirectToAction, which places another HTTP request—a GET this time—that invokes the DisplayShoppingCart method.

What if you have a client-side web application—for example, a Single-Page application? In this case, you use some JavaScript to trigger the call to a Web API or SignalR endpoint. The task completes, but this time there’s no strict need for the web back end to execute a second task programmatically to get back to the presentation. The nature of the client makes it possible to display feedback in the form of an acknowledgment message:

$("#buttonBuy").click(function() {
// Retrieve input data to pass
...
$.post(url, { p1: ..., p2: ... })
.done(function(response) {
// Use the response from the task endpoint to refresh the UI
...
});
});

A similar mechanism applies when the client is a desktop or mobile application, whether it’s Microsoft Windows, Windows Store, Windows Phone, Android, iOS, or something else.


Image Important

In the case of an ASP.NET front end, the use of a redirect call to refresh the user interface is doubly beneficial because it defeats the notorious F5/Refresh effect. Browsers usually keep track of the last request and blindly repeat it when the user presses F5 or refreshes the current page. Especially for tasks that update the state of the system, reiterating a post request might repeat the task and produce unwanted effects—for example, the same item can be added twice to the shopping cart. A page refreshed after the task through a redirect leaves a GET operation in the browser memory. Even if it is repeated, no bad surprises can show up.


Tasks triggered programmatically

A system that exposes a public HTTP API is subject to receive calls from the outside. The call merely consists of the invocation of a method through an HTTP request, and the response is just an HTTP response. Here, the fundamentals of HTTP rule over all. Here’s a sample Web API template for tasks to be invoked programmatically:

public class ExternalRequestController : ApiController
{
public HttpResponseMessage PostDeliveryRequest(DeliveryRequest delivery)
{
// Do something here to process the delivery request coming from a partner company
...

// Build a response for the caller:
// Return HTTP 201 to indicate the successful creation of a new item
var response = Request.CreateResponse<String>(HttpStatusCode.Created, “OK”);

// Add the location of new item for their reference
var trackingId = ...;
var path = “/delivery/processed/” + delivery.PartnerCode + “/” + trackingId;
response.Headers.Location = new Uri(Request.RequestUri, path);
return response;
}
}

In this example, you have a Web API controller that receives delivery requests from a partner company. The request is processed and generates a tracking ID that must be communicated back to indicate the success of the operation.

There are various ways you can do this, and it mostly depends on your personal perspective regarding Web APIs. If you’re a REST person, you would probably go with the code shown earlier. If you’re more inclined toward remote procedure calls (RPCs), you can just return the tracking ID as a plain string in a generic HTTP 200 response.

Formalizing commands and events

All software systems receive input from some front-end data source. A data source can be any number of things, like a sensor connected to a hardware device that pumps real-time data, a feed asynchronously provided by a remote service, or—the most common scenario—a presentation layer equipped with a comfortable user interface. The input data travels from the front end to the application layer, where the processing phase of the input data is orchestrated.

Abstractly speaking, any front-end request for input processing is seen as a message sent to the application layer—the recipient. A message is a data transfer object that contains the plain data required for any further processing. In such an architecture, it is assumed that messages are fully understood by the recipient. Such a definition of a message leaves room for a number of concrete implementations. In most cases, you might want to start with a Message base class that acts as a data container:

public class Message
{
// Optionally, a few common properties here.
// The class, however, can even be a plain marker with no properties.
...
}

The front end can deliver the message to the application layer in a number of ways. Commonly, the delivery is a plain method invocation—for example, an application service invoked from within an ASP.NET MVC controller method. In more sophisticated scenarios, such as where scalability is the top priority, you might want to have a service bus in your infrastructure that also supports brokered messaging. In this way, you ensure delivery of the message to the intended recipient under any conditions, including when the recipient is not online.

Events vs. commands

There are two types of messages: commands and events. In both cases, messages consist of a packet of data. Some subtle differences exist, however, between events and commands.

A command is an imperative message that sounds like an explicit request made to the system to have some tasks performed. Here are some other characteristics of a command:

Image A command is directed at one handler.

Image A command can be rejected by the system.

Image A command can fail while being executed by some handler.

Image The net effect of the command can be different depending on the current state of the system.

Image A command doesn’t generally trespass the boundaries of a given bounded context.

Image The suggested naming convention for commands says that they should be imperative and specify what needs to be done.

An event is a message that serves as a notification for something that has already happened. It has the following characteristics:

Image An event can’t be rejected or canceled by the system.

Image An event can have any number of handlers interested in processing it.

Image The processing of an event can, in turn, generate other events for other handlers to process.

Image An event can have subscribers located outside the bounded context from which it originated.


Image Note

The key difference between CQRS and the Event Sourcing architecture we’ll cover in Chapter 12, “Introducing Event Sourcing,” is this: in an Event Sourcing scenario, messages can be persisted to form a detailed and exhaustive audit log. This gives you a great chance at any later time to look back at what has happened within the system. Once you have the record of all that happened, you can set up a what-if elaboration, replay events to figure out the current state, and extrapolate models of any kind.


Writing an event class

In terms of source code, commands and events are both classes derived from Message. Dealing with commands and events through different classes makes the design of the system more logical and simpler overall. Here’s a sample command class:

public class CheckoutCommand : Message
{
public string CartId { get; private set; }
public string CustomerId { get; private set; }

public CheckoutCommand(string cartId, string customerId)
{
CartId = cartId;
CustomerId = customerId;
}
}

Conversely, here’s the layout of an event class.

public class DomainEvent : Message
{
// Common properties
...
}

public class OrderCreatedEvent : DomainEvent
{
public string OrderId { get; private set; }
public string TrackingId { get; private set; }
public string TransactionId { get; private set; }

public OrderCreatedEvent(string orderId, string trackingId, string transactionId)
{
OrderId = orderId;
TrackingId = trackingId;
TransactionId = transactionId;
}
}

As you can see, the structure of event and command classes is nearly the same except for the naming convention. A fundamental guideline for designing domain events is that they should be as specific as possible and clearly reveal intent.

As an example, consider the form through which a customer updates the default credit card he uses in transactions. Should you fire a rather generic CustomerUpdated event? Or is a more specific CreditCardUpdated event preferable? Both options lead to a working solution, but which option works for you should be evident and stand on its own. It depends on the ubiquitous language and the level of granularity you have in place. We believe that a finer granularity here is a significant asset.

We generally recommend that the intent of the event be made clear in the name and all ambiguity be removed. If some ambiguity around a single event surfaces, you’ll probably find that it’s safer to have two distinct events.

Which properties should you have in the base class DomainEvent?

We’d say that at a minimum you want to have a timestamp property that tracks the exact time at which the event was fired. Moreover, you might want to have a property containing the name (or ID) of the user who caused the firing of the event. Another piece of data you might want to have is a version number that handlers can use to determine whether they can or cannot handle the event. The point here is that the definition of an event might change over time. A version number can help in this regard.

The implementation of the version number is completely up to the team. It can be a Version object as well as a string or a number. It can be bound to the application build number, or it can even be referred to the version of the event class:

public class DomainEvent : Message
{
public DateTime TimeStamp { get; private set; }
public DomainEvent()
{
TimeStamp = DateTime.Now;
}
}

An event class should be considered immutable for the simple reason that it represents something that has already happened. Immutable here means that there should be no way to alter the value of properties. The combination of private setters, no write methods, and a plain constructor will do the trick.

Handling commands and events

Commands are managed by a processor that usually is referred to as a command bus. Events are managed by an event bus component. It is not unusual, however, that commands and events are handled by the same bus. Figure 10-3 presents the overall event-based architecture of a CQRS solution. This architecture is more standard and is an alternative to the one we mentioned earlier that was based on the TS pattern.

Image

FIGURE 10-3 The command stack of an event-based CQRS architecture.

Any interaction that takes place in the user interface generates some requests to the system. In an ASP.NET MVC scenario, these requests take the form of controller actions and methods in the application layer. In the application layer, a command is created and pushed to some machinery for actual processing.

The bus component

The command bus holds a list of known business processes that can be triggered by commands. Active instances of such processes can be further advanced by commands. Processing a command can sometimes generate an event within the domain; the generated event is published to the samecommand bus or to a parallel event bus, if any. Processes that handle commands and related events are usually referred to as sagas.

The command bus is a single class that receives messages (requests of executing commands and notifications of events) and finds a way to process them. The bus doesn’t actually do the work itself; instead, it selects a registered handler that can take care of the command or event. Here’s a possible template for a bus class that handles commands and events. We use the interface IHandles as a placeholder for actions. The interface has a single void method:

public interface IHandles
{
void Handle(T message);
}

The bus uses the interface to handle both commands and events:

public class Bus
{
private static readonly Dictionary<Type, Type> SagaStarters =
new Dictionary<Type, Type>();
private static readonly Dictionary<string, object> SagaInstances =
new Dictionary<string, object>();

public static void RegisterSaga<TStartMessage, TSaga>()
{
SagaStarters.Add(typeof(TStartMessage), typeof(TSaga));
}

public static void Send<T>(T message) where T : Message
{
// Publish the event
if (message is IDomainEvent)
{
// Invoke all registered sagas and give each
// a chance to handle the event.
foreach(var saga in SagaInstances)
{
var handler = (IHandles<T>) saga;
if (handler != null)
handler.Handle(message);
}
}

// Check if the message can start one of the registered sagas
if (SagaStarters.ContainsKey(typeof (T)))
{
// Start the saga creating a new instance of the type
var typeOfSaga = SagaStarters[typeof (T)];
var instance = (IHandles<T>) Activator.CreateInstance(typeOfSaga);
instance.Handle(message);

// At this point the saga has been given an ID;
// let's persist the instance to a (memory) dictionary for later use.
var saga = (SagaBase) instance;
SagaInstances.Add(saga.Data.Id, instance);
return;
}

// The message doesn't start any saga.
// Check if the message can be delivered to an existing saga instead
if (SagaInstances.ContainsKey(message.Id))
{
var saga = (IHandles<T>) SagaInstances[message.Id];
saga.Handle(message);

// Saves saga back or remove if completed
if (saga.IsComplete())
SagaInstances.Remove(message.Id);
else
SagaInstances[message.Id] = saga;
}
}
}

The bus has two internal dictionaries: one to map start messages and saga types, and one to track live instances of sagas. In the latter dictionary, you can typically have multiple instances of the same saga type that are bound to different IDs.

The saga component

In general, a saga component looks like a collection of logically related methods and event handlers. Each saga is a component that declares the following information:

Image A command or event that starts the process associated with the saga

Image Commands the saga can handle and events the saga is interested in

Whenever the bus receives a command (or an event) that can start a saga, it creates a new saga object. The constructor of the saga generates a unique ID, which is necessary to handle concurrent instances of the same saga. The ID can be a GUID as well as a hash value from the starter command or anything else, like the session ID. Once the saga is created, it executes the command or runs the code that handles the notified event. Executing the command mostly means writing data or executing calculations.

At some point in the lifetime of the saga instance, it might be necessary to send another command to trigger another process or, more likely, fire an event that advances another process. The saga does that by pushing commands and events back to the bus. It might also happen that a saga stops at some point and waits for events to be notified. The concatenation of commands and events keeps the saga live until a completion point is reached. In this regard, you can also think of a saga as a workflow with starting and ending points.

Events raised by sagas pass through the bus, and the bus passes them as messages to whomever subscribed to that event. Raising an event from within the saga requires code like that shown here:

// Raise the PaymentCompleted event
var theEvent = new PaymentCompletedEvent( /* add transaction ID */ );
theEvent.SagaId = message.Cart.Id;
Bus.Send(theEvent);

From the application layer, you invoke the bus as shown in the following example. This example simulates a scenario in which an ASP.NET MVC application is called back from the payment page on a bank service gateway:

public ActionResult CompleteProcessOrder(String transactionId)
{
// Retrieve shopping cart from session state
var cart = RetrieveCurrentShoppingCart();

// Prepare and queue a Process-Order command
var command = new ProcessOrderCommand(transactionId, cart.CartId);
Bus.Send(command);

// Refresh view: in doing so, results of previous command might be captured, if ready.
return RedirectToAction(“Done”);
}

As you might have noticed, the command bus doesn’t return a response. This is not coincidental. The refresh of the user interface—wherever necessary—is left to a subsequent read command that queries data from any place—storage, cache, or whatever—where output is expected to be.

What we mostly want from a command bus is to decouple the application layer from the domain services. Doing so opens up new opportunities, such as handling domain service calls asynchronously. Other scenarios that the command bus can simplify are adding cross-cutting filters along the way, such as transactions, logging, and general injection points for optional logic. In any of these cases, all you need to do is change the command-bus class—no changes are required to the application layer and domain services.


Image Important

As battlefield experience grows around CQRS, some practices consolidate and tend to become best practices. Partly contrary to what we just stated about async domain services, it is a common view today to think that both the command handler and the application need to know how the transactional operation went. Results must be known, and if the command needs to run asynchronously, it should be designed as an event rather than as a command.


The combined effect of commands and events

When you write systems based on very dynamic business rules, you might want to seriously consider leaving the door open to making extensions and changes to certain workflows without patching the system.

Think, for example, of an e-commerce website that launches a marketing campaign so that any user performing certain actions through the site will accrue points on some sort of a customer-loyalty program. Because of the huge success of the campaign, the company decides to extend the incentives to more actions on the site and even double the points if a given action (say, buying suggested products) is repeated over time. How would you effectively handle that?

You can certainly fix and extend the application layer, which turns out to be the nerve center that governs the workflow. Anyway, when the business logic is expressed through the composition of small, independent and even autonomous commands and events, everything is much easier to handle. This is a classic scenario for collaborative systems. Let’s say you use the flowchart in Figure 10-4 to implement the process of an order being submitted.

Image

FIGURE 10-4 A sample workflow that handles the purchase of a few products.

Having the entire business logic of the workflow in a single method might make maintenance and testing problematic. However, even in the simple case of having split blocks in the workflow in simple commands, it might be difficult to do things properly. Let’s consider the update of the customer’s status in the fidelity program after an order is processed.

Instead of having the Register-Order command invoke the Update-Fidelity-Card command, isn’t it more flexible if the Register-Order command just fires an event saying that a new order has been created? With this approach, a handler for that event can kick off and check whether the customer is eligible for Gold status and fire another event for other handlers to jump in and update databases. This would isolate each action, keep each form of validation small and in reusable pieces, and keep the amount of logic in each command to the bare minimum.


Image Note

When it comes to events, we highly recommend that the naming convention reflect the occurrence of something. For example, the generic Not-In-Store and Is-Gold events should be implemented through classes with more evocative names, such asItemWasNotInStoreWhenPayingEvent and CustomerReachedGoldStatusEvent.


Sagas and transactions

Looking back at Chapter 8 where we introduced the Domain Model architecture, we can say that a saga is the evolution of the application service concept.

In an application service you orchestrate the tasks that serve a user request. The application service takes care of setting up transactions--possibly distributed transactions--and like a state-machine it coordinates the next step until the end of a workflow is reached. Abstractly speaking, a saga does the same except that it removes the need for a single component to know about the entire workflow and uses events to break an otherwise monolithic workflow into the combined effect of smaller command handlers raising events to coordinate with others.

This makes the entire solution a lot more flexible and reduces the surface of code subject to changes when business changes.

In addition, a saga reduces the need of distributed transactions when long-running processes are involved that span across multiple bounded contexts. The structure of a saga--a collection of rather independent command and event handlers--lends to easily define compensating behavior for each command executed. Thus if at some point a saga command fails, it can execute the compensating behavior for what it knows has happened and raise an event at the saga level to undo or compensate the work completed.


Image Note

In the case of failures, saga methods might receive the same event twice or more. It might not be easy to figure out the event is a repeat. If it can be figured out, methods should avoid repeating action (idempotency)


Downsides of having a command bus

There are contrasting feelings about the command bus. A common criticism is that it just adds an extra layer and contributes to making the code less easy to read. You need to go through some more classes to figure out what happens.

This aspect can be mitigated by using proper naming conventions so that the name of the handler matches the name of the command class being queued to the bus. In this way, you know exactly where to look, and looking into the implementation of methods takes only a few more clicks than with application services, as discussed in past chapters.

With classic application services, you see the call to the method right from controllers and can use a tool like ReSharper to jump directly to the source code. With a command bus, you see the command and know by convention the name of the class in which you need to look.

Readymade storage

Most real-world systems write data and read it back later. Before CQRS appeared on the horizon, reads and writes took place within the same stack and often within the same transactional context. CQRS promotes the neat separation of queries and commands and pushes the use of different stacks. As you saw in this chapter, though, the internal architecture of the two stacks can be significantly different. You typically have a domain layer in the command stack and a far simpler data-access layer in the query stack.

What about databases, then? Should you use different databases too: one to save the state of the system and one to query for data?

Optimizing storage for queries

Many real-world systems use a single database for reading and writing purposes—and this happens regardless of CQRS architectures. You can have distinct command and query models and still share the same database. It’s all about the scenarios you deal with.

Our thought is that using CQRS—at least in its lightweight form of using different models—is mostly beneficial for every system because it splits complexity, enables even different teams to work in parallel, and can suggest using something even simpler than a Domain Model for each stack. So CQRS is also about simplification.

However, for most practical purposes, using a single relational database is still the best option. When CQRS is used for scalability in a highly collaborative system, however, you might want to think about a couple of other aspects. (CQRS with a single database is sometimes referred to ashybrid-CQRS.)

In the query stack, what you query for has nearly the same schema as the view model. Most likely, these view models come from tables that are largely denormalized and that have very few links to other tables. In other words, the tables you really need are specialized for the queries you need to run.

A question that naturally arises is this: do you actually need a relational database for queries? We say you don’t strictly need a relational database, but it’s probably the best option you have, even though NoSQL solutions might be handier at times. More importantly—we’d say—you don’t strictly need the same database where the state of the system is persisted.

Creating a database cache

If you’re using one database to store the state of the system and another database to store data in a format that is quick and effective to query for the purposes of presentation, who’s in charge of keeping the two databases in sync?

This seems to be the final step of nearly any command whose action affects the query database. To avoid giving too many responsibilities to each command, the most obvious approach is that any command whose action affects the query database fires an event at the end. The event handler will then take care of updating the query database. The query database doesn’t fully represent the business behind the application; it only contains data—possibly denormalized data—in a format that matches the expectations of the user interface. For example, if you’re going to display pending orders, you might want to have a query database just for the information you intend to display for orders. In doing so, you don’t store the customer ID or product ID, just the real name of the customer, a full description of the product, and a calculated total of the order that includes taxes and shipping costs.

Stale data and eventual consistency

When each stack in CQRS owns its database, the command database captures the state of the system and applies the “official” data model of the business domain. The query database operates as a cache for data readymade for the presentation’s needs. Keeping the two databases in sync incurs a cost. Sometimes this cost can be postponed; sometimes not.

If you update the query database at the end of a command, just before the end of the saga or at any point of the workflow when it makes sense, you automatically keep the command and query databases in sync. Your presentation layer will constantly consume fresh data.

Another approach that is sometimes used in the concrete implementations consists of delaying the synchronization between the command and query databases. The command database is regularly updated during the execution of the command so that the state of the application is consistent. Those changes, though, are not replicated immediately to the query side. This typically happens for performance reasons and to try to keep scalability at the maximum.

When the query and command databases are not in sync, the presentation layer might show stale data and the consistency of the entire system is partial. This is called eventual consistency—at some point, the databases are consistent, but consistency is not guaranteed all the time.

Is working with stale data a problem? It depends.

First and foremost, there should be a reason for having stale data; often, the reason is to speed up the write action to gain greater scalability. If scalability is not a concern, there’s probably no reason for stale data and eventual consistency. Beyond that, we believe that very few applications can’t afford displaying stale data for a short amount of time (with “short amount of time” being defined by the context). In many write-intensive systems, writes are sometimes performed in the back end and only simulated on the presentation to give the illusion of full synchronicity. The canonical example is when you post on a social network and the following two things happen:

Image The back end of the system is updated through a command.

Image At the end of the command, the DOM of the page is updated via JavaScript with the changes.

In other words, what appears as the real status of the system is simply the effect of a client-side command. At some point, however, in a few seconds (or minutes or days) something happens that restores full consistency so that when the page is displayed from scratch reading from the server, it presents aligned data.

Eventual consistency is commonly achieved through scheduled jobs that run periodically or via queues that operate asynchronously. In this case, the saga ends by placing an event to a separate bus, possibly on a persistent queue. The queue will then serve events one at a time, causing the handler to fire and update the query database.

Summary

A decade ago, DDD started an irreversible, yet slow, process that is progressively changing the way many approach software architecture and development. In particular, DDD had the positive effect of making it clear that a deep understanding of the domain is key. And above everything else, DDD provided the tools for that: a ubiquitous language and bounded context.

Initially, DDD pushed the layered architecture with an object-oriented model as the recommended architecture for a bounded context. Years of experience suggested that a single domain model to handle all aspects of the business logic—specifically, commands and queries—was probably taking the complexity up to a vertical asymptote. From that point, there was an effort in the industry to find a different approach—one that could retain the good parts of DDD while making the implementation simpler and more effective.

CQRS was one of the answers.

CQRS propounds the separation between domain layers and the use of distinct models for reading and writing—a brilliantly simple idea that, however, had a far broader scope than first imagined. CQRS is perfect for highly collaborative systems, but it also can serve well in simpler scenarios if you apply it in a lighter form. That’s a fundamental point and—as we see it—also the source of a lot of confusion.

CQRS is primarily about separating the stack of commands from the stack of queries. Realizing this architectural separation is not necessarily complex or expensive. However, if you take CQRS one or two steps further, it can really deliver a lot of added value in terms of the ability to manage much more business complexity while making your system much easier to scale out. To achieve this, you need to restructure and largely rethink the command and query sides and introduce LET, command buses, handlers, domain and integration events, and (likely) distinct storage for reads and writes. This flavor of architecture is ideal for collaborative systems, but it can be expensive to use in some systems. In the end, full-blown CQRS is a good solution, but not for every problem.

Finishing with a smile

To generate a smile or two at the end of this chapter, we selected three popular but largely anonymous quotes in the spirit of Murphy’s laws:

Image To understand what recursion is, you must first understand recursion.

Image Before software can be reusable, it first has to be usable.

Image Ninety percent of everything is CRUD.

In addition, we saved two funny but deep pearls of wisdom for last, from two people who contributed a lot to the software industry and computer science as a whole:

Image If debugging is the process of removing software bugs, then programming must be the process of putting them in. (E. W. Diijkstra)

Image C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do, it blows away your whole leg. (Bjarne Stroustrup)