Architects and architecture today - Foundation - Microsoft .NET Architecting Applications for the Enterprise, Second Edition (2014)

Microsoft .NET Architecting Applications for the Enterprise, Second Edition (2014)

Part I: Foundation

CHAPTER 1 Architects and architecture today

CHAPTER 2 Designing for success

CHAPTER 3 Principles of software design

CHAPTER 4 Writing software of quality

Chapter 1. Architects and architecture today

The purpose of software engineering is to control complexity, not to create it.

—Dr. Pamela Zave

At the very beginning of the computing age, the costs of hardware were largely predominant over the costs of software. Some decades later, we look around and find the situation to be radically different. The industry made incredible progress, and hardware costs have fallen dramatically. Software development costs, on the other hand, have risen considerably, mostly because of the increasing complexity of custom enterprise software development.

This situation has created the need for a set of precepts to guide engineers in the design of such systems. Appropriated from the construction industry, the term architecture has become the common way to describe the art of planning, designing, and implementing software-intensive systems. When the two of us were teenagers the “Love Is” comic strip (http://www.loveiscartoon.com) was at its peak of popularity. Each installment contained a piece of teenage wisdom about love. One cartoon said something like, “Love is a necessity, not a luxury.” Well, that’s precisely the point with architecture in software.

In this first chapter, we try to share our vision of architecture and how you figure it out and then implement it. In doing so, we’ll touch on the role that architects play in the process and the basic facts we’ve seen ruling the mechanics of software projects. Needless to say, our experience is just ours. Although we’ve been working on some interesting projects of various sizes, our experience is still limited to what we’ve seen and the mistakes we’ve made. Nonetheless, we like to think of this book as the excuse for you to start a longer and more focused conversation within your team about the way you actually build things that work.


Image Note

Although some definitions you find in this book come from international standards, others reflect our personal opinions, experiences, and feelings. We will cover the well-recognized best practices on architecture, but we will also comment from own experience. We hope this combination will help you relate the, sometime dry, international standards to what you will find in the real world.


What’s software architecture, anyway?

One of this book’s authors had, in the past, frequent interaction with an architecture studio. One day, a question popped up for discussion: What’s architecture? Is it an art? Or is it just building for a client?

In software, the term architecture precisely refers to building a system for a client. That’s it: no more and no less.

This said, finding a more detailed description of what’s involved in building a system for a client is the toughest part. We often wonder whether there’s really a point in it. A million times we ask ourselves this question, and a million times we find the same answer. It’s hard, but let’s give it a try. At worst, it ends up being a brainstorming session, and brainstorming is never a bad thing.

So let’s try to clarify what “software architecture” is or, at least, what we intend it to be.


Image Note

The term architecture goes hand in hand with the term architect. Architect is a role, and we’ll discuss later in the chapter the responsibilities we associate with the role of an architect. However, the term architect still makes people think of a profession. Unfortunately, the professional figure behind the term is not universal. That title is qualified by adjectives such as enterprise, solution, security, and so forth. Unless we specify otherwise, we mean the professional figure sometimes described as a software architect or solution architect.


Applying architectural principles to software

Many of us grew up as software experts first thinking that software is somehow related to the world of construction, and later fighting that notion. The analogy was first used in the software industry to express the need to plan and design before building computer programs. However, a fundamental difference exists between designing and building habitable structures and designing and building usable software systems.

Software exists to automate business processes and human actions; civil architecture artifacts exist as a service to the community. Usually, civil architecture services are planned and realized with best intentions and to be useful to the greatest number of people. Their aim isn’t to satisfy just a few stakeholders. In addition, civil architecture artifacts are costly and complex enough that nobody ever thinks of making variations to the approved project on the fly. Or, at least, variations occur only occasionally.

Things go differently in software.

Software is typically created for a small group of stakeholders; some of them pay from their own pocket to get something that helps their organization function better. Requirements, therefore, are continuously refined, added, removed, and reprioritized. This approach to software requires agility and doesn’t wed well with the idea of a big upfront design like you have in civil architecture.

In a nutshell, today the architectural parallelism between construction and software is not as close as it was a couple of decades ago. However, many dictionaries still list a software-related definition of the term “architecture.” And software architecture is described as “the composition, integration, and interaction of components within a computer system.” It is certainly a definition everybody would agree on. But, in our opinion, it is rather generic and abstract.

Hence, it doesn’t work.

We think that software professionals should agree on a more detailed explanation that breaks down that definition into smaller pieces and puts them into context.

Defining the architecture from a standard viewpoint

Many seem to forget that a standard definition for software architecture exists. More precisely, it’s been around since September 2000 when the American National Standards Institute/Institute of Electrical and Electronics Engineers (ANSI/IEEE) standard 1471, “Recommended practice for architectural description of software-intensive systems” was released. Over the years, the document also became International Organization for Standards/International Electrotechnical Commission (ISO/IEC) standard 42010. Those interested in reading the full standard can link to the following URL: http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=45991.

Software architecture is concerned with the organization of any software-intensive system that solves a problem and achieves its mission in full respect of stakeholders’ concerns. In extreme synthesis, this is the official definition of the term software architecture you get from international standards papers.

Stakeholders are defined as all individuals interested or concerned about the building of the system. The list includes the builders of the system (architects, developers, testers) as well as the acquirer, end users, analysts, auditors, and chief information officers (CIOs).

Concerns are defined as any interests that stakeholders can have about the system and the influence they exercise on the system, whether developmental, technological, operational, organizational, economic, or legal.


Image Note

We know that a lot of people get scared when they run across acronyms like ISO, ANSI, and the like. We also tend sometimes to skip these references because they, well, just sound boring and mindful of the abundant doses of theory we received at university. Anyway, standards papers are full of information we just agree with. In this chapter, we reference standards mostly to say that in some areas (such as, processing user requirements) a lot has been done and it has been done well enough to reuse it.


Diagram of a software-intensive system

Figure 1-1 summarizes the core part of the official diagram that describes software architecture according to papers recognized by the ISO, IEC, and IEEE.

Image

FIGURE 1-1 A diagram summarizing the software architecture viewpoint from ISO, IEC, and IEEE.

A system lives in a context, and this context influences the design of the system by driving some developmental and operational decisions. Although current standards envision the system as a composition of interconnected components, architecture also establishes some firm points that are hard to modify later. In a way, expressing software development in terms of architecture boils down to making some key decisions that affect the development life cycle and, ultimately, the quality of the resulting system.

Our very own blend of software architecture

When new people join our companies, we offer them—regardless of skills and roles—our personal view of software architecture.

We’re Italian, and in Italy we’re rigidly serious about coffee. When it comes to an espresso, there are not that many different types of coffee. The only recognized categorization is between a good espresso and a not-so-good espresso. But when we enter Starbucks, like everybody else, we have our own favorite blend of coffee.

We like to extend the coffee analogy to software architecture.

As with espresso, in this world we see only good architecture and not-so-good architecture. To keep with the analogy, we’re restricted to just architecture that results in a working system. We don’t consider here bad architecture resulting in failing systems, just like we don’t drink bad espresso! When served bad espresso, an Italian doesn’t complain, he just walks out of the bar and shares the appropriate (and respected) feedback.

When it comes to laying out architecture, just like we do at Starbucks, we have our own vision of it. Our vision can be depicted by a plain graph with nodes and arcs, but we feel it expresses how we think and how we work. (See Figure 1-2.)

Image

FIGURE 1-2 Our essential vision of software architecture.

Acknowledging requirements

The mission of the system is expressed through a set of requirements. These requirements ultimately drive the system’s architecture.

In rather abstract terms, a requirement is a characteristic of the system that can be either functional or nonfunctional. A functional requirement refers to a behavior the system must supply to fulfill a given scenario. A nonfunctional requirement refers to an attribute of the system explicitly requested by stakeholders.

Are the definitions of functional and nonfunctional requirements something standard and broadly accepted? Actually, an international standard to formalize quality characteristics of software systems has existed since 1991.

A quick look at the ISO/IEC 9126 standard

As a matter of fact, failure to acknowledge requirements is one of the most common causes—often, the primary cause—that leads straight to the failure of software projects. The ISO/IEC 9126 standard defines a general set of quality characteristics required in software products. The standard identifies six families of quality characteristics, articulated in 21 subcharacteristics. The main families are functionality, reliability, usability, efficiency, maintainability, and portability. Table 1-1 explains them in more detail and lists the main subcharacteristics associated with each.

Image

TABLE 1-1. Families of quality characteristics according to ISO/IEC 9126

Subcharacteristics are of two types: external and internal. An external characteristic is user oriented and refers to an external view of the system. An internal characteristic is system oriented and refers to an internal view of the system. External characteristics identify functional requirements; internal characteristics identify nonfunctional requirements.

The newer ISO/IEC 25010 standard supersedes ISO/IEC 9126 and was issued in March 2011. ISO 25010 has eight product-quality characteristics and 31 subcharacteristics.

Functional requirements

A functional requirement defines a required function in a software system. A function is described in terms of input, behavior, and output. The main issue of functional requirements is the description of the expected behavior. Unfortunately, the description often isn’t as clear and understandable as it should be.

In the ISO/IEC 9126 document, functional requirements are described as the capability of software to provide functions that meet stated and implied needs when the software is used under specified conditions. Software architecture depicts the what; software design depicts the how. However, we believe we too often forget about the third axis—the when.

You might remember the Ariane 5 disaster back in 1996. In brief, during its maiden flight, the Ariane 5 rocket crashed 40 seconds after takeoff. According to the official report, the rocket self-destroyed as the result of a chain-reaction triggered by an unhandled exception—overflow in a number conversion. The unhandled exception bubbled up in an uncontrolled way to presenting incongruent orbital numbers to the self-destroy module. Correctly interpreted as evidence of the rocket being irreversibly lost in space, self-destruction was, in the end, just a savvy choice. The irony was that the module where the exception went unhandled was not supposed to be running, because it was unnecessary, at the time of takeoff. So engineers knew that incongruent numbers could be received during takeoff, but instead of trapping exceptions, they just assumed it would never be necessary because the module would not function at takeoff!

Therefore, the when dimension of requirements does matter.

How should your Internet-based client application behave in the case of poor connectivity? Should it fail? Should it try to wait and recover? Should it be nice to users? Should it use cached data and switch silently to offline mode? There’s no obvious answer; it all depends on the requirements that fall under attributes like reliability and recoverability. It’s your responsibility as an architect to find answers—sometimes asking more questions in a second and more thoughtful pass. Having those answers leads to a better understanding of the system and, subsequently, reduces the likelihood of doing the wrong thing or not coding a required feature.

Ideally, all functional requirements are gathered before development starts and never modified once development has started. More likely, though, many functional requirements are discovered or fully understood only during development sprints. This creates the need for revisiting the implementation and sometimes the architecture of modules and, in tough cases, even subsystems. Failing to do this, starts the notorious “Big Ball of Mud” syndrome we discuss in the next chapter.


Image Important

In the beginning of our careers, we were more prone to mistaking for gold whatever came out of interviews with customers. Like a funny Dilbert cartoon, it was like we were blaming ourselves for not being able to fully deal with requirements, no matter how numerous, disorganized, contradictory, and partial they were. (By the way, check out http://dilbert.com/strips/comic/2001-04-14.) “The customer is always right” was our motto, and customer requirements were like the Table of Law. Over the years, we learned that the customer always has the final word on any functional decisions, but it’s our responsibility to prospect for a sound range of options. And we build the range of options out of successfully acknowledging raw requirements we get from interviews.


Nonfunctional requirements

A nonfunctional requirement refers to an attribute of the system explicitly requested by stakeholders. Typical attributes are things like scalability, security, or perhaps accessibility. While functional requirements are mostly about code and implementation, nonfunctional requirements are mostly about architectural choices and, subsequently, aspects of the system that are hard to change later. Nonfunctional requirements can affect technologies used in a project as well as technical solutions.

For example, how would you address the need for extreme scalability?


Image Note

We have still not met a customer who doesn’t claim that extreme scalability of the contact-list application is vital for the survival of the company and the health of investors’ portfolios. Regardless of what’s actually being said, and sometimes done, our two cents are that true extreme scalability needs are quite uncommon. Unless, of course, you are really building the next Twitter or Facebook!


Scalability is the ability of a software system to provide the same performance as the number of connected users grows, thus generating a lot more work and traffic. For a system to scale well, reads and writes must occur in the shortest possible amount of time. This could mean, for example, using caching extensively in reading and making use of asynchronous writing.

Similarly, security can be effectively addressed by designing the system so that critical modules can be deployed to isolated machines. Accessibility is best addressed if you choose ASP.NET MVC rather than, say, ASP.NET Web Forms for the implementation of the site. This is because ASP.NET MVC offers a lot more control over the rendered HTML.

In general, nonfunctional requirements must be acknowledged at the same time as functional requirements. The latter produces software specifications; the former helps with implementation strategies and inspires technological choices. Functional and nonfunctional requirements are inherently linked, and both are subject to change continually. Nonfunctional requirements, though, raise the need to make some hard decisions.


Image Note

Several friends who helped us write this book had a comment at this point. While they could agree word for word with the previous section, they wanted to clarify that in many cases functional requirements also impact architectural choices and aspects of the system that are hard to change later. Absolutely!


Gathering requirements

There are tons of books that explain how to gather and write requirements. So you can learn that, for example, a good requirement addresses one and only one thing, is not ambiguously written, can be easily traced back to a business or stakeholder requirement, is not obsolete, and so forth.


Image Note

If you’re looking for a book that explains the theory of software requirements, we recommend Software Requirement Patterns by Stephen Withall (Microsoft Press, 2007).


What happens in practice, though, is that business people describe what they believe they want and developers build what they believe those business people described. No matter what effort you put in to gathering of requirements, there’s always something that was withheld, omitted, forgotten, or just expressed in a way that was clear to some but not to others. At the foundation of this communication issue, there’s the fact that business people and developers use different vocabularies.

We are strong believers that architects and developers should speak the same language as the business people.

Regardless of how, say, a voucher is actually coded, to fully implement it you must understand what it is. Even more importantly, you need to understand what the business people mean by it. As an architect, you should not expect that business people understand your language, which is made of things like database tables, services, and protocols. It’s you, instead, who should make any necessary effort to know and understand the entities that populate the business domain.


Image Important

The point of having technical people speak the same language as business people is the foundation of a key concept we introduce in Chapter 5, “Discovering the domain architecture.” This concept is ubiquitous language, and it is one of pillars of the Domain-Driven Design (DDD) methodology.


How we deal with requirements

A simple but effective practice to process functional requirements is grouping them by categories, as described earlier in Table 1-1. What we usually do is create a Microsoft Office Excel document with one tab for each category. Next we go through the actual list of requirements we gathered from interviews and simply map each to an existing category.

When done, we read it back and stop on all those tabs that are either empty or that have just one or two requirements. If no requirements are found, say, under the attribute of portability, we stop and wonder if we know enough about it and if we asked enough questions. Having no explicitportability requirement might mean that portability is not a requirement, or it could mean we just don’t know about it.

In particular, portability relates to the software’s ability to be used within different environments. Having no such requirements, say, in a service back end should immediately raise the question of whether the back end should be usable only from web clients or whether support also for iOS and Android clients is essential.

What’s architecture and what’s not

When you think about creating or defining the architecture of a software system, you first try to identify a possible collection of interacting components that, all together, accomplish the requested mission. In international standards, there’s no mention for any methodology you can use to decompose the system into more detailed pieces. Let’s say in the first step you a get a conceptual architecture and some different views of it. In a second step, you need to get closer to a functional and physical architecture. How you get there is subjective, even though a top-down approach seems to be a reasonable strategy. You decompose components into smaller and smaller pieces, and from there you start building.

The actual implementation of the breakdown process depends on the methodology selected for the project—the more you are agile, the more the breakdown process is iterative and articulated in smaller and more frequent steps. The output of the breakdown process is a set of specifications for the development team. Also, the content and format of the specifications depend on the methodology. The more you are agile, the more freedom and independence you leave to developers when implementing the architecture.

Defining the borderline between architecture and implementation

The constituent components you identified while breaking down the system represent logical functions to be implemented in some way. The design of components, their interface, their responsibilities, and their behavior are definitely part of the architecture. There’s a border, though, that physically separates architecture from implementation.

This border is important to identify because, to a large extent, it helps to define roles on a development team. In particular, it marks the boundary between architects and developers. Over the years, we learned that architects and developers are not different, say, types of fruit, like apples and oranges. They are the same type of fruit. However, if they are apples, they are like red apples and green apples—distinct flavors, but not a different type of fruit. And neither flavor is necessarily tastier.

The border between architecture and implementation is recognized when you reach a black box of behavior. A black box of behavior is just a piece of functionality that can be easily replaced or refactored without significant regression and with zero or low impact on the rest of the architecture. What’s above a black box of behavior is likely to have architectural relevance and might require you to make a hard-to-change decision.

So what’s our definition of good architecture? It is architecture in which all hard-to-change decisions turn out to be right.

The science of hard decisions

There are aspects and features of a software system that are hard (just hard, not impossible) to change once you enter the course of development. And there are aspects and features that can be changed at any time without a huge effort and without having a significant impact on the system.

In his book Patterns of Enterprise Application Architecture (Addison-Wesley, 2002), Martin Fowler puts it quite simply:

If you find that something is easier to change than you once thought, then it’s no longer architectural. In the end architecture boils down to the important stuff—whatever that is.

To sum it up, we think that under the umbrella of the term architecture falls everything you must take seriously at quite an early stage of the project. Architecture is ultimately about determining the key decisions to make correctly as early as possible in the project, but wishing you could make them as late as possible.

Hard decisions are everywhere

When we talk about hard architectural decisions, we are not necessarily referring to decisions about design points that can be difficult and expensive to change later. Hard-to-change decisions are everywhere and range from the definition of conceptual layers to the signature of a constructor.

To illustrate the point, let’s go through a few examples of architectural points that can run into budget limits and deadlines if you have to touch them in the course of the project.

The first example is changing the organization of the business logic. There are several approaches for the design of the business logic: Transaction Script, Table Module, Domain Model, Domain Model with separation between commands and queries, and Event Sourcing, just to name a few. Once you have opted for, say, Table Module (which means, essentially, that you’ll be putting logic in repository-like components built around database tables like you probably did for years around DataSets and DataTables), moving to a domain model is hardly something you can do in an afternoon. That change of direction requires nontrivial changes in the data access layer and in the application (service) layer, and probably also in the presentation layer.

A second example is switching to a different library for the same task. Suppose you developed some functionality around a given library. One day, the client pops up and lets you know that a new company policy prevents the IT department from buying products from a given vendor. Now you have a new, unexpected nonfunctional requirement to deal with, but at what cost? In the best case, you can get a similar tool from an authorized vendor or, perhaps, you can build a similar tool yourself. Alternatively, you can consider introducing a radical change into the architecture that makes that library unnecessary.


Image Note

If you attended some of the conferences Dino spoke at in the past couple of years, you may have heard about this experience already. In short, Dino’s company bid for building a time-sensitive mobile app for four distinct platforms: iOS, Android, Windows Phone, and BlackBerry. The bid was made on the assumption that PhoneGap would be a feasible option. Unfortunately, a prototype built with PhoneGap was rejected by the customer, bringing the team back to square one and with four fewer weeks to get the job done.

The team was left with one of two unpleasant options: doing the work four times in a fraction of the time or inventing something new. The team managed to reuse some of the PhoneGap markup on a mobile site and incorporated web views in the frames of four basic, mostly static and graphic-only native apps. Function-wise, it worked beautifully, and there was even time to survive one silly reject of the app by Apple. Stress-wise, though, it was a pain in the proverbial rear end.


A third, and maybe unusual, example of hard decisions is changing a class member’s modifier. In general, when you use the sealed and virtual modifiers, you take on a not-so-small responsibility. In C#, by default each class is unsealed and each method on a class is nonvirtual. In Java, instead, things go differently for methods, which are all virtual by default. From a design perspective, sealed classes are preferable. In fact, when a class is sealed from the beginning you know it—and you create your code accordingly. If something happens later to justify inheritance of that class, you can change it to unsealed without breaking changes and without compromising compatibility. Nearly the same can be said for virtual methods and the visibility of classes and class members, which are always private by default. The opposite approach doesn’t work as smoothly. You often can’t seal a class or mark a virtual method as non-virtual without potentially breaking some existing code.

Context makes decisions hard

About four years ago, during a conference, we heard a prominent speaker say a great piece of truth for that moment in time. He proclaimed that mobile and cloud computing were the biggest headache for CTOs as they try to figure out what to do with both.

This little memory introduces a painful type of architectural hard decision. It happens when you know that a given piece of technology, or perhaps a pattern, can be helpful but you have no concrete evidence for that. You look around for case studies and eagerly hope to find them, or just any sort of literature that can help you rule in something or rule out something.

Decisions on well-known best practices are easy to make because there are lots of case studies. However, making decisions about new technologies, like NoSQL, is really rather difficult because case studies of real-life applications are few and far between and sometimes even biased.

The architecture process

Processed by the chief architect, requirements are communicated to the development team and then implemented. Everybody would agree that good architecture and good design speed up the actual development of modules. But how would you actually go about building applications?

In principle, building software is the joint effort of three groups of people: project management, development, and testing/quality assurance (QA). The splitting of tasks is nearly perfect in theory. Project management is responsible for product specifications, the definition of quality standards, and schedule. Development ensures that ideal algorithms are used and code is written according to the most appropriate patterns. Finally, the QA team stresses the behavior of the application, with the declared purpose of breaking it so that it can be updated and made stronger.

The three teams must work together at any point in the process, although some project management needs to be done before development begins and testing can’t start until some code has been built. The whole point of software development is defining the engagement rules for the teams and defining how they should interact. A software development methodology attempts to do this.

Methodologies fall essentially into two main types: Waterfall and Agile. Waterfall-based projects tend to proceed sequentially across a number of phases, culminating with the release of the software. Agile-based projects tend to be circular and iterate through tasks several times until the application is ready or, sometimes, until the deadline is looming. Choosing the methodology, in a way, determines the type of architectural process.

Upfront architecture

Upfront architecture is when everything must be set in stone before you start. A lot of design work is done before starting to write code. Coding is mostly seen as the actual translation of well-defined ideas into the compiling of instructions.

The Waterfall model dates back to the 1970s. It is a model in which software development proceeds from one phase to the next in a purely sequential manner. Essentially, you move to step N+1 only when step N is 100 percent complete and all is perfect with it. Figure 1-3 shows a sample of the Waterfall model.

Image

FIGURE 1-3 The Waterfall model

In the real world, some overlap between phases is acceptable, but the emphasis remains on making plans, creating budgets, and arranging the work schedule at one time. This approach is great for clients because it gives them a number of apparent certainties. The reality is different, however. It is not uncommon that projects end up running over the budget and the deadline and sometimes even fail to deliver on expected requirements. This poor result is largely because of the big design effort that is required up front, which just doesn’t get along with the actual dynamics of modern software.

The root problem of software design is that requirements change quickly over time to the point that some requirements might be different by the time development ends. No clients are willing to pay for software that they know is not going to satisfy all requirements. At the same time, clients don’t like at all to be subject to variable costs that might grow as changes and additions are applied to the artifact.

As a result, the Waterfall model is considered a thing of the past, and you can blame its demise on the whole idea that software development is a form of engineering.

Emerging architecture

Today’s view of the architecture process is that any team should start developing as soon as possible to get early feedback and improve the software based on the real code. This means moving things fast, accepting and even embracing change, delivering some value early, and welcoming feedback.

Emerging architecture is a process that results from the incremental building of the software. After the initial startup, the project goes through a series of iterations that include design, coding, and testing. Every iteration produces a deliverable but incomplete version of the system. At each iteration, the team enters design changes and adds new functions until the full set of specifications are met. Figure 1-4 provides a graphical view of the iterative process.

Image

FIGURE 1-4 The iterative model

Iterative development forms the foundation of Agile methodologies. The term Agile was deliberately selected to symbolize a clear opposition to heavyweight methods such as the Waterfall model. The principles behind Agile methods are listed in the “Agile Manifesto,” which was first published in 2001 and which you can find at http://agilemanifesto.org.

When an Agile project starts, only a few requirements might be fully defined, but you know for a fact that many more will show up or need to be clarified between this point and the end of the project. With an Agile mindset, this is not an issue. Development is articulated in iterations. At the start of an iteration, you negotiate with the customer the subset of the existing requirements that should be implemented. During the iteration, you focus on a single requirement at a time and implement it. At the end of the iteration, you deliver a working piece of software. It might be incomplete, but it works. Next you go with another iteration that focuses on another set of requirements. If something changed in the meantime or proved to be wrong, refactoring is in order. And the process continues until there’s nothing more to add. The length of an iteration is measured in weeks—often, two weeks. In a word, an agile process is agile enough to react to changes. And changes in the business are the rule, not the exception.

Agile methodology is a blanket term. When you refer to an Agile methodology, you aren’t talking very precisely. Which methodology do you mean, actually?

The most popular Agile methodology for software development is Extreme Programming (XP). In XP, phases are carried out in extremely short iterations that take two weeks to terminate. Coding and design proceed side by side. For more information on XP, visithttp://www.extremeprogramming.org.

Scrum is another popular Agile methodology, but it is more aimed at managing projects rather than just developing code. Scrum is not prescriptive for any software development model, but it works very well with XP as a method for developing code. For more information on Scrum, have a look at Agile Project Management with Scrum by Ken Schwaber (Microsoft Press, 2004).


Image Important

Agile architecture is sometimes presented as an oxymoron, like saying that if you’re agile you don’t do any architecture analysis, you just start coding, focus on functionality, and completely ignore modeling. Honestly, our take is that although Agile doesn’t preclude any such behavior, this vision is mostly mythical. In our experience, Agile mostly means just the opposite and that architecture and modeling concerns are addressed throughout the entire development life cycle.


Emerging architecture with upfront analysis

It may be worth mentioning a third option that sits somewhere in the middle of Waterfall and Agile. It is a process that comprises a good deal of preliminary upfront analysis and then starts with a classic Agile methodology. Some call this a very disciplined Agile approach; some also refer it to as a sprint zero thing.

More generally, though, we find that all software development methodologies share a few common attributes: a number of phases to go through, a number of iterations to produce the software, and a typical duration for a single iteration. All phases execute sequentially, and there’s always at least one iteration that ends with the delivery of the software. The difference between methodologies is all in the order in which phases are entered, the number of iterations required, and the duration of single iterations. After accepting this premise, the step to adopting Agile methods is much smaller than you might think at first.

Who’s the architect, anyway?

As you’ve seen, architecture is mostly about hard-to-change decisions. And someone has to make these decisions.

The design of the architecture is based on the analysis of the requirements. Analysis determines what the system is expected to do; architecture determines how to do that. And someone has to examine the whats to determine the hows.

The architect is the professional tying together requirements and specifications. But what are the responsibilities of an architect? And what are the required skills?

An architect’s responsibilities

According to the ISO/IEC 42010 standard, an architect is the person, team, or organization responsible for the system’s architecture. The architect interacts with analysts and the project manager, evaluates and suggests options for the system, and coordinates a team of developers.

The architect participates in all phases of the development process, including the analysis of requirements and the architecture’s design, implementation, testing, integration, and deployment.

Let’s expand on the primary responsibilities of an architect: acknowledging the requirements, breaking the system down into smaller subsystems, identifying and evaluating technologies, and formulating specifications.

Acknowledging the requirements

In a software project, a few things happen before the architect gets involved. Swarms of analysts, IT managers, and executives meet, discuss, evaluate, and negotiate. Once the need for a new or updated system is assessed and the budget is found, analysts start eliciting requirements, which are typically based on their own knowledge of the business, company processes, the context, and feedback from end users.

When the list of requirements is ready, in many cases the project manager meets with the architect and delivers the bundle, saying more or less, “This is what we (think we) want; now you build it.”

The architect acknowledges the requirements and makes an effort to have them adopted and fulfilled in the design.


Image Important

A moment ago, we mentioned yet another role: the project manager. The project manager is a role that may be defined differently in different companies. We think of it as the role responsible for deciding on the methodology, scheduling work, tracking, reporting, and serving as a valuable link between technical and business people. This role can be played by the same individual who plays the role of the architect. When this happens—and it is not unusual—requirements flow directly from the domain experts to the development team. If other people are in the middle, the risk we see is that domain knowledge gets contaminated, like that children’s game where one child whispers a phrase into another child’s ear. The second kid talks to a third kid and so on, until the sense of the original phrase is irreversibly lost. It is critical that requirements expressed in Ubiquitous Language flow from domain experts to the development team with no middlemen or just in a pass-through layer.


Breaking down the system

Based on the requirements, the architect expresses the overall system as a composition of smaller subsystems and components operating within processes. In doing so, the architect envisions logical layers, services, or both. Then, based on the context, the architect decides about the interface of layers, their relationships to other layers, and the level of service orientation the system requires.


Image Note

At this stage, the architect evaluates various architectural patterns. Layering is a common choice and the one we are mostly pursuing in this book. Layering entails a vertical distribution of functionality. Partitioning is another approach where all parts are at the same logical level and scattered around some shared entities—such as an object model or a database. Service-Oriented Architecture (SOA) and Hexagonal Architecture (HA) are patterns that tend to have components (services in SOA, adapters in HA) operating and interacting at the same logical level. Micro-services is another recent architectural pattern centered on the idea of specialized and isolated components.


The overall design will be consistent with the enterprise goals and requirements. In particular, the overall design will be driven by requirements; it will not lead requirements.

The resulting architecture is ideally inspired by general guidelines, such as minimizing the coupling between components, providing the highest possible level of cohesion within components, and giving each component a clear set of responsibilities.

The resulting architecture is also driven by nonfunctional requirements, such as security, scalability, and technologies allowed or denied. All these aspects introduce further constraints and, to some extent, delimit the space where the architect can look for solutions.

Finally, the architect also strategizes about tasking individual developers, or teams of developers, with each of the components based on the layout of the system.


Image Important

There are no absolute truths in software architecture. And no mathematical rules (or building codes like in structural engineering) to help in making choices. Company X might find architecture A successful at the same time company Y is moving away from it to embrace architecture B. The nice fact is that both might be totally right. The context is king, and so is gut feeling.


Identifying and evaluating technologies

After acknowledging requirements and designing the layers of the system, the next step for the architect entails mapping logical components onto concrete technologies and products.

The architect typically knows the costs and benefits of products and technologies that might be related to the content of the project. The architect proposes the use of any technologies and products that he reckons will be beneficial and cost effective for the project.

The architect doesn’t choose the technology; based on his skills, the architect just makes proposals.

The architect might suggest using, say, Microsoft SQL Server 2014 because of its new clustered column store indexes or perhaps opting for Single-Page web application supported by an ASP.NET Web API back end. Similarly, the architect might raise the point of using a local NoSQL document store instead of some cloud-based table storage.

Who does make the final decision about which technologies and products are to be used?

Typically, it is the project manager or whomever manages the budget. The architect’s suggestions might be accepted or rejected. If a suggestion is rejected, using or not using a given product or technology just becomes a new nonfunctional requirement to fulfill, and that might influence, even significantly, the architecture.

Formulating specifications

The architect is ultimately responsible for the development of the system and coordinates the work of a team of developers. Technical specifications are the means by which the architect communicates architectural decisions to the developers.

Specifications can be rendered in various forms: UML sketches, Microsoft Word documents, Microsoft Visio diagrams, or even working prototypes.

Communication is key for an architect. Communication happens between the architect and developers, and it also happens between architects and project managers and analysts, if not users. A great attribute for an architect to have is clarity of language.

The interaction between architects and developers will vary depending on the methodology chosen. And also the involvement of project managers, analysts, and users varies based, essentially, on the level of agility you accept.

The role of the architect

Architecture implies a role that is generically referred to as “the architect.” According to ISO/IEC, there are not various types of architects. An architect is an architect. Period.

However, if you look around (and examine resumes), you see quite a few definitions of architect. In the end, it’s an overused word, and it really means very different things depending on the context, the company, or even the country.

How many types of architects do you know?

In the United States, an enterprise architect (EA) has almost nothing to do with application development because the person in that role focuses for, say, 90 percent of the time on IT-related business strategy, orchestration, and infrastructure. In an extreme synthesis, EA is the figure that weds business and IT. A candidate for this role is expected to know very little about software architecture subjects and probably nothing about things like layered architecture or DDD.

In many companies, the role responsible for hard decisions, and in charge of proposing approaches and technologies, is not even labeled as an “architect” but gets the title of lead developer or something similar.

So you can find labels such as enterprise architect, infrastructure architect (IA), technology-specific architect (TSA), and even solution architect (SA). All these distinctions are kind of misleading because they attempt to break into parts what is ultimately an atomic, yet complex, role. In our opinion, it creates unnecessary categorization and lays the groundwork for confusing, who-does-what scenarios.

In this book, we go with the ISO/IEC definition of an architect, which is the “person, team, or organization responsible for the system’s architecture.” When mapping this concept to the widest possible range of companies, it turns out that what we call architect is a software (or solution) architect or even a lead developer.

Architect roles

You might have noticed it already, but if you go to Microsoft TechEd you see that the Architecture track has almost no sessions about real issues related to software development and architecture. For this reason, all the gazillion DDD sessions we submitted in years past have been regularly rejected. Among the Microsoft TechEd events staff, architect mostly means a role concerned with enterprise architecture. And all DDD sessions at Microsoft TechEd are under the Development track!

It is fine to have multiple architects on the same project team. Likewise, it is fine, if not desirable, that different architects have slightly different skills. However, as we intend it in this book, and regardless of the official titles, architects have significant exposure to code. They work out the design of the system but then work closely with developers to ensure proper implementation.

As we see things, an architect is, among other things, a better and more experienced developer. We don’t believe there’s value in having architects who just speak in UML and Visio and leave any implementation details to developers. At least, we’ve never found it easy to work with these people when we’ve crossed paths with them.

Common misconceptions about architects

Mostly as a result of the various meanings associated with term architect, a set of misconceptions has grown out of the mass of personalized definitions and interpretations. Let’s go through a few of them and, we hope, clear up a few.

The architect is an analyst

This is a false statement. An architect is simply not an analyst.

At times, an architect might assist analysts during elicitations to help clarify obscure requirements or smooth out weird and fancy requirements added to the list just for “completeness.” At times, an architect might participate in meetings with stakeholders. But that’s it.

In general, an analyst is a person who is an expert on the domain. An architect is not (necessarily) such an expert. An analyst shares with an architect his own findings about how the system should work and what the system should do.

This common misconception probably originates from the incorrect meaning attributed to the word analyst. If the word simply indicates someone who does some analysis on a system, it is quite hard to deny the similarities between architects and analysts. Some 30 years ago, the term system analyst was used to indicate a professional capable of making design considerations for a system. But, at the time, software wasn’t as relevant as it is today; it was merely a (small) part of an essentially hardware-based system.

Today, the roles of an analyst and an architect are commonly recognized as being different. And hardly ever does an architect play the role of an analyst.


Image Note

Given that roles are not always neatly separated, especially in small companies, it can happen that the same person serves as an analyst and architect. This simply means there’s a person in the company who knows the business and processes well enough to come up with functional requirements and translate them into specifications for developers. The roles and responsibilities are still distinct, but the distinct skills for each are found in the same individual.


The architect is a project manager

Is this another false statement? It depends.

The architect is responsible for the system’s architecture, and that person coordinates and guides the development of the system. The project manager represents stakeholders and manages the project by choosing, in the first place, a methodology. The project manager is then responsible for ensuring that the project adheres to the architecture while proceeding within the limits of the timeline and budget.

If we look at the role of the architect and the role of the project manager, we find that they are distinct. Period.

However, it is not unusual that one actor ends up playing two roles. Like in the theater, this hardly happens in large companies, but it happens quite frequently in small companies.

In summary, if you want to be a software architect when you grow up, you don’t necessarily have to develop project-management skills. If you have skills for both roles, though, you can try to get double pay.

The architect never writes any code

This is definitely an ongoing debate: Should architects write code? There are essentially two schools of thought.

One school thinks that architects live on the upper floor, maybe in an attic. Architects then step down to the developers’ floor just for the time it takes them to illustrate, using diagrams, what they have thought about the system. After this, they take the elevator up, collect their things, and go out to play golf. When on the course, they switch off their cell phones and focus on the game. When done, if they missed a call or two, they call back and explain to dummy developers what was so clear in the diagram that nobody on the developers’ floor could understand. According to this school of thought, architects never, ever dirty their hands with even the simplest C# statement. C#? Oh no, the latest language they’ve been exposed to is probably Pascal while in college and Visual Basic at home.

Another school of thought thinks, instead, that every architect is a born developer. To take the metaphor one step further, we could say that the class Architect inherits from the class Developer and adds some new methods (skills) while overriding (specializing) a few others. Becoming an architect is the natural evolution in the career of some developers. The basic differences between an architect and a developer are experience and education. You get experience by time on the job; you get education from studying good books and taking the right classes. In addition, an architect has the ability to focus her vision of the system from a higher level than an average developer. Furthermore, an architect has good customer-handling skills.

An architect might not write much production code. But she writes a lot of code; she practices with code every day; she knows about programming languages, coding techniques, libraries, products, tools, CTPs, and she uses the latest version of Visual Studio. In certain areas of programming, an architect knows even more than many developers. An architect might be able to write tools and utilities to help developers be more productive. And, more often than you might think at first, the architect is just a member of the development team. For example, an architect writing production code is an absolutely normal occurrence in an Agile context. It is also a normal occurrence in small companies regardless of the methodology. At the same time, having an architect who writes production code might be absolutely weird to see in some large-company scenarios, especially if a traditional and non-Agile methodology is used.

What about the two of us? To which school do we belong?

Well, Andrea is more of an architect than Dino because he works on the fifth floor. Dino, on the other hand, is closer to development because he has quite a few highly technical ASP.NET books on his record and, more importantly, works on the second floor. We don’t play golf, though. Dino plays tennis regularly, whereas Andrea likes squash better. We just have been denied access to the first school of thought.


Image Note

In no other area of engineering is the distinction between those-who-design and those-who-build as poorly accepted as it is in software. The distinction exists mostly through postulation rather than flowing from a public recognition of skills.

The canonical comparison is with civil architecture. Bricklayers have unique skills that engineers lack. No bricklayer, though, will ever dream of questioning designs or calculations simply because they lack the skill to make the decisions themselves. They do their own work the best they can, taking full advantage of having the building work delegated to them.

In software, the situation is different because architects and developers have common roots. The more skilled a developer is, the more he feels encouraged to discuss design choices—and often with reason. The more the architect loses contact with everyday programming, the more he loses the respect of other developers. This generates a sort of vicious circle, which magically becomes better as you switch to an Agile methodology.


Summary

Architecture is a necessity and not a luxury for modern software. Assuming there was ever a time in which architecture was optional, that’s no longer today. And the difference is all in the complexity required by modern software.

It is a common perception that software can be compared to civil engineering; however, when civil engineers set out to build, say, a bridge, a bridge gets built. In addition, the bridge almost always functions correctly and is built fairly close to the original budget. You can hardly say the same for many software projects. When it comes to software, it is sometimes very uncertain what stakeholders will eventually get out of their commitment. It is more certain, instead, that the original budget might be exceeded and that any deliverables are likely to be different, to some extent, from expectations.

Why is it so with software?

Overall, we believe that software development can hardly be subjected to fixed and rigid rules as with other disciplines, such as civil engineering. Software development is not pure engineering because it is contaminated by abundant doses of design, plain creativity, and even psychology. In addition, software has a very dynamic nature because it is relatively slow to build but needs to stay in sync with continuously changing business requirements. Software practices change so rapidly that, at any time, it’s really hard to capture and store the current state of the art.

In this chapter, we focused on software architecture and the architect role in an attempt to capture the essence of both. In the next chapter, we move the focus toward what architects actually do when they try to apply architecture to a problem domain. In the next chapter, we discuss more of the mechanics of software projects and the scary thing that make projects fail—the big ball of mud.

Finishing with a smile

Here’s a humorous twist on some of the topics discussed in this chapter:

Image Adding manpower to a late software project makes it later. (Fred Brooks, “The Mythical Man-Month”, Addison-Wesley, 1995)

Image Nothing ever gets built on schedule or within budget. (http://www.murphys-laws.com)

Image Failure is not an option; it’s included with the software. (http://www.murphys-laws.com)