Pragmatic Enterprise Architecture (2014)
Strategies to Transform Information Systems in the Era of Big Data

Prologue

Whenever I learn something, I often wonder how I must have thought about things prior to knowing what I learned. I don’t know if every reader ponders this, but if you do, try to drop me a note afterward to let me know the difference between your thoughts before and after reading this book.

To begin, I find that information has entropy in a manner similar to that of matter and energy in physics. As such, entropy in information is the randomness of information that occurs naturally. Information is random and becomes more random unless some force imposes order upon the information. Whether or not others realize the importance of doing so often determines whether we make advances in discovering new things. Prior to written language, the sciences moved at an awfully slow pace. Collecting information allows others to share it and to think about it more concretely, in a way, that is what this book is all about.

Few books have been written about enterprise architecture. This could be due to the fact that the few managers who are successful with it do not have sufficient time for the commitment necessary to write a book. As luck would have it, this author has had prior experience writing books and articles and, with a proclivity toward architecture, realized that various articles could be planned out over a number of years to help develop the framework of a useful book. That said, one thing for sure is that this author did not plan on having such an interesting story to write about when it finally came time to creating the book.

There are widely varying perspectives on what Enterprise Architecture is and what it should be, including where it should report into within large organizations, what the qualifications should be to run it, and what authority that individual should have without running the risk of further debilitating an already cumbersome IT organization. Further delaying delivery of software that business needs to either compete in the marketplace or survive the global winds of economic and regulatory change, or just being blamed for further delaying software delivery, is one of the fears that executives face.

This book will address reality by looking at the big picture that is reasonably common to a global company. Most conglomerates are global, and this includes various types of industries, such as financial, retail, manufacturing, energy, health care, or government, such as the U.S. government, which is a global conglomerate itself.

As we all know, some books move slowly, carefully developing and revealing their knowledge to the reader in minute detail, while others move at a rapid pace, thereby covering vast quantities of material within the confines of the front and rear cover. Since your time is valuable, this book is the latter. As such we attempt to provide a framework that encompasses the vast reaches of enterprise architecture, without belaboring the issue.

As for its usefulness, it is perhaps most like the film, “The Matrix,” which is a relatively recent cultural reference where characters can download the ability to extend themselves into a new area or skill. After you read a chapter, you may not be able to fly a helicopter, but this book can get you going in a variety of architectural topics while preparing you with an ample supply of key concepts that are pertinent to the topic to allow someone with a moderate background in IT the ability to dive in and be effective in early conversations. As examples, if the topics were Big Data or Data Center Consolidation Life Cycle (DCCLC), after reading those sections the reader would be equipped to have an intelligent conversation about many of the issues.

Much of the story depicted in this book represents the experience from within a number of U.S. Fortune 50 financial conglomerates that will remain nameless, although the savvy individual would have no trouble deducing their probable identities. Most of these organizations employ an approach to IT that is commonly known as a federated model, where there is the concept of a federal government (corporate) that maintains responsibility for shared services in conjunction with a set of states (lines of business) that maintain responsibility for local services to rapidly meet the needs that are unique to the local business unit.

Within large organizations, there are always areas that most would agree should be centralized, such as the shared services associated with communications, procurement, and human resources, as well as some areas that have clear advantages to being decentralized back out to the lines of business, such as front office operations, distribution channel operations, back office operations, and operational business intelligence capabilities associated with specific areas that rely upon the in-depth knowledge of the business domain.

Similarly, there are also a collection of areas that must participate within the lines of business that must roll up to a counterpart at the corporate level, such as risk management, legal, compliance, audit, regulatory reporting, accounting, brand management, public relations, government relations, and customer relations.

As one will see, understanding the many parts of a large organization is essential for a potentially successful enterprise architecture practice. If this were not the case, then enterprise architecture should be renamed to “IT Architecture,” or in a smaller scope, “Software Development Architecture,” or in its smallest scope, “The Application and Technology Inventory Tracking Department.”

The Main Characters

Keywords

epigenetics

tribal culture

tribal methods

organizational fitness

The main characters of this journey include a collection of individuals who are either on the IT or business side, as well as some that straddle both, such as the new hybrids of “technology savvy business people” and “business savvy IT people” that cross what is often that demilitarized zone that exists between pure business and pure IT.

Among them are characters within Enterprise Architecture for which proper names have not yet been established. We will identify these newly established characters as we address the need for architectural disciplines that must span the company horizontally across geographic locations and lines of business.

That said, there are also several characters that many of us are already familiar with, such as the Chief Information Officer (CIO), Chief Enterprise Architect (CEA), and the heads of Auditing, Legal, Legal Compliance, Regulatory Compliance, HR Compliance, and IT Compliance. A couple of recently emerging ones also include the Chief Data Officer (CDO), which has been created to confront the challenges such as navigating the approval process to source data from disparate lines of business, and the Chief Customer Officer (CCO), which has been established to focus on customer-related strategies, such as providing an overall better customer experience than does the competition.

It is also important to recognize the way that various stakeholders approach their roles.

As examples:

- CIOs tend to think and communicate in terms of expense areas providing automation services, application stacks that support business capabilities and lines of business, as well as strategic objectives and resource consumption relative to meeting those objectives,

- CEAs tend to think and communicate in terms of technology stacks and infrastructure, as well as application development team governance, standards, and data landscape complexity,

- CDOs tend to think and communicate in terms of business intelligence platforms and use cases, as well as data quality and master data management, and

- CCOs tend to think and communicate in terms of customer retention, emerging markets, designing the end-to-end customer experience, and understanding the cause of customer complaints and customer behaviors.

Culture

Keywords

Chief Information Officer (CIO)

Chief Enterprise Architect (CEA)

Chief Data Officer (CDO)

Chief Customer Officer (CCO)

How these characters interact and how they relate to one another is largely a function of the approach of how management organizes reporting structures, but organizational culture is the dominant factor that determines whether and how an organization comes together to work toward greater effectiveness.

Perhaps the single most significant message that the CEA can convey to the executive management team is that enterprise architecture is actually the area that determines the epigenetics of the IT organization. As such, enterprise architecture is best positioned to make the IT organization responsive and efficient in reacting to the business direction of executive management and IT management.

This may appear at first to be a fine line to traverse, but let’s explore the subject of epigenetics to better understand the distinction between the roles of the Chief Information Officer (CIO) and the Chief Enterprise Architect (CEA).

We will begin by making an unusual analogy, which is that the role of the CIO is more like the genes of an organism, and the CEA being more similar to epigenetics.

Epigenetics

Although many have not heard of epigenetics before, it is a cornerstone in science for understanding how designs are expressed for building living things. Unlike a single sheet of paper, called a blueprint, that can express how a house should be built, the way designs of living things are represented is somewhat more dynamic.

For a little background in the field of genetics, DNA carries the instructions for building proteins that are the key components of each living cell and body. Genes are long snippets or chunks of DNA that are the basic units of inheritance representing traits of living organisms that are passed on from parents to children. Genes determine how long a protein should be, how much to make, and when to make it. Genes also determine how to edit DNA strings to make different proteins. This is how the body can make about a million different proteins from merely 20,000 genes.

Just to complete our taxonomy, chromosomes are strands of DNA arranged in pairs wrapped around spherical proteins called histones. Each chromosome strand contains many genes. Humans, for example, have 46 chromosomes.

Genes provide all of the directions for how to build a body and the systems that keep that body healthy. In this sense, genes contain all of the commands for subordinate components to do everything that needs to be done to build and operate the body from an executive management perspective.

We know however that DNA alone does not determine the outcome for building an organism, any more than a general can control the outcome of a military operation by issuing commands to his commanders. The orders may be brilliant, but depending upon the preparedness of the troops, a variety of outcomes are always possible.

When it became obvious that DNA and their genes do not determine the outcome for building each cell of an organism, scientists had to look at more than just the genes. They eventually realized that while all of the genes were present on the DNA, it was the environment that determined which genes would be expressed and which genes would not.

Scientists learned that at the microscale each cell knew whether it should develop into a heart cell, brain cell, taste bud, or any other cell, while at a macroscale they learned that our environment determines what genes are expressed over our lifetime through a series of chemical mechanisms that influence gene expression as we grow up and age.

As a result, genes can be expressed to generate a healthy fully functioning individual or they can be expressed in some suboptimal way as a response to a variety of environmental factors such as stress or a lesser abundance of food or shelter.

Expectedly, when the environment is more conducive to a healthy way of life, genes are expressed differently, and in a better way than when the environment is unhealthy.

Similarly, an IT environment can be determined by the chance, which is what many organizations achieve as the outcome of their activities, or the IT environment itself can be influenced in a positive direction, ultimately improving the ability of the organization to express the commands of its executive management.

In that manner, enterprise architecture works behind the scenes to influence the greater environment of automation. As we will see in this book, over even a short period of time, this influence can help the organization be far more prepared to act effectively on commands from the CIO, and from the head of application development.

First, we will try to understand a little better how an influence over an organization can have such a profound effect.

Degrees of Civilization

Organizations express themselves in certain ways. As groups of individuals increase in size, a culture naturally develops from its individual members. For the most part, immediate families and extended families behave in certain ways. Similarly for the most part, small tribes, larger tribes, and collections of neighboring and perhaps related tribes also behave in certain ways. For example, a leader is usually chosen by each tribe.

Tribal culture can be rather sophisticated with many rules for getting along with other individuals and tribes. In tribal societies, sophisticated dispute resolution often evolves for trading pigs, cows, or women in an attempt to avoid escalating cycles of conflict. But in the end, it is quite different than the culture that results when compared to modern cultures that develop from there being the higher influence of a State. (The World until Yesterday, Jared Diamond, 2012, Penguin Group, ISBN 987-1-101-60600-1)

In short, the influence of a state is what makes it relatively safe for citizens within a society to deal with individuals whom they have never met and do not know. Strangers know that there are police, courts, and judges that exist as part of the state. Civilization as we know it is the product of a higher level influence upon our society.

This includes many quality of life advancements that are available within our society for reasons such as the fact that individuals can easily communicate and conduct commerce with others whom they have never met. In contrast, it is common in tribal societies to chase away or kill anyone not related to the tribe as a way to protect the resources and safety of the tribe.

In this book, we will see how a modest, effectively staffed, organized, and well-led enterprise architecture team can become a higher level influence across an organization. While they are not police, they so have a number of ways to influence the environment.

These include methods, such as:

- frameworks that act as accelerators to get the work that needs to be accomplished done with less effort,

- subject matter expertise to mentor IT staff across the organization at the moment that additional expertise is needed,

- standards that incorporate the interests of organizational stakeholders that otherwise could not monitor each and every application team across a large organization, and

- direction that has been thought through and approved by the CIO for making technology and application decisions.

In short, the role of the modern enterprise architecture organization is to develop the appropriate level of civilization across automation resources to ensure the preparedness of automation resources across the organization to rapidly act on the directions given by the CIO. Tribal methods must give way to a more advanced more cooperative and more intelligent way of reacting to the commands of executives.

In other words, scattering the bones of enterprise architects outside the offices of a tribal line of business or application team should no longer occur, except in only the most extreme circumstances.

Organizational Fitness

In simpler terms, enterprise architects become the fitness leaders of the organization to improve the health and effectiveness of the IT organization. Enterprise architecture shapes the automation personnel into a leaner more effective set of resources that have the right architectural disciplines at their disposal to get the job done. The role of enterprise architecture is to be prepared to meet the evolving needs of the IT organization, while significantly reducing the emergence of IT health-related issues and the overall cost of addressing those issues.

The role of enterprise architecture is surprisingly like that of a physical fitness leader. In fitness, the more difficult thing is to make the time and apply the effort to exercise and to eat properly. Exercising means work. Once one develops a culture that enjoys exercise, then it is not as difficult to get the motivation to exercise.

Eating properly means eating three meals a day and choosing the right things to eat. It may even mean eating more than before, but eating more healthy things to displace the urge and ability to choose unhealthy foods. The worse thing to do is to start skipping meals as a way to lose weight.

At first skipping meals seems to get results. The weight starts coming down and the waist begins to slim down; however, the true effects remain hidden for some time. Instead of improved health, the strategy of skipping meals leads to a cycle of poor health and damage to body structures and systems.

The hidden effects include:

- decrease in bone density,

- loss of muscle mass, and

- damage to organs.

The ultimate logical conclusion of what enterprise architecture is can be best defined as the combination of three things.

They are that which creates the most appropriate:

- environment across IT that will allow it to properly express the commands of executive management,

- degree of civilization that facilitates rapid and effective cooperation across the organization, and

- level of fitness that ensures the long-term health of the IT organization that will allow it the agility to meet the needs of the CIO.

It is the epigenetics of an organization that positively influences the health of the organization by fostering the appropriate enablers to express themselves by intelligently shifting the habits of the organization to exercise their minds and adopt healthy habits. Effective enterprise architecture staff members help automation personnel by mentoring them to get their work done in ways that have healthy long-term effects.

The alternative is an unresponsive bunch of tribes that have bad health habits.

Automation in Perspective

For the most part, while automation itself does not develop the strategies and business environment that make a company more successful as a business, when automation is inefficient and/or ineffective, it can certainly drag a company down. A healthy IT organization is even more important in highly regulated industries where new demands may be placed on IT to deliver information to regulators and/or calculate and adhere to new capital requirements from across a large number of countries and a much larger number of automation systems.

Toward that end, this book will show one way in which enterprise architecture can shape the development of automation services going forward such that it can deliver services and solutions at an entirely new level of quality and efficiency.

The key is that automation resources have the appropriate level of health that they can deliver, and then it sometimes helps if they actually chose to deliver.

Shortcuts

Many companies are simply reducing IT budgets and choosing to outsource their entire set of automation delivery capabilities; however, simply reducing the food supply to an organism or outsourcing the feeding of an organism will not facilitate its development into a healthy and responsive body. In fact, it only increases that organism’s chances of expressing even less desirable traits and even failing.

Too often in today’s economic environment, the board of directors of large public companies are selecting executive management who cut costs in automation that like the strategy of skipping meals, leaves the organization less healthy and less responsive. All too often the techniques used create the impression that the goal for many of these individuals in leadership roles is to simply cash out, whether that is the intent or not.

It comes down to the arrangements that the board of directors make with their executive management team. In the vast world of unintended consequences, it always comes down to the issue of always being wary of what one wishes for.

As an example, when one sees the number of large organizations that are borrowing money at low interest rates simply to have the organization buy back its own stock, one has to wonder what the business objective is.

While it is perfectly fine to buy back one’s own stock out of profits, unless management is planning to take the firm private or defend it against a hostile takeover, borrowing to buy back one’s stock does not enhance market share, improve efficiency, increase productivity, nor reduce its debt.

The motivation for leadership to borrow money to drive up the price of the stock is most likely to drive up the value of stock and stock options. Once this is accomplished, the company will naturally roll over its financing at low rates until such time as interest rates increase. Higher interest rates are generally a good deterrent for this kind of situation arising.

When interest rates eventually increase, which they will, the organization will struggle to survive. In the meantime, the previous leadership team will in all likelihood will have already cashed out, leaving new leadership and shareholders with the prospect of having to issue more stock and/or sell off parts of the company.

The flip side of this of course is that there are other insightful leadership teams that recognize the business opportunities created by poorly managed organizations. They recognize opportunities for positioning themselves to buy poorly managed organizations at bargain prices and for expanding their market share when less effectively managed companies are driven into financial difficulty.

Outsourcing is a common shortcut. Although outsourcing can be properly implemented as part of an intelligent business strategy, it is often the result of management not knowing what to do and having the desire to avoid doing a lot of work and assuming a great deal of risk while doing that work even if it is probably the right course of action.

In the end, outsourcing is not a substitute for creating an environment across IT that will allow it to better express the commands of executive management, it will not raise the degree of civilization that will facilitate rapid and effective cooperation across the organization, nor will it improve the level of fitness that can help ensure the long-term health of the IT organization that will allow it the agility to meet the needs of the CIO.

The problem is often so difficult that often the company that was outsourced to then outsources the automation to another company.

Rise and Fall of Organizations

It is not easy to retain a company’s greatness. Just look at the list of the top companies in the Fortune 500 every 10 years over the past century. However, as numerous organizations decline, others rise. The issue becomes understanding the factors that contribute to making companies rise. (Good to Great, Jim Collins, 2001, Harper Business, ISBN 978-0066620992)

It comes down to good management and creating a culture that is conducive to being a strong competitor, and then building that into the culture so that it survives the executive management team that established it.

How does a good idea take hold?

The ability to spread good ideas and practices to other areas of an organization determines the effectiveness of a company in the marketplace. Good ideas and practices can emerge within any department where talent is able to thrive, such as enterprise architecture, solution architecture, a particular application development team, or a CTO organization over potentially prolonged periods of time.

The ability to make good things persist across an organization has much to do with the incentives to identify and champion good ideas. On the other hand, it is frequently staggering how many times individuals and teams have come together and developed great ideas, concepts, and work, only to have them disappear into the randomness of information.

Part of our mission here is to find ways to institutionalize the right kind of environment, the necessary degree of civilization, and the habits that retain a healthy level of fitness. Limiting oneself to eight weeks of fitness will not achieve anything worthwhile.

How to Read This Book

To a large extent, his book was meant to be read sequentially due to the way it introduces and builds upon concepts. To a large extent, this book is intended to depict the scope of enterprise architecture and as such, many sections are quite small, sometimes just a paragraph to lend a sense of the topic. For example, the section on artificial intelligence is only a page, whereas the previous book I wrote is only about the subject of artificial intelligence. Likewise, every section in this book can be an entire book on its own and I only expand into several pages when it is critical to do so. You will also find that many sections can be viewed as subsections of prior sections. Please don’t let that disturb you.

For the most part, it is the industry understanding of the many topics of this book and how they relate to one another that this book is addressing. The scope of enterprise architecture I would assert has been previously misrepresented.

If you are an architect and have a specific near term need, then jumping to that chapter is a good idea. That said, it is best to come back and read up to those areas sequentially when you have the time.

One additional point worth pointing out is that this book is about enterprise architecture in the era of Big Data and quantum computing. While we will discuss these two topics more than most, they are individually large topics each worthy of their own book.