The Mikado Method (2014)

Preface

We once worked on an application that was used to configure the behavior of a specialized electronics module in an expensive hardware product. Our application was a part of the delivery of that product. One day, news reached our software development team that Sales had unexpectedly landed a contract with a new client. At first we didn’t know what to make of it, but in time we learned that the new client needed a version of our application that was almost identical to the one we had. This was fortunate. But after further investigation, we realized that there were certain parts of the application code that under no circumstances could be shared between the new client and our old ones. We had to come up with a solution to this problem, or we couldn’t deliver to the new client at all.

It finally boiled down to two possible solutions. We could duplicate the whole application, or we could restructure first and break out the parts that couldn’t be shared. Our team argued for well over a week before we reached consensus. We decided that the best solution was the restructuring approach: restructure first, and then add the new functionality. This shouldn’t take more than a month, or maybe two, we thought to ourselves.

We started working, and guess what? The work was more complex than we could imagine! The approach we chose was supposed to be simple, and we thought we could get away with moving some classes to new packages, fixing the compilation errors, and then adding the new functionality. But as we tried to split the application and separate sensitive information from the more general parts, we ran into trouble.

We had anticipated some trouble, but not that much. After every change we made, we got about 20 compiler errors, and as we tried to fix them, a myriad of new errors came up. When we were just about done with the new errors, the compiler moved ahead a bit, only to reveal another round of errors. Fixing those errors led to more restructurings and of course more errors.

More than two weeks went by, and we had well over a thousand files checked out and more than two hundred compiler errors. By that time, the high-level system tests, our indicator of a healthy application, had long since been put out of play because we couldn’t even compile the application.

To make matters worse, the rest of the team was moving along at their regular pace, and our efforts at keeping the code in sync became more and more difficult. It became obvious that we couldn’t keep up with the rest of the team, and that the complexity of the change was more than overwhelming. We were trying to behead a Software Hydra—for every head we cut off, two more grew out.

We still remember the day when we swallowed our pride and made the tough decision to revert everything we had done so far. Only later did it dawn on us that reverting the code was a defining moment in our software careers. Apart from realizing that it’s never too late to turn back, we had also discovered how to approach our particular problem, and other problems like it.

Our insight was that in order to get the system in shape, we had to do almost all of the things we had already done, but we needed to do them in the opposite order. You might think this sounds crazy, but the key was to take the errors and turn them into prerequisites.

Brownfield Development

You’ve likely worked with systems like the one we described—software hydras that aren’t easy to behead. For us, this is more or less the norm. We’ve had to clean up lots of messy code, refactor smaller and larger parts of systems, and restructure complex architecture. We’ve even tried rewriting code from scratch, also known as greenfield development. But we’ve noticed that the field doesn’t stay green very long. We’ve long since faced it; we’re stuck with brownfield development, whether we like it or not.

The Software Hydra—for every head we cut off, two more grew out.

We, the developers, hold the fate of the code in our hands, and we’re the only ones who have the power to improve it. It’s our responsibility to keep the code clean and fit for its purpose. This means we have to be able to add code and improve our own code and the code of others. If computer programs don’t improve, they’re doomed to a slow death.

How do we approach large improvement efforts? How do we deliver while the pressure is high, because we must deploy functionality once a month, or even more frequently than that? How do we deal with huge systems that are hard to comprehend, and whose complexity makes it hard to keep all the necessary details in our heads at once? We morph systems into a new shape!

The Goal: Morphing a System

Most organizations that deliver software don’t enjoy halting the development of new functionality. The stakeholders want to see value coming out as money is put in. As a result, software developers have to be able to cope with the current situation as future problems are encountered, solved, and sometimes avoided entirely.

If we want to be successful software developers, we need to learn how to morph an existing system into a desired new shape. When we say morph, we mean perform the necessary actions and steps that take us from a system of one shape to another without breaking things in between. The transitions between the different states need to be smooth and possibly add value themselves.

What complicates the process of morphing software even more is the fact that it’s more or less the norm to develop software as a team. Software is very seldom developed by a single individual, and working in groups makes distributing the work possible. But teamwork also adds another dimension of complexity, and it doesn’t take long to realize that the need to communicate necessary improvements within a team is essential for any real progress.

The Mikado Method

The Mikado Method is our guide to morphing. It helps us visualize, plan, and perform business-value-focused improvements over several iterations and increments of work, without ever having a broken codebase. It enhances communication, collaboration, and learning in software development teams. It also helps individuals and programming pairs stay on track while doing their day-to-day work. The framework that the method provides can help the whole team morph any system into the desired new shape.

The method itself is straightforward and simple: after you’ve decided on a goal, you start to build a model of required changes and dependencies by doing quick experiments.

We try to accomplish the goal in a straightforward fashion. When we can’t, we create a graph of prerequisite nodes. We use the information from our quick experiments to determine whether a change can or can’t be implemented right away. If a problem occurs, we roll back to the original state. By solving the prerequisites first, we work our way backwards, and the graph we build guides us toward our goal.

The Name of the Method

The Mikado Method gets its name from the Mikado game.

Mikado is a pick-up sticks game originating in Europe. In 1936 it was brought from Hungary to the USA and was mostly called pick-up sticks. This term is not very specific in respect to existing stick game variations. Probably the “Mikado” name was not used because it was a brand name of a game producer. The game got its name from the highest scoring (blue) stick “Mikado” (Emperor of Japan). The Buddhistic Chien Tung also contains a stick called “emperor.” (Wikipedia, February 2011, http://en.wikipedia.org/wiki/Mikado_(game))

To start the game, you drop the bunch of straw-sized sticks onto a surface, creating a pile. The goal is to pick up as many sticks as possible to score points, and preferably the Mikado stick, because it’s the highest-scoring stick. If you’re unable to pick up a stick without moving another one, the stick is put back and the turn goes to the next player. As long as you can pick up sticks without moving other sticks, your turn continues.

The trick is to pick up the easy sticks first—the sticks that have no other sticks on top of them. Eventually, by using that strategy, a player can pick up the Mikado stick and likely win the game.

The similarities between the Mikado game and restructuring software are striking. There are loads of dependencies and structural complexities in code that have to be taken into account. These dependencies have to be navigated and changed with care until you have an opportunity to do something without breaking stuff.

Characteristics

These are some characteristics of the Mikado Method:

· It fits nicely in an incremental process.

· It’s very lightweight (pen and paper, or whiteboard).

· It increases the visibility of the work.

· It provides stability to the codebase while you’re changing it.

· It supports continuous deployments by finding a nondestructive change path.

· It improves communication between people.

· It enhances learning.

· It aids reflection on the work done.

· It leverages different competencies, abilities, and knowledge.

· It helps collaboration within a team.

· It scales by enabling distribution of the workload over the team.

· It’s easy to use.

A Thinking Tool

When we introduce the Mikado Method to people who develop software, we often see how they change the way they look at, approach, and talk about large structural improvements. Their perspective shifts from a very analytical view of problems to a more practical approach, where the focus is on removing the minimum number of obstacles at a time in order to achieve real results.

The effect is shorter, focused conversations that are about finding the changes that can be made without breaking the code, instead of rigorous analyzing and guessing. They get more insight on how to deal with the unnecessary complexity of a software system, and the method serves as a thinking tool when they solve difficult problems.

Even though we’re becoming increasingly seasoned developers ourselves, we still find it challenging to decide where to start digging and where to take the code. The Mikado Method not only helps us find a starting point, it also shows us where to go and tells us when we’re done.

We hope you’ll find the Mikado Method useful, just as we have.

About this Book

As a codebase grows large and more complicated, and they often do, there usually comes a time when you want to improve portions of it to meet new functional requirements, new legal requirements, or a new business model. You may also just want to change it to make it more comprehensible. For small changes, you can keep things straight in your head, but for larger ones the chances of getting lost on a sea of broken code increases dramatically.

As you desperately try to navigate that sea, it’s easy to start labeling the code. You can come up with all sorts of names for the code, especially bad code that isn’t fit for its purpose. Legacy code is one of the more popular terms, which literally means code someone (else) wrote and that you are now responsible for. Michael Feathers, in Working Effectively with Legacy Code (Prentice Hall, 2004), suggested it can be defined as code without tests, which has since become the de facto definition.

Another term for bad code is big ball of mud, popularized by Brian Foote and Joseph Yoder in their paper Big Ball of Mud (http://laputan.org/mud/). This paper describes a big ball of mud as “a haphazardly structured, sprawling, sloppy, duct-tape-and-baling-wire, spaghetti-code jungle.” Some synonyms for this type of code are crap or a mess, and we can think of quite a few more in our native language (Swedish).

When code like that needs to change and trouble appears, it’s easy to label the code legacy, a big ball of mud, a mess, crap, or just impossible to work with. If it’s really bad, developers often distance ourselves from the code and start lobbying for a rewrite, because changing the code is perceived as too hard.

In addition to the big-ball-of-mud systems, there are at least two more types of systems that need attention. First, there are systems that look well organized, maybe even having high test coverage, but under the surface they’re hard to work with or to extend. Second are the systems that look good and have served well, but don’t anymore.

This book is about changing and improving all these types of systems. It offers a way to regain control over your codebase. We call it the Mikado Method.

Roadmap

Chapter 1 is an introduction to the Mikado Method. After reading this chapter, you’ll understand how the method works at a high level. For some people, this will be enough to start experimenting on their own problems.

Chapter 2 shows the mechanics of the method with the use of two small examples. This chapter explains how to use the method when working with code.

Chapter 3 goes into the theory of the method in depth. After reading this chapter, you’ll know the nuts and bolts of the method, and understand why and how it works.

Chapter 4 is a guide that shows how you can use the method in different working constellations, and when to start using the method on a problem. After reading this chapter, you’ll understand how to get the most out of the method in a team, a pair, or on your own.

Chapter 5 presents a longer example that mimics a somewhat real scenario. This chapter will show you how to include tests and how to deal with monolithic code, a common problem for many software developers.

Chapter 6 is about emergent design and the design principles we use when we want to change a system for the better. After reading this chapter, you’ll know which direction to take your code in when you change it.

Chapter 7 presents some recurring patterns we’ve come across when we use the method—patterns related to both drawing graphs and implementing changes. This chapter will give you ideas about how to simplify your graphs and how to go about tricky, but common, changes to code.

Appendix A is a deep dive into the concept of technical debt. This appendix will help you recognize the different types and sources of technical debt, and mitigate those at an early stage.

Appendix B is about setting the stage for an improvement effort. This appendix outlines the most common preparations needed before heading out on an improvement effort. You’ll also get some tips on what to do afterward, to sustain the improvements you’ve made.

Appendix C contains an example in JavaScript that shows how you can use the Mikado Method to approach changes in an environment that uses a dynamically typed language.

The code in the book

The code from the longer examples in the book is available at GitHub at the links shown in this section. It is also available as a download from the publisher’s website at www.manning.com/TheMikadoMethod.

You will find the code from chapter 2 at https://github.com/mikadomethod/book-example-1. In the branches of that repository, you can find the detailed steps of the two examples in that chapter.

The example in chapter 5 is available at https://github.com/mikadomethod/book-example-2. For this example, there’s one branch for each graph in the chapter, with the names of the branches mapping to the goals of the graphs, respectively. The prerequisites are mostly mapped to a commit, and in some cases a couple of prerequisites are bundled in a commit.

The example in appendix C is available at https://github.com/mikadomethod/book-example-3. This example also contains detailed branches for the steps in the book.