Scrum Shortcuts without Cutting Corners: Agile Tactics, Tools, & Tips (2014)
Chapter 7. Monitoring and Metrics
Ensuring that a Scrum project stays on track requires frequent tuning and fine-tuning. Thankfully, Scrum’s core set of sprint activities provides us with regular opportunities to monitor progress. Anecdotal observation can be greatly assisted with a selection of carefully selected metrics, tracking boards, and team synchronization practices.
The following three shortcuts focus on some useful techniques and tools to help gauge how your project is shaping up.
Shortcut 19: Metrics That Matter provides a selection of effective metrics that can be used for gauging ongoing progress. Shortcut 20: Outstanding Stand-Ups offers a selection of tips and tricks to ensure that the daily scrum doesn’t become an endless talk-fest. Finally, Shortcut 21: Taming the Task Board provides advice for getting the most out of your team’s visual centerpiece.
Shortcut 19: Metrics That Matter
You’ve introduced Scrum and your team is out of the blocks, sprinting away! Things are going pretty well until some wise guy from senior management comes up to you and says something along the lines of, “Sooo, this whole Scrum thing sounds great in theory, but what metrics are you going to use to actually demonstrate to us how effective it really is?”
Like it or not, people just love to measure and compare, so when it comes to avoiding metrics, you can run, but you can’t hide. In this shortcut, I offer you some suggestions about what metrics actually matter when it comes to implementing Scrum.
Types of Metrics
The most important piece of advice that I can offer when it comes to metrics is to use them only for good, not for evil. Considering that there aren’t any readily available global definitions of good metric and evil metric, I have come up with my own:
Good metric: Used as a signal to help the team identify roughly where things are at and, more importantly, as a guide to help the team inspect and adapt its processes to improve over time
Evil metric: Used as an inflexible indicator for micromanaging an individual’s performance over time and, more importantly, for beating people up and killing morale
I look at grouping metrics into two main categories:
Metrics for Scrum projects (the focus of this shortcut)
Metrics for broader Scrum rollouts (the focus of Shortcut 28)
Four Meaningful Metrics
The following sections walk you through a selection of four project-related metrics that I find particularly helpful:
Sprint burndown
Enhanced release burndown
Sprint interference
Remedial focus
Sprint Burndown
The sprint burndown is a forecasting metric that assists in tracking progress throughout the current sprint.
How Is It Generated?
The sprint burndown is generated in the following manner:
1. For each day in a sprint, plot the sum of the remaining times for all tasks in the sprint backlog.
2. Draw a connecting line between the current day’s total and the previous day’s total (see Figure 7.1).
Figure 7.1. A sprint burndown chart, updated on a daily basis.
When Is It Generated?
The sprint burndown is generated at the end of each day of a sprint, excluding the final day, which is dedicated to the sprint review, the sprint retrospective, and planning for the subsequent sprint (see Shortcut 8).
What Is It Telling You?
The sprint burndown metric acts as a daily gauge for the Scrum team to help manage its workflow and track progress.
If the chart is trending behind schedule (see Figure 7.2), it could be reflecting the fact that
New tasks were added to the sprint backlog (that weren’t anticipated during sprint planning).
Some of the task estimations were incorrect.
Team members had taken some unplanned time off.
Impediments had hampered progress.
Figure 7.2. The team is clearly behind schedule; time to speak to the product owner about possibly decreasing scope.
Of course, it is possible that all four factors had come into play, causing the now expected delay.
Following sprint planning, many teams draw a straight, diagonal (theoretical) line from the top of the y-axis values to end of the x-axis values and use it as a benchmark for the actual burndown line. I warn against using this approach because it can easily create an inaccurate perception of progress. The problem with this line is that sprint progress rarely mirrors the theoretical line on a day-to-day basis. Many sprint burndown lines actually burn up for the first few days because of new discoveries before beginning their downward descent as the team gathers momentum. By including the theoretical line, a curious stakeholder may get the misleading impression that the team has fallen behind after only a day or two.
How Can You Act On It?
If the sprint burndown clearly indicates that the team is not going to reach the sprint goal, then apart from doing everything in your power to help remove any impediments (see Shortcut 9), you should have a discussion with the product owner to assess whether any scope can be removed. If the slip is due to inaccurate estimating of tasks, analyzing why the estimates were wrong can help improve the sprint planning accuracy for the next sprint.
Sprint burndown charts can also paint a rosier picture (believe it or not) if they trend steeply toward an early completion of the sprint backlog (see Figure 7.3). If this is the case, the burndown should prompt the product owner to prepare the next sprint-ready product backlog items (PBIs) (see Shortcut 11) for additional consumption before the upcoming sprint planning session.
Figure 7.3. The team is clearly ahead of schedule; time to speak to the product owner about adding scope.
Enhanced Release Burndown
The enhanced release burndown is inspired by Mike Cohn’s Alternative Scrum Release Burndown Chart (2002).1
1. To find out more about Mike Cohn’s Alternative Scrum Release Burndown Chart, go to www.mountaingoatsoftware.com/scrum/alt-releaseburndown/.
How Is It Generated?
The enhanced release burndown chart is generated in the following manner:
1. For each sprint, plot the sum of the remaining points for all PBIs in the product backlog designated for the next release.
2. Draw a trend line relating to the data points in step 1.
3. For each sprint, plot (as negative y-axis values) the sum of the story points for any PBIs added to the product backlog after the start of the project (if applicable).
4. Draw another trend line that relates to the data points in step 3 (see Figure 7.4).
Figure 7.4. An enhanced burndown chart, updated after each sprint.
When Is It Generated?
The enhanced release burndown metric is generated at the end of every sprint.
What Is It Telling You?
This metric signals what the development team’s rate of progress is relative to the scope’s rate of change. The point where the two trend lines (hopefully) intersect indicates roughly how many sprints will be required to complete the release. If the trend lines run parallel to each other (or diverge), it is an ominous indication that the release will theoretically never see the light of day (see Figure 7.5).
Figure 7.5. Ominous signs that this release might never see the light of day. Better improve practices, remove impediments, or decrease scope.
How Can You Act On It?
If the two trend lines do not intersect or the expected release duration is intolerable, then either the rate of progress needs to increase (by improving practices and/or removing impediments) or the scope needs to be reduced.
Sprint Interference
Sprint interference is a productivity metric that assists teams with their sprint capacity planning.
How Is It Generated?
Sprint interference is generated in the following manner:
1. For each sprint, plot the sum of the time spent by any of the developers for any non-sprint backlog tasks.
2. Draw a trend line that relates to the data points in step 1 (see Figure 7.6).
Figure 7.6. Be sure to note the trend in this graph to assist in estimating your team’s capacity at the next sprint planning session.
When Is It Generated?
The sprint interference metric is generated during sprint planning.
What Is It Telling You?
By providing visibility on the time spent handling historical sprint disruptions, this metric helps you to estimate the potential sprint capacity for the upcoming sprint (the amount of time that the development team should allocate to sprint backlog tasks). This is especially helpful if you have adopted commitment-based sprint planning (see Shortcut 8).
How Can You Act On It?
In any given sprint, there will be a range of external organizational disruptions that simply can’t be avoided. This metric assists in quantifying these disruptions and can also indirectly help to identify what are unavoidable disruptions (such as company meetings) and what are avoidable impediments (such as constantly having to fix inadequate equipment).
Remedial Focus
Remedial focus is a quality metric that enables teams to gauge how much of their collective effort is being spent on remedial bug-fixing as opposed to working on new requirements.
How Is It Generated?
Remedial focus is generated in the following manner:
1. For each sprint, plot the total velocity (the sum of the points for all PBIs including both new functionality and bugs).
2. For each sprint, plot the sum of the points for all bug-related work.
3. Draw a trend line that relates to the data points in step 2 (see Figure 7.7).
Figure 7.7. This indicates that quality is improving. The effort spent on bug-fixing is going down while velocity remains consistent.
When Is It Generated?
The remedial focus metric is generated at the end of each sprint.
What Is It Telling You?
This metric monitors the fluctuations in product quality by measuring the percentage of each sprint that is spent working on bugs as opposed to new functional requirements.
In addition, by quantifying the makeup of the total velocity, additional insight can be garnered. For example, the total velocity may be trending upward, which on the surface would indicate positive improvement. However, if the amount of bug-related work is also trending upward, it indicates that the level of quality is slipping (see Figure 7.8). As such, the increase in total velocity could in fact indicate that the team is just getting faster at fixing its own bugs—somewhat of a backhanded compliment.
Figure 7.8. Even though velocity is increasing, it’s not all good news, as quality is decreasing. Time to revisit the definition of done.
How Can We Act On It?
If the time spent on bugs isn’t trending downward, it is a clear indication that the level of inherent quality is insufficient. This should prompt the Scrum team to revisit the definition of done to tighten up the quality requirements.
Beware of Analysis Paralysis
These four metrics form a small subset of the potential metrics that can be derived and utilized during a Scrum project. But be warned: incessantly generating metrics (just because you can) may cause you to contract an illness that Scrum on the whole tries to combat—analysis paralysis.
Finally, it is vitally important to reinforce the point that metrics need to be handled with care and used for good, not for evil. Use metrics as signals to help your team improve their practices and definitely not for micromanaging and measuring individual performance.
Shortcut 20: Outstanding Stand-Ups
Ask anyone in the sales department what Scrum is, and apart from mentioning the colorful, sticky-note-decorated task board (see Shortcut 21), they will more than likely mention the daily scrum, also known as the team stand-up.
The daily scrum is the regular pulse of the team. A pulse needs to be steady, consistent, and lively to remain healthy, and this shortcut gives you a few tips to help ensure that these sessions keep humming along like clockwork.
When and Where?
First things first: I highly recommend that the daily scrum be conducted as a stand-up rather than a sit-down. It is a subtle yet important distinction. The simple act of standing provides the team with a sense of liveliness to help kick-start the day, and it also ensures that the session stays short and sharp to prevent aching legs!
I am not a believer in imposing the time of the daily scrum on the team: the time should be decided as a group. However, you should encourage everyone to commit to a time that works for them that is as early as possible in the day. Of course, if your team is disparately located, then your daily scrum time will need to accommodate the various time zones.
Once there is an agreed-upon time, you can start to reinforce some guidelines. Here are three that I am a fan of:
The daily scrum should start on the dot irrespective of who is late.
Anyone who is late without either prior warning, a super excuse, or an extremely funny excuse (that induces laughter from every team member) needs to make a financial contribution to the late jar (this can go toward an agreed-upon charity).
Your daily scrum should look like a nice tight circle (or semicircle around the task board) rather than an amorphous blob (see Figure 7.9).
Figure 7.9. Your daily scrum should look like a nice tight circle (or semicircle) instead of an amorphous blob.
What Should Be Covered?
The typical questions that each developer answers during the daily scrum are
What did I achieve yesterday?
What do I hope to achieve today?
Do I have any impediments or blocks?
Although these questions seem straightforward, I have a few specific tips to make your daily scrums more effective. First, everyone should reference the tasks on the task board when discussing what they achieved and what they hope to achieve. This way, you ensure that the task board is up to date (if anyone forgot the night before).
In reality, it should take each team member only about 30 seconds to run through the three questions. However, the problem is that at least a couple of updates every day will typically spawn spinoff discussions that can drag the entire team into a black hole of debate (until everyone realizes that their legs are aching after standing for half an hour!). It is very easy for a daily scrum to get hijacked by implementation detail, so I highly recommend that whenever you get a whiff of tangential discussion, jump in and say “offline,” or if you want to be more subtle, slowly start raising your hand. What you are communicating with this prompt is the following: “I know that this is an important discussion, but let’s get through all of the updates, and then at the end, anyone who is required to participate in the discussion can hang back.”
GIFTS
Agile consultant and blogger Jason Yip explains the key purpose of the stand-up using an apt acronym, GIFTS (Yip 2011), which I like as a simple mnemonic:
Good Start—Good Start means that the stand-up meeting should give energy, not take it. Energy comes from instilling a sense of purpose and urgency.
Improvement—We can’t fix problems we don’t know about so a large part of stand-ups is about exposing problems to allow us to improve. Improvement is not just about problem solving though. Sharing better techniques and ideas is also important.
Focus—The stand-up should encourage a focus on moving work through the system in order to achieve our objectives, not encourage pointless activity.
Team—Effective teams are built by regularly communicating, working, and helping each other. This is also strongly tied with team members helping each other with shared obstacles.
Status—Status is about answering a couple questions:
How is the work progressing?
Is there anything else interesting that the team should know?
Multiple Teams
Your project may include multiple parallel Scrum teams that share common interfaces (both technical and communicative). A popular option in this situation is to run a scrum of scrums stand-up (an additional stand-up with a representative from each team). This is a good option, but I personally prefer the use of stand-up ambassadors who act as observers in the other groups to pick up on any potential contention and/or lessons (see Figure 7.10). This way, there are fewer potential breakpoints in the communication channel. These ambassadors are typically the most senior developers in each of the groups. If you adopt this approach, it is important to stagger the kick-off times of each team’s daily scrum so that ambassadors can attend.
Figure 7.10. Stagger the kick-off times for each team’s daily scrum so that ambassadors can attend.
Ignore the ScrumMaster
I like the way Ken Schwaber (2004) puts it when he states that the daily scrum is supposed to be an opportunity for the group to briefly “socialize and synchronize.” It is not a roll call or micromanagement session where the team reports into the ScrumMaster and/or product owner. I often find that some team members get into the habit of directing their update to the ScrumMaster only. If you notice a team member addressing the ScrumMaster without making eye contact with anyone else, a tip is to start slowly turning around or looking up at the ceiling—I have found that the habit is quickly broken with this physical cue.
Some Extra Touches
I recommend that before the daily scrum formalities begin, encourage (but don’t contrive) any light banter and joking around—it’s always nice to start the day on a positive note feeling energized instead of feeling like you’re at a military roll call!
I also like to use the daily scrum as the forum for awarding the infamous broken build punishment. The team can really flex their creative muscles when collectively determining their approach to this (see Figure 7.11). Some suggestions include anything from the relatively innocuous monitor that flashes the culprit’s name to the more extreme practice of placing a moldy loaf of bread on the transgressor’s monitor (Keith 2010)—this was much easier prior to flatscreens! My longstanding preference sits somewhere in the middle of this spectrum whereby the culpable party is awarded the Akubra (Australian cowboy hat) to wear for the rest of the day (including to lunch!).
Figure 7.11. Broken build “punishments” can range from innocuous to extreme.
While the daily scrum can certainly flow in a stock-standard direction, I like to keep everyone on their toes by utilizing some sort of randomizer. For example, you can use a small soccer ball that gets passed around from team member to team member in any order; anyone who isn’t listening (or is a terrible soccer player) will get a little embarrassed as the ball slips through his or her legs.
It’s Hitting the Big Time!
I see a bright future for the daily scrum. Along with the task board, it is a popular Scrum element that I’m already seeing cross the chasm into non-software-related teams to great effect; even the mainstream Wall Street Journal (Silverman 2012) has reported on its growing popularity!
The daily scrum is a simple concept, but without care it can quickly turn into a daily mess! So, try out some of these mess-mitigating tips, and never forget Conway’s law:
Organizations which design systems . . . are constrained to produce designs which are copies of the communication structures of these organizations. (Conway 1968)
Shortcut 21: Taming the Task Board
For those sitting on the Scrum periphery, probably the most recognizable element of any Scrum project is the visual team task board. Sitting as the focal centerpiece and gathering point for the team, this colorful, sticky-note-adorned board has almost become the symbol of Scrum. With popularity comes variety, and these days there is no shortage of variations when it comes to board configuration and form factor. While there really is no right or wrong way to create your Scrum centerpiece, you won’t be surprised to find that I certainly have strong opinions on the matter!
Digital or Physical?
Personally, I was a little baffled when I first saw a physical Scrum task board, utilizing low-tech, paper-based sticky-notes, marker pens, and sticky-tack! Why, oh why, would we, as bleeding-edge technical hotshots, go back to the dark ages and use antiquated tooling instead of nice, shiny, oversized monitors projecting slick charts? Well, the answer to the question is simple: human psychology. The pioneers of the task board certainly knew what they were doing when they made the simple, physical board the de facto standard.
The bottom line is that there is something satisfying and gratifying when you get up off your seat, walk to the board (it really isn’t that much exercise, people), pick up a sticky-note, and slap it into the Done column. I feel that the visceral “ceremony” of this movement really appeals to our natural sense of achievement because of the visual recognition of work completed (particularly in an industry where most of the hard work is invisible to others). Also, it doesn’t hurt to make the physical working environment more colorful and stimulating, does it?
Materials Needed to Go Old School
To set up a physical task board, you need the following:
Large whiteboard/wall/pane of glass
Blue painter’s tape (to create the columns)
Large ruler (for your rows)
Whiteboard marker (also for your rows)
Sticky-notes (two colors)
Setting Up Your Columns
Columns can be set up in a variety of ways. My preference is the following:
Not Started | In Progress | Ready to Verify | Done
Rows of Sticky-Notes
The rows represent the sprint backlog items, including the PBIs (and associated tasks) that will be focused on during the sprint. Don’t use your tape for the rows (just the columns) because the rows will obviously vary per sprint (and retaping every couple of weeks will get very annoying). Basically, each row will be dedicated to a single PBI and its associated tasks.
Each sticky-note represents a specific task item. Try to make each constituent task of a PBI a “vertical,” independently testable slice; otherwise, the Ready to Verify column won’t be as meaningful on a task-by-task basis. I talk in more detail about this task-splitting process in Shortcut 8.
Sticky-Note Content
If you are also using a software tool to help manage your Scrum artifacts, there is a fine line between wasting time replicating details (that have already been captured digitally), on the one hand, and not jotting enough detail on the sticky-notes, on the other hand. The trick is to write just enough on the note to make it identifiable. I recommend that your sticky-notes include the task ID number (that is automatically generated by the software tool), the initials of whoever has taken on the task, a few words describing the task, and the current time remaining (see Figure 7.12).
Figure 7.12. Example content for a sticky-note.
Any unplanned work should also be captured on the task board, though I recommend using a different-colored sticky-note (see Shortcut 12). This way you can clearly identify potential improvement areas in the sprint planning process.
Generating the Burndown
I’m a fan of the sprint burndown chart (see Shortcut 19), and if you are using Scrum software, you should be able to automatically generate it on a daily basis. However, even with this option, I prefer updating it manually. It takes less effort to extend the line by a one-day segment than to reprint another chart. In addition, I enjoy the ceremony of updating the burndown in front of the team just before officially kicking off the daily scrum, as I find it adds a healthy air of anticipation!
Some Important Decoration
There are several other artifacts that you might consider printing out in a nice big, bright font and sticking on the wall near the task board (see Figure 7.13).
Figure 7.13. The task board in all its glory along with some “important decoration.”
Sprint Goal
The sprint goal (see Shortcut 8) is the umbrella objective that the team is aiming to achieve, so it would be remiss not to display this headline prominently.
Retrospective Goals
After the previous sprint retrospective, the team should have determined the priority process improvements to focus on in the upcoming sprint (see Shortcut 23). It’s easy to forget about these actions when the team is heavily engrossed in the actual functional work, but if these actions aren’t kept front of mind, then the team will get caught in a vicious cycle where continual improvement is relegated to an afterthought. I have seen teams use a completely separate task board purely for retrospective tasks, but I recommend printing these goals out and sticking them on the project wall.
Definitions and Principles
In the early stages of a project new to Scrum, the team must absorb a lot of new information. Remember that familiar processes and definitions have been turned on their heads, and breaking habitual thinking doesn’t happen overnight. To combat those old habits, displaying the new definitions of commonly referenced elements such as impediments (see Shortcut 9) and bugs/issues (see Shortcut 16) can be very helpful.
Keeping It Real!
In the very early stages of a development project, before the various user stories morph and amalgamate into a slick, well-packaged final product, the team might occasionally lose sight of the big picture and final goal. Following are a couple of suggestions to help everyone keep their eyes on the prize.
High-Level Mock-Ups
In many cases, before the first sprint has kicked off, the product owner will have worked up some high-level mock-ups of the key user interfaces (even if they are just hand-drawn sketches at this stage). Irrespective of how rough they are, it is a great idea to stick a copy of them near the task board to give everyone a constant reminder of the big picture.
Customer Quotes
Unless your team is developing the first product in a start-up, your organization has existing users who, on the whole, really appreciate what you do (otherwise you would be sitting at home playing Xbox). No doubt these users have occasionally offered feedback on what they like and dislike about your product(s). You may hear things such as, “We love the simplicity behind product ABC,” or perhaps, “Product XYZ is so responsive compared to your competitors.” To ensure ongoing success, it is prudent to keep these user signals front of mind. How many times have you seen great products lose their edge because they digress from the core ingredients that made them popular in the first place? I know from firsthand experience that providing visibility on these types of user quotations prompts much more scrutiny when the product owner or other stakeholders start suggesting the inclusion of that extra bell or whistle.
Party Time!
After a particularly difficult sprint, you might be pretty darn excited about throwing all of those sticky-notes straight into the nearest trash can. If trashing the notes provides you some much needed therapy, then go right ahead. However, another suggestion is to keep them all (in the bottom of some drawer) and decorate the release-party room with them for some nostalgia (or perhaps nausea)!
Although the task board might currently be perceived as an exclusive agile tool, I often discuss its efficacy with leaders operating in fields as varied as the legal profession and school teaching, so sit tight, as I don’t think it will be long before these colorful boards start to take over the world!
Wrap Up
The three shortcuts discussed in this chapter focused on a selection of tactics, tools, and tips to help gauge how your project is progressing. Let’s do a quick recap of what was covered:
Shortcut 19: Metrics That Matter
Differentiating between good and evil metrics
A selection of meaningful metrics: the sprint burndown, the enhanced release burndown, sprint interference, and remedial work effort
The importance of avoiding metric overkill
Shortcut 20: Outstanding Stand-Ups
Daily scrum logistics: when, where, and what should be covered
Options for synchronizing multiple teams
Dealing with common daily scrum dysfunctions
Shortcut 21: Taming the Task Board
Factors to consider when deciding between a digital or physical board
How to set up a physical task board
Additional artifacts to consider embellishing your task board with