Scenario-Focused Engineering (2014)

---

The 10 capabilities we describe are sorted into four categories:

Define direction Teams thrive under leaders who put customer experience first. Leaders who provide clear focus set up their teams to successfully deliver what is most important to their customers.

Customer focus Teams need to develop a deep understanding of their customers by gathering and analyzing various types of data and then use that data to drive decisions. As many of the examples in this book describe, paying attention to customers is not an upfront activity alone. It needs to occur consistently throughout development and delivery.

Complete experiences Understanding customers is the first half of the equation. The second half is for teams to be sure that they create the right experience. These capabilities describe the essential behaviors that enable teams to ship complete end-to-end experiences.

Iterative habits For teams to go from good to great in delivering innovative, high-quality products quickly and efficiently requires specific skills in iteration and experimentation.

Define direction: Drive customer excellence from the top

Leaders must passionately believe that customer-focused iteration is essential to business success and take full responsibility for the customers’ experience. By establishing a team culture that keeps customers front and center, leaders hold their teams accountable for delivering great experiences that delight customers.

What great looks like

Leaders build a team culture that values customer focus, collaboration, and iteration.

Leaders hire with the intent to build a team with skills that cover all capabilities and ensure that all members of the team are respected and utilized to their fullest.

Leaders define and broadly communicate a clear strategic direction, including business opportunities and target customers.

Leaders provide inspiration, space, resources, processes, tools, and support to enable their teams to put customers and their experience at the forefront of the team’s work.

Leaders expect consistent execution across all the capabilities and deliberately set improvement goals.

Leaders create both implicit and explicit rewards to encourage the right behaviors across the team.

Leaders empower individuals to experiment, try new ideas, and respond to customer feedback and not wait for approval to make decisions that benefit customers.

Leaders negotiate agreements with partner teams to support a customer-focused, iterative process and to get commitments for delivering a seamless experience across organizational boundaries.

Leaders pay close and regular attention to customer-experience metrics in addition to business and execution metrics.

Leaders establish and support a dedicated position that has responsibility and authority to ensure alignment and follow-through on customer focus across the organization.

Typical roadmap

Basic Leaders focus primarily on technology or the logistics of running the business and make important product or engineering decisions centrally. The team’s culture generally does not reflect a high degree of customer focus or collaboration, and no leader is explicitly responsible for the customer experience. Success and team rewards are based on business and execution metrics and do not include specific customer-experience metrics.

Emerging A handful of customer champions drive new customer-focused or iterative activities on the team. This effort is mostly bottom-up, with varied amounts of visibility and/or priority with top leaders. As the champions drive these activities, they gain some support and also experience some resistance. Some customer-focused tools, resources, reward systems, and other processes are put in place but may not be uniformly adopted by the team. These customer champions begin to realize that to make the deeper changes they think are necessary, the team culture needs to change.

Exemplary A leader with authority and resources insists on a culture and practices that embrace customer focus and drives adoption broadly across the team and with partner teams. The leader empowers his or her team to iterate with customers and rarely vetoes a team decision once the initial strategy is solidified. The leader holds the team accountable to customer-experience metrics and creates rewards to encourage good practices. The leader sets the tone that all members of the team are respected and utilized to their fullest, including designers, planners, and researchers. A comprehensive set of tools, processes, and other resources are deployed and used consistently. The organization reflects a customer-driven culture, from top to bottom, and this culture sticks over the long term.

Define direction: Choose target customers strategically

Identify a few specific target customer segments that are aligned to your business strategy and brand and that give you the best chance for carryover to a broader market. If you try to build for everyone, you get a generic experience that isn’t quite right for anyone.

What great looks like

A small number of target customer segments are identified and prioritized based on their alignment with current business strategy. These segments reflect the most valuable or strategic customers that are driving the market.

Target customers are segmented through behavioral characteristics (life cycle, level of knowledge/experience, job responsibilities, interests) and not just demographics (sex, age, geography).

Customer segmentation is derived from a range of data, including telemetry to identify the specific customer behaviors that align to business goals and qualitative analysis to discern emerging trends.

Identification of the target customer is based on and validated through research.

Target customer segments are narrow and specific (“soccer moms” rather than “consumers”).

Definition of target customer segments also requires agreement about which customers are not targeted and why.

Targets are chosen to maximize productive carryover into other market segments. The team tracks whether expected carryover actually occurs as the product is developed and shipped.

Target customers are selected with an eye toward whether the customer is right for the brand.

Target customers are understood and embraced by all partners and stakeholders (engineering, sales, marketing, support, operations, partners).

All work is aligned and optimized for the chosen target customers.

Typical roadmap

Basic A number of broad target customer segments are considered. The team has a basic understanding of these customer segments and how each is valuable to the business. Marketing, business strategy, and experience goals are developed in isolation and may be aligned to different target customers. The team resists tight prioritization of specific target customers in favor of a broad, general market approach.

Emerging A comprehensive set of target customers is defined and often mapped by the customers’ position on the value chain. Target customers are aligned to experience goals and business strategy and are compared with customers targeted by competitors. Adjacent customer segments and a carryover model are defined. Engineering teams consistently base their work on the needs of target customers, but these target customers may not be the same as those defined within the marketing and sales organizations.

Exemplary A small set of well-defined and prioritized target customers is fully aligned with business goals, marketing plans, and all aspects of engineering work. Sophisticated carryover models are developed and validated through customer feedback and business results, and the choice of target customers and the carryover model are adjusted as needed. The same definition of the team’s target customers is used for all touch points on the customer journey—from prepurchase through usage to support—and by all organizational functions—engineering, sales, marketing, support, and operations.

Customer focus: Extract deep insights from diverse data

Collect diverse types of customer data and synthesize that data into deep, actionable insights about unarticulated customer needs. Uncover the needs that your customers didn’t know they had and that your competition hasn’t noticed yet, and use those insights to drive differentiation and significant business results.

What great looks like

Data is gathered from all four research quadrants: QUAL/SAY, QUAL/DO, QUANT/SAY, and QUANT/DO. This includes site visits, interviews, surveys, usage telemetry, customer support data, social media, sales data, competitive info, and so on.

A team identifies unarticulated customer needs—what customers can’t describe directly—especially needs not yet met by competitors.

Key insights (the why behind the customer’s needs) are identified. Often, these are deeply held beliefs, values, pain points, and root causes that explain the customer’s motivation.

Insights are shared with the entire engineering team, including partners and stakeholders.

A team develops a strong sense of empathy for the customer.

Customer research data consistently influences decisions in planning, engineering, and business strategy.

A research portfolio covers all aspects of a team’s customers and their environment, including needs, behaviors, emotions, market situation, competitive offerings, and trends.

Research findings are broadly communicated and easily accessible by team members.

When conflicting needs are identified in the data, the team has a process for making prioritizations.

A consistent screener is used across all planning and user research to identify whether research participants are good representatives of a target customer segment.

Data is gathered continuously to track the pulse of the dynamic marketplace and latest customer trends.

Typical roadmap

Basic The team gathers limited customer research and has a surface-level awareness of customer needs, often biased by personal opinions. Research sources (including telemetry) are ad hoc, based on the usage of an existing product or service or competitive analysis. The understanding of customer needs is influenced by customer feature requests, stated needs, and required product fixes. Knowledge of the customer data resides in disciplines dedicated to research but is not shared widely among engineers. Decisions are often made based on personal experience and usage of the product, and not on customer data, research, or feedback.

Emerging The team gathers customer data from diverse sources and consistently identifies unarticulated needs. The team analyzes the data to solve existing problems, generate new product features, and improve quality from release to release. Knowledge about customers is shared widely among engineers, and some engineers develop empathy for the customer. There is limited coordination between customer data used to drive business decisions and customer data used to drive product decisions, which can cause misalignment and conflicting priorities.

Exemplary A portfolio of research is thoughtfully planned and executed at the right time to influence both business planning and engineering with fresh and relevant insights. The team’s schedule allows enough time to wallow in the data for the discovery of insights. The team generates new insights about customer needs that competitors are not yet meeting and is able to make predictions about emerging trends. These insights use and integrate diverse data from marketing, planning, data science, and user research. They are clearly articulated to the entire engineering team and engender a sense of empathy for customers. The team is fluent in making tradeoffs when there are competing requirements among different customer targets.

Customer focus: Continuous learning with customer feedback

Use direct customer feedback to refine your understanding of customers and analyze how they react to your proposed solutions. Get feedback continuously throughout the product’s lifetime. Relying on this feedback to guide your product decisions keeps you honest about what customers need and value.

What great looks like

Customer feedback is collected continuously throughout the product cycle, from early concepts through delivery, release, and ongoing use.

Multiple types of feedback are leveraged, representing all four quadrants: QUAL/SAY, QUAL/DO, QUANT/SAY, and QUANT/DO. Techniques include usability, A/B testing, focus groups, usage telemetry, instrumentation, surveys, informal interviews and observations, and others.

The team has an innate curiosity and passion for the customer and looks for creative ways to get feedback.

The team relies on customer feedback to get from its first idea to the right idea. The team understands that its first ideas are probably incorrect or suboptimal.

Feedback is used to quickly influence product decisions and direction, typically in the next iteration.

A pipeline of target customers is developed that makes gathering feedback easy for the entire team, with customers scheduled and available to provide feedback at least weekly.

A consistent screener is used across all planning and user research to ensure that feedback participants are in the target customer segment.

Social media and telemetry are constantly monitored for customer feedback. Feedback is interpreted according to its alignment with target customers. Emerging problems are discovered and fixed proactively rather than as the result of customer support calls.

Typical roadmap

Basic The team receives customer feedback occasionally, often close to shipping. The feedback is focused on validating that features work rather than on validating that the team is solving the most important customer needs. Changes based on feedback are usually made in the next release. Feedback is not consistently tracked or reported. The team relies on its intuition and judgment to make decisions (absent timely customer feedback). Teams directly implement what customers ask for instead of looking for and acting on deeper customer needs.

Emerging A team gets customer feedback for every major integration milestone throughout the product cycle. Feedback is focused on target customers, defined by a screener, and is used to validate customer needs, the completeness of the solution, and customer delight. Team members are learning how to actively listen and interpret feedback. Changes based on the feedback are made quickly and consistently. Feedback and resulting actions are consistently tracked and reported. The team predominantly uses QUANT/DO data and might have trouble making decisions quickly, preferring to collect more data. The team might overlook QUAL data in favor of QUANT data.

Exemplary The act of gathering customer feedback is integrated into a team’s day-to-day work and is part of the rhythm of every iteration. The team is eager to get customer feedback frequently. Feedback is used to better understand customer needs but also to validate and fine-tune specific solution ideas. The analysis of feedback is triangulated among different data types and rationalizes conflicting findings. Heuristic methods are used successfully to find obvious problems not requiring direct customer input, enabling the team to iterate more quickly. Sources such as social media and telemetry are used proactively as means to identify emerging experience and operational problems. Results and analysis of feedback are shared with all partners. The team has struck a balance between data and judgment and knows when to stop collecting more data and make a decision.

Customer focus: Cultivate a long-term customer relationship

Value long-term customer relationships over short-term revenue or the efficiency of a single transaction. Teams should understand that every customer touch point influences how customers feel about their product, service, and brand. An honest dialog with customers creates an ongoing, dynamic, authentic relationship, not just promotion and marketing.

What great looks like

Customers are treated with respect in order to build a positive, long-term relationship.

Customer relationships are nurtured by consistently sharing either directly or indirectly which changes were made based on customer feedback.

Metrics are defined to measure overall customer satisfaction, sentiment, and engagement.

A broad range of methods, including social media, surveys, and telemetry, is used to track the overall customer experience at every stage of the customer journey, from prepurchase through ongoing use, to upgrade or abandonment.

Customer satisfaction and sentiment are monitored closely because drops in these measures can be an early warning of a problem, and recovering lost ground takes a long time.

The team understands which problems have the highest impact on customer satisfaction and focuses on proactively identifying and fixing those to minimize the impact on customers.

Customer behavior is tracked and analyzed to determine which behaviors best predict customer loyalty, engagement, satisfaction, and future use of a solution.

A detailed plan is created for prioritizing quick responses and resolutions of customer issues, even if a response or decision means delaying ongoing product development work.

When a customer experiences a problem, the problem is fixed quickly and completely and steps are put in place to prevent that problem from occurring again. The highest customer satisfaction comes from having a problem fixed, not from having no problem at all.

A service mentality pervades the team: customers must continually value the product, service, and brand to keep choosing your solution over competitive offerings and spend their time and money with you.

The entire team has constant visibility to current customer satisfaction and sentiment scores and understands what factors most affect those scores.

Typical roadmap

Basic Overall customer satisfaction and sentiment is low or middling. Different disciplines (engineering, marketing, design, and so on) are isolated and customer-facing content is promotional in nature. The customer has difficulty contacting the company. Customer support and the resolution of customer issues are primarily reactive.

Emerging The team tracks satisfaction measures appropriate to each channel (product satisfaction vs. customer support, etc.), which are optimized and aimed at verified target customers. The team responds to customer issues but does not follow through systematically and does not remeasure satisfaction after an incident. The team is aware of current brand equity and current levels of general customer satisfaction.

Exemplary The entire team is able to view the customer experience from the outside in and knows not only the sentiment scores but specifically what drives those scores. A variety of customer satisfaction, sentiment, and overall experience measures are constantly monitored and are closely aligned to business goals. The team has developed a plan for prioritizing responses to customer issues and is able to investigate and respond to changes required. The team follows up on customer-experience measures to make sure that the team’s actions and changes actually improve the score.

Complete experiences: Define complete experiences

Customers expect complete, seamless end-to-end experiences that solve problems, not bundles of technology or individual features. Focus on fully solving real-world problems in delightful ways. Describing these experiences through storytelling and outcomes develops empathy that will carry the team through the hard work of delivery.

What great looks like

End-to-end customer experiences are defined to address customer insights and the real-world problems and opportunities for the chosen target customer.

A small number of complete experiences are identified and prioritized so that they can be built to a high level of quality and detail.

Experiences are described based on deep customer insights or unarticulated needs to create deep customer delight and differentiate your solution from the competition’s.

Storytelling, customer outcomes, and metrics (with goals) are used to create a structured frame that specifies the intended customer experience.

Customer emotions are explicitly considered when a team defines end-to-end experiences.

An end-to-end experience is clearly articulated and validated through customer feedback before a team makes feature, technology, or engineering decisions.

All engineering, business, operations, marketing, and human factors are considered to coordinate a seamless, integrated end-to-end experience.

As engineering, business, operational, and marketing deliverables are specified, each aligns to the priority end-to-end experiences.

End-to-end experiences are prioritized over individual features, feature improvements, or optimizations.

A long-term experience strategy is defined that spans multiple releases or even years.

Typical roadmap

Basic The team focuses on features and functionality with local optimization, low integration, and little immersion in customers’ needs. Stories or scenarios describe a feature or a use case, include many implementation details, and don’t describe the customer’s motivation or problem. Planned solutions often focus on ad hoc features rather than a coherent path through the entire experience. The team tracks and manages work based on feature lists, which can be very long.

Emerging The team uses storytelling, outcomes, and metrics, based on research, to focus engineering plans for a specific release and to articulate business, technology, and experience opportunities and constraints. Planned solutions are expected to be complete and smooth at the component level and to deliver high value seamlessly within a specific product. The team tracks and manages work using both experience framing and feature lists.

Exemplary The team articulates and aligns experience plans to the long-term vision of many releases and uses that to make decisions on where to invest or divest. Experiences reflect deep insights about unarticulated customer needs and create opportunities for significant differentiation from competition. Planned solutions are expected to deliver high-value, seamless experiences across all customer touch points, including with partners and competitors. The team aligns all engineering work to experiences and uses these experiences to track and manage the project.

Complete experiences: Deliver complete experiences

Organize work and the team to effectively deliver integrated, seamless solutions that enable complete end-to-end experiences. Invest in systems and processes that support reliable delivery with a focus on quality and attention to craftsmanship.

What great looks like

Solutions enable high-value, seamless experiences across all customer touch points and the larger ecosystem, including partners and competitors’ platforms, services, and devices.

An owner is assigned to each experience, who has the resources and authority to hold all dependencies accountable to smooth execution and 100 percent completion with a high degree of craftsmanship.

Work plan is tightly scoped to deliver the designed experience on an achievable schedule. The work plan is based on a deep understanding of the intended experience.

All work is tracked and organized to deliver a complete end-to-end experience. This includes all functionality as well as all resources, such as help and marketing materials.

Each experience is defined with detailed success criteria, including task outcomes, sentiment metrics, and long-term engagement goals.

Technology investments are prioritized according to how each contributes to the intended customer experience. Infrastructure and platform investments are scoped to the actual customer-experience requirements. Execution decisions and tradeoffs are based on empathy and customer feedback about customer needs and desires.

Bug triage and design-change decisions are guided by the impact of the proposed change on the overall experience.

The team is brutally honest about the state of the customer experience and whether the success criteria are being met.

The team values shipping complete experiences and resists shipping partial solutions. Work that is not contributing to the overall experience is cut.

Hard cuts are made early to maximize the resources available to finish the most important experiences with high quality.

Tight integration with customer feedback verifies both the experience quality and the impact of any cuts.

Typical roadmap

Basic The team delivers incomplete or broken experiences and consistently attempts to do too much in a given release. The focus is on shipping features rather than on delivering a coherent whole. Experience quality is not tracked systematically. The team is easily distracted by exciting new opportunities.

Emerging The team aspires to deliver complete end-to-end experiences. All experiences have clearly defined success criteria, and their completeness and quality are measured continuously. Bug triage and design-change decisions are guided by the current state of the experience and the assessed impact on the experience. Experiences that would be broken at the time a solution is shipped are cut from the release or the solution is held until these experiences are ready. However, the team might make cuts at the last minute that leave gaps in the overall experience.

Exemplary The end-to-end experience is optimized across all customer touch points and is consistently high quality and polished. All engineering work is organized and tracked around delivering great end-to-end experiences, including resources such as help, marketing materials, product acquisition, and customer-support systems. An owner is assigned to each experience, and that owner has the resources and permission to hold all dependencies accountable to smooth execution. Triage ensures that all released solutions fulfill complete, seamless, end-to-end experiences with a high degree of craftsmanship. Hard cuts are made early to allow the team to focus. The team enables smooth integration with the entire ecosystem, including competitors’ platforms, services, and devices. The team has a clear understanding of the customer’s expectation for the experience and what it needs to do to deliver it.

Complete experiences: Track and react to specific metrics

Well-defined customer experience metrics, in concert with business metrics and technology performance metrics, set an explicit quality bar for all experiences. Carefully tracking these metrics holds a team accountable for delivering the intended experience to delight customers.

What great looks like

Metrics are captured consistently at every iteration through instrumentation, customer testing, and observation.

A metrics portfolio includes a balanced view across system-health indicators such as availability and page load; business-health indicators such as click-through rates, conversion, revenue, and adoption; and customer-experience-health indicators such as task success, discoverability, session abandonment, and continued usage.

Standardized customer-experience metrics are defined across the team for both performance (task completion statistics and error rates, for example) and perception (measuring the useful, usable, and desirable dimensions of an experience).

Vanity metrics are avoided, such as counting clicks or downloads when your business goals are based on usage.

The team holds itself accountable to experience-metrics goals and does not ship until these goals are met.

The team is able to evaluate the accuracy of measurements, align them with target customers appropriately, interpret them, and use them to drive decisions.

Instrumentation is used strategically to capture defined metrics and is a regular part of code check-ins.

Automated tools are developed and used to collate, track, and report on metrics.

Interim builds are continuously validated against stated metrics and success criteria.

Latest measurements are published broadly on a scorecard, a dashboard, or another format that is visible to the entire team.

Metrics determining success criteria are reviewed regularly for how well they align to the customer experience and business goals and are revised if needed.

Typical roadmap

Basic The team defines ad hoc technical performance metrics, with a general focus on system availability, performance, and reliability. Experience metrics are few and usually not actionable. There are no perception metrics.

Emerging The team measures a range of technical, business, and experience metrics systematically. The team relies on this data to make decisions. Success metrics are measured and tracked for every experience. Experience metrics are used alongside technical and business metrics to determine overall readiness for shipping. The team publishes a scorecard to track and communicate experience health status.

Exemplary All team members rely on knowing the real-time experience health status through dashboards and are skilled at evaluating the accuracy of the data, interpreting it, and using it to drive decisions. Telemetry is in place and data exhaust is analyzed for patterns of customer behavior. Perception data is considered equally alongside performance data.

Iterative habits: Envision multiple experiences

Discover the best ideas by exploring broadly, by encouraging wild ideas and playfulness. Deliberately diverge and consider multiple options for every decision to seek a globally optimal solution. Discover the best ideas by combining and recombining ideas in new ways to achieve a breakthrough.

What great looks like

The team starts projects by generating a large number of diverse ideas in different modes, including written, physical, and visual.

The team is proficient at brainstorming, holistic systems thinking, and sketching to visualize ideas.

The team conducts facilitated brainstorming, lateral thinking, and idea-generation activities at the right time in the Fast Feedback Cycle, as well as informal idea generation individually and in small groups as needed.

Throughout the entire project cycle, as the project progresses and more is learned, the team generates ideas at the appropriate level of scope in the iteration funnel.

The team enforces a rule to not make any solution or design decision until at least a handful of credible, diverse ideas are generated.

Ideas are combined and recombined to find new solutions that incorporate the best aspects of multiple ideas.

A deep degree of trust exists across the team, enabling a safe environment for wild ideas to be floated and considered.

Team members know how to give and receive feedback so that it is heard and considered.

The team is aware of tunnel vision and has strategies to delay or mitigate its effects.

The team takes time to marinate to enrich idea generation.

The team recognizes when it has explored the problem space fully by generating and exploring enough ideas to move forward.

Typical roadmap

Basic Ideation is focused mainly on brainstorming feature ideas that are based on the available technology, not on solving customer needs. Envisioning and ideation sessions occur sporadically, if at all, and do not yield changes in the technology-dictated direction. Relatively few ideas are generated before a decision is made. Team culture encourages picking an answer quickly.

Emerging The team has a vanguard group that has responsibility for envisioning and ideation. Big-picture ideas generated by that group form the product roadmap for the broader engineering team. Ideation skills in the vanguard group are high, but those skills are not broadly distributed in the engineering organization. Managers of the vanguard group have high tolerance for ambiguity, but there is a natural tension with other managers, who press for locking down plans sooner. After the product roadmap is defined, the team switches to production mode and rarely ideates during decision making.

Exemplary An entire team embraces diverging activities and ideation throughout the engineering process—at the right times and in rhythm with the Fast Feedback Cycle and the iteration funnel. The broad team and their partner teams understand that to find an optimal, innovative idea they have to start with many ideas. The team environment encourages consideration of wild ideas and unconventional approaches at appropriate times. The team routinely uses diverging activities to find creative solutions to problems and to solve customer needs.

Iterative habits: Learn fast through deliberate experimentation

Frequently and deliberately try out ideas with customers, testing your hypotheses, and use that feedback to inform the next iteration of your work. Iterations across a project follow a curved funnel shape as teams systematically winnow many ideas to a few ideas and finally reach an optimal solution.

What great looks like

Experimentation (iteration with customers) is used to solve all types of problems, including product and service development, process and workflow improvements, and business model questions.

The team is rigorous about forming hypotheses and validating them with customers as a core and widely practiced engineering and business discipline.

The team is able to switch in unison between diverging and converging and maintain a rapid rhythm to enable consistently fast iterations.

A smooth funnel shape of iteration is maintained throughout the product development process.

The team expends minimum effort to produce rapid prototypes at multiple levels of fidelity, appropriate to the scope of the iteration funnel.

The team intentionally keeps multiple solutions alive through several iterations until an optimal solution emerges.

The team and its leaders recognize that failing fast enables a more efficient product development process.

Team culture tolerates ambiguity and healthy risk and remains nimble in changing market conditions.

Management actively discourages solutions that don’t emerge from an iterative design process involving multiple alternative designs and customer feedback.

The team has processes, schedules, and tools in place (including an A/B testing infrastructure) that allow time for and support iteration.

Typical roadmap

Basic Experimentation is done ad hoc in pockets of the organization. A team consistently iterates ideas to improve them over time but rarely compares multiple viable solutions to a given problem. The team may revisit decisions or repeat debates because of a lack of either consensus or customer feedback that indicates which solutions are better. The team has difficulty abandoning ideas that don’t test well. Most ideas and decisions come from a select set of team members who have a reputation for being visionaries. The team follows the visionaries’ lead and does not feel much need for customer feedback to inform iterations.

Emerging A team tests multiple solution options at regular intervals. The team uses low-fidelity methods to quickly build experiments and test proposed solutions with minimal investment. Experimenting with multiple prototypes for all significant investments is standard operating procedure for the team. Customer feedback is consistently used to inform decisions. Management regularly probes to understand how many solutions have been considered and embraces churn in the design phase because managers know that this diminishes much more costly churn during the implementation and stabilization phases and increases the likelihood of a successful release. The team is willing to abandon ideas that don’t test well to focus on ideas that do. There is a system in place for A/B testing.

Exemplary A team tests new solution ideas with customers often and regularly has multiple experiments in progress at any time and at many different levels of fidelity. The schedule allows ample time for iteration, and tooling is in place to support experimentation. The team uses experimentation fluidly to solve product problems as well as to iterate the business model, marketing plans, and process improvements. Failing fast is rewarded and carries no stigma. The team is able to scale up the volume and speed of experimentation and user feedback as needed. The team and the team’s managers are very comfortable with ambiguity and allow projects sufficient design time to ensure that the team converges on the best solution. The team is driven by quality rather than by time.

good product and a delightful product is hugely dependent on the final fit and finish. These are often the issues that are not fixed. “We’ll get to them next time,” but next time never comes around because of other priorities. Get it right the first time.

Customize Scenario-Focused Engineering to your organization

Every organization is different. What worked for us at Microsoft may not work in your organization. Similarly, things that we tried that didn’t work because of our scale, culture, and even the type of experiences that we deliver may prove to be breakthroughs in your environment.

Don’t even think about throwing out everything your organization does and replacing it with the tools and methodology described within these pages. Use the best of the tools, processes, and mindsets that you already have to build on. The culture change that I’ve described will be much easier if you can fold it into how your team or organization already works. You might just pick two to three things that you want to focus on for a period of time.

This chapter discusses the importance of driving change top-down and bottom-up. In fact, make sure that you are looking at all levels of your organization and identifying key influencers and key partners. Again, this will vary based on your organization. If you can get buy-in from a leader at the top, that may be critical to making the changes you need.

One of our senior vice presidents asked to have a regular meeting to review a scorecard of the most important scenarios that the organization was focused on. There were endless debates and an evolution of what scenarios to include, and more importantly what scenarios not to include. There were even more debates on how to measure each of the scenarios. A key part of driving culture change was getting the organization to embrace the fact that “red” on a scorecard was okay. At the end of the day, the meeting was not the target. The meeting forced a set of hard discussions at every level of the organization and was a key to getting the culture change to happen.

In organizations where there isn’t support at the highest levels, you can still be successful in building a grassroots movement that has a huge impact on your customers. But don’t forget about middle management. If you can get middle management onboard with the culture change that you want to drive (or, even better, if you are middle management), you will have a much better chance of influencing the people above you, the people below you, and all of their peers.

Also, don’t forget to make use of the skills and perspectives from across your entire organization, depending on the size and role specialization. If you are an engineer and lucky enough to have a design and design research (or user research) team, they have years of training in most of the topics described in this book. If you are a designer or a design researcher, go find your kindred spirits in engineering.

Finally, and maybe most importantly, don’t forget to apply what you learned in this book to rolling out Scenario-Focused Engineering in your team. Who is the customer? In this case, your team and your organization. How do you get to deliver great experiences that customers love? Iteration. Iterate on the process. Experiment and try new things.

Even as this book was going to press, we tried to cram in new things we had learned. Maybe you find that writing a SPICIER scenario is great for your organization. Or maybe there is a better way to frame the problem. In certain groups we now use scenarios alongside a short list of jobs that the product or experience is meant to accomplish with specific metrics that we can map to how well we need to deliver on those jobs to exceed user expectations. By the time you pick up this book, we will probably have evolved our thinking and best practices even more. Look for ways to evolve the concepts within this book and adapt them based on you and your team.

Be patient and don’t give up

Creating experiences that customers love is very hard work. Changing the culture of an organization to be successful at creating experiences that customers love is just as hard, if not harder.

When we started working on rolling out Scenario-Focused Engineering across a large organization at Microsoft, we started with a team self-evaluation. This was a self-assessment for a group of leaders to rate how mature they were in becoming a Scenario-Focused Engineering organization. It was a 5-point scale. There was general consensus that the team was fairly mature in many ways with respect to designing for customers, but we rated ourselves at about a 2 to 2.5 on the scale. A couple of years later I gave a talk on lessons learned within our team and we rated ourselves about a 3.5.

Over those few years we had presented the workshop to thousands of employees and driven a shift in the organization. Teams were more focused on their customers, more focused on end-to-end thinking, and there was significantly more focus on the details. We had a lot to celebrate. But at the same time we had (and continue to have) work to be done.

This is not meant to scare you away from this journey. This is meant to simply communicate that this is a long path. I can say personally that being a small part of the culture change at Microsoft has been one of the most rewarding parts of my career. If you take this one step at a time, I hope that you will also find the same rewards. If you do nothing but improve how you personally design a customer experience, you have made a difference for your customers. If you can bring your team along, then you start to have an exponential impact.

Along the way, look for people to help. I mentioned the opportunity to work with so many folks across Microsoft, many who contributed to this book, and many more who contributed to this effort. These individuals were more than a group passionate about driving this change at Microsoft. They were a support network. We’ve bounced ideas and challenges off one another. We’ve leaned on one another when it looked like we weren’t making any progress or when it felt like progress was glacially slow. Find your own support system. Find those people who are as passionate about delighting customers as you are. Heck, buy them a copy of this book or at least lend them yours. Say “Yes, and . . .” a lot. Keep learning. Be patient and enjoy the journey. And most importantly . . .

Go forth and delight!