Improving the Test Process: Implementing Improvement and Change - A Study Guide for the ISTQB Expert Level Module (2014)

Chapter 9. Critical Success Factors

Looking back at the chapters so far, you can readily identify several factors that could be called critical to the success of a test process improvement program. You have to choose the right improvement approach to achieve a particular goal, you need a solid process for implementing improvement, you need the skills and organization to implement improvements. And last but not least, you need to consider the process of change and what it means to those involved.

This chapter takes a look at those factors we consider critical to success. There will be some familiar factors mentioned, but some new aspects will also be discussed. The critical success factors are discussed in two distinct sets.

Section 9.1.1, “Getting Started,” looks at the first set of success factors. They are primarily related to the initial phases of an improvement project and can be linked to the initiating and diagnosing phases from the IDEAL improvement framework (section 2.4.2).

Section 9.1.2, “Getting the Job Done,” includes the second set of success factors, which are related to the implementation phases of an improvement project.

9.1 Critical Success Factors

Syllabus Learning Objectives

9.1.1 Getting Started

Several factors are often found in the initial phases of an improvement program that can be considered critical to success [Broekman and van Veenendaal 2007]. Failure to recognize these key factors leads to an immediate and considerable risk for executing a planned improvement program. As indicated in the introduction to this chapter, the key factors are all related to activities in the initiating and diagnosing phases of the IDEAL model. Several key factors are described below:

Clear, Measurable, and Realistic Objectives for the Improvement Process

It is necessary to clearly define and record the goals of an improvement program. Why do we improve our test process? As indicated in chapter 6, the goals of the change program are determined in the initiating phase and are subsequently developed or made concrete in the establishing phase. The goals must be known to everyone involved. What is the direction the organization wants to take and why? All this should be recorded in the test policy, which is derived from the quality policy and the organizational policy (section 6.1.2). The test policy globally describes the organization’s philosophy about testing. In TMMi, the details of the test policy are elaborated upon within the process area Test Policy and Strategy.

We have unfortunately seen too many improvement initiatives run into problems because of inadequate goals. The problem is usually that inexperienced people simply want to get right on with the assessment task and don’t want to “waste time” talking to stakeholders about goals. In this case, difficulties usually start to occur later in the improvement program when stakeholders need to be convinced of improvement recommendations. They will simply ask why, and more often than not, no convincing answer can be given—all that assessment work gone to waste; what a pity.

Management Commitment and Sponsorship Are Available

Is quality important enough to the organization? How does the organization deal with a system of inferior quality at a milestone? Is the organization driven by budget, deadline, or quality? The answers to questions like these show the actual management commitment with regard to testing and quality. Without sufficient management commitment and an explicit sponsor at the management level, an improvement process is highly likely to fail. Acquiring management commitment is discussed in the “build sponsorship” activity in the initiating phase (section 6.2.4).

Critical success factor An element necessary for an organization or project to achieve its mission. Critical success factors are the critical factors or activities required for ensuring success.

The Need to Improve

Before employees in an organization are willing to contribute to improvements, they must feel the need to improve (people frequently talk of “pain points” when referring to such needs). For example, this need can be to reduce the large number of defects when a system goes into production or to reduce the time required to run a test project. The need to improve must be clearly and frequently communicated, preferably by management. Improvement goals need to support the defined need to improve.

Test Improvement Organized as a Formal Project

During the “charter infrastructure” activity in the initiating phase, the organization that will support the improvement process is set up (section 6.2.5). It is highly recommended to choose a project structure with elements such as assignment, steering group, project lead, responsibilities, planning, milestones, deliverables, and reports. Change programs are often complex for several reasons. Our experience shows that creating an official project structure contributes to the change being taken seriously. As a defined project, the effort becomes visible in the organization, and it’s clear that the employees who contribute to the improvement program don’t just do it as an “extra” but rather as an assigned project.

People Involved Have Sufficient Time Scheduled for Participation

The recommendation is to discuss the resources available for the change during the “charter infrastructure” activity instead of delaying it until the “develop approach” activity in the establishing phase. It is important that management realizes that choices must be made. For example, allowing employees four working hours a week to spend on improvement may sound reasonable, but that often proves not to work in practice. When the employees working on the (test) project are pressed for time, they often do not have (or are not given) the extra four hours to spend. In addition, a small amount of time can easily become eaten up by the changes that occur between finishing one activity and starting another. It takes time to get back into the current improvement task; apart from the obvious loss of precious time, this can also result in repeating tasks by mistake or even losing momentum completely. An alternative can be to take a number of people off projects and assign them to the improvement program for a minimum of three days a week. This strategy leads to focus and progress and therefore to a timely delivery and measurable results.

Ambitions Mapped to the Maturity of the (Development) Organization

In the “characterize current and desired states” activity of the diagnosing phase (section 6.3.), it is important to look at the test processes as well as the other processes on which testing strongly depends. Without a minimum form of maturity in these processes, improving the test process will prove very difficult, although not impossible.

The principal areas of the development process to look at are project planning, configuration management, requirements development, requirements management, and the release process. Without a mature project planning process, establishing a thorough test planning process will be very difficult, due to the many dependencies between testing and development activities. When test design techniques are applied, the quality of the test basis (e.g., requirements) is of major importance. Any changes in the test basis must also be made known to the test team. Absence of configuration management often leads to nonreproducible findings or findings that “suddenly” reappear in a later release.

In our experience, one of the most common problems that influences testing relates to the process used to bring new releases of software into productive use. Typically, releases are scheduled on a quartely or monthly basis and may be planned as a mix of major and minor releases. A common factor of many releases is failure to limit the number of new requirements and changes to an amount that can be developed and tested before the planned release milestone. We have noted a tendency to put too many new or changed requirements into a release, which then places strain on the development and testing process. Faced with oversized releases, corners are often cut in the test process such as, for example, reducing the time spent on test planning. A test process assessment may identify test planning as a weak point, but the problems cannot be solved in the test process; better release planning is required first.

9.1.2 Getting the Job Done

The previous section described how that important first hurdle of getting started must be overcome. For good results, momentum must be upheld in the program and everyone involved must stay focused. This section describes a number of important factors that help to determine the success of a test improvement program.

Clear time frames for improvements and length of feedback cycles are defined. These are not too large so that momentum can be maintained.

Process improvement is in essence a long-term process. However, to retain motivation, it is important to score so-called quick wins early in the process. The supporters within senior management also want to be shown evidence of regular success to justify their support for the improvement team. Whenever success is achieved, for example during the pilot, this must be communicated clearly and repeatedly (“do good things and talk about them”). When you describe success using examples that are specific to the organization or its individual employees, the enthusiasm for the improvement program generally grows.

We have found regular project meetings to be a good opportunity to add an extra agenda item for reporting on test process improvement progress. Care has to be taken not to exaggerate successeses and to be honest; people will grow tired of success stories that have been polished up just so there is something positive to report.

Level of resistance will depend on the success or failure of previous improvement efforts. Manage resistance and perform marketing.

Initially, most people naturally resist change. Trying to push through that resistance often has the opposite effect. It is better to try to convince them by marketing the improvement program and carefully noting the stages that people go through when they are affected by change (see chapter 8). For example, publish the achieved results or stories from people who have experienced the positive effects of the improvements in a regular newsletter. Whenever resistance is detected, it is advisable to talk about it with the employee concerned. In this conversation it is important that the person who is managing the change also listens to the objections of the employee. Resistance is often a result of uncertainty about the need and objectives of the change project, unfamiliarity with the approach of the change project, or uncertainty about the employee’s own situation.

Use existing practices if they are already available; don’t change for the sake of changing. If something that is available is not being used, first investigate the reasons.

Even though it seems apparent that a lot needs improving, this does not mean everything is wrong. There have been many unsuccessful attempts to improve; sometimes some test projects are executed in a more controlled way than others are. It is likely that several employees have had good ideas, but they have not succeeded in deploying them for one reason or another, which may have been outside of their control. Involve those people and encourage them to have ideas. Resistance is less when changes are derived from one’s own ranks.

During assessment interviews, we tend to ask people the question, “What do you think needs changing here?” Often you will get insight into why those good ideas sometimes fail.

Involve external consultants as needed for specific knowledge and skills, for example, but do not let them take full responsibility for the improvement project.

External consultants can positively add to a project with their knowledge and experience. When they are involved as a dedicated part of an improvement project, they can keep it going despite other activities being scheduled for the organization. However, the external consultants are there to advise, and they must not make decisions on company procedures; this must be decided by the employees of the organization. This is why in every improvement program internal employees should be the primary resources involved, supported as needed by external consultants. In addition, improvements must be anchored in the organization so the effort and momentum continues after the consultant leaves.

We have seen organizations become overdependent on external consultants. Remember, a good consultant puts customers into a position in which they can move forward and continue improving without external support. Consultants who make themselves indispensible might be popular with their employers, but frequently they are not providing long-term benefits to their customers.

Guard consistency

Be sure to maintain consistency between the many individual parts of an improvement program. The total sum of all improvements must be integrated and work as one.

Other factors

Of course, there are numerous other success factors based on many practical experiences (see the case study in section 9.1.3). Some of these are not described in detail here but are worth mentioning:

Clear, measurable, and realistic improvement targets for every cycle

Process ownership identified and organized

Control and monitoring of all steps in the change management process (see chapter 8)

Test professionals involved when defining and implementing the improvements

Other stakeholders involved where problems lie outside the testing discipline (e.g., quality of specifications, change and release management processes)

Stable project team that works well together and buys into the change/vision

Tools to support and/or enable test improvements considered (see section 2.5.4)

Available knowledge and skills of people involved considered. This covers not just testing in general but also areas related to the improvement process and skills for the improvement approach(es) to be used (e.g., specific model, analysis techniques).

Human factors, such as learning styles, personality types, and attitudes considered

Awareness of external standards, which may be mandatory (e.g., FDA for the medical industry)

Overall process and terminology defined up front to ensure that the various components of the improvement strategy are aligned and part of an overall framework

Relationships built with all affected stakeholders (e.g., software process improvement officers, quality assurance and human resources departments)

Internal approval and/or regulatory processes obeyed

Alignment with other improvement initiatives ensured

9.1.3 Critical Success Factors: A Case Study

One of the authors was doing a consultancy assignment a few years ago at an organization that developed highly complex multidisciplinary safety-critical systems. It was probably the most successful organization he had been involved in when it comes to setting up a test center of excellence and implementing test improvements. We tried to analyze what really made the difference. Many things came to mind, but there were some essential success factors that stand out:

As always, trying to improve or manage an organization using just a top-down approach doesn’t work.

A combination of top-down and bottom-up is usually most successful.

During our analysis, we came up with two top-down and two bottom-up critical success factors.

Top-Down Approach: Test Policy

A test policy, if well written and with the right level of detail, provides direction on the improvements, states the values of the testing organization, shows how testing adds value to the overall business objectives, and defines some key test performance indicators. It is not without reason that the TMMi model has Test Policy and Strategy as one of its main process areas already at maturity level 2. At this company, the test policy was established in a way that added value (as it should do). It was written on one page only, well distributed, and displayed on walls near the coffee machine and copier to ensure that everyone was aware of it. The test policy was then re-discussed on a yearly basis in a brown paper session with managers and all test managers. Are the values still correct? What are the main problems we encounter? What should we focus on this year? How do we add business value? These are examples of questions that were discussed in the session, with the result being documented in an updated test policy.

Top-Down Approach: Leadership

Whenever we have been really successful in our careers, there has almost always been a manager that made the difference. Many managers immediately come to mind when we think of successful projects. Somehow it is sad that success should depend on one person only, but real leadership can make a difference. Leadership has been described as the “process in which one person can enlist the aid of others in the accomplishment of a common task.” In this organization, a real leader was encountered. This was someone with a strong personality, who was there for his people when things got rough, was able to motivate them when needed, had a vision toward the future of the company and the role of testing, but also, and probably above all, was a pleasant, honest human being with integrity. Leadership is much more than providing resources. Real leaders just make the difference. Find them!

Bottom-Up Approach: The Test Managers’ Meeting

One of the things we learned from Watts Humphrey many years ago is that critical stakeholders during change management are those directly responsible for the project, such as, for example, the project manager and test manager. They tend to be directly confronted with delays due to new and innovative processes, while their job is to deliver a quality product on time. Thus they are usually not too enthusiastic when it comes to process improvements to their projects. At this company, although a person was assigned as a process improvement facilitator, the test managers’ meeting was the main driver of any improvement. At their weekly meeting, the managers would discuss, in addition to (test) project issues, improvement ideas, the status of improvement actions, and so on. They were the ones to decide whether an improvement had added value or not, thereby ensuring that the improvements were practical. In addition, using the test policy as their reference framework, they maintained a strong focus on the defined objectives. When a decision was made to define and implement an improvement action, a volunteer was sought among the test managers who would drive the specific action. Note that the improvement actions were not process oriented only but also concerned tasks like setting up and organizing a training session, tool selection, and developing a supporting spreadsheet template. As a result, little or no change management actions were required for the test managers, who are often “difficult” stakeholders. They were now in charge of doing things they perceived had added value to the projects, either short or long term.

Bottom-Up Approach: Everyone Involved

Although the test managers were driving the improvements, they would not necessarily do it all themselves. Normally they would take the action back to their team, discuss it, and make it a team assignment. In this way, all testers became involved in building a high-quality test organization and test process improvement. There was almost no need for change management; it was already part of their test process. Of course, implementation could still be difficult for many reasons, but at least there was little to no resistance and people had a positive mindset toward the changes. Important in this context was also a monthly test knowledge sharing meeting where testers and test managers would present their personal experiences regarding a new template, a test design technique, using a tool, and so on. Again, the test improvement coordinator was acting as a facilitator rather than a manager. A test process was being built that was already everyone’s process.

You may notice that the critical success factors described earlier are strongly related to change management and people issues. We believe people do make the difference. Striving to become a testing center of excellence using just a process orientation will fail sooner or later. Real leadership, a clear policy, mission and direction, and a focus on change management and people issues can make it happen. But remember, changes do take time!

9.2 Setting a Culture for Improvement

Syllabus Learning Objectives

9.2.1 Defining “Improvement Culture”

When you start a new assignment at an organization, it doesn’t take long to get a feel for their improvement culture. Are people open to the idea of improvement or is their enthusiasm somehow blunted? Do they respond to initial questions like, “Where do you see your test process two years from now?” with an enthusiastic flood of ideas and objectives, or do you see people simply shrugging their shoulders and being indifferent? Body language and responses given to general questions about the “way things are done here” are good “gut feeling” indicators for improvement culture.

When defining improvement culture, we often talk about this kind of “feeling,” but there are also more specific aspects and factors to consider that influence an organization’s overall approach and attitude regarding improvement. Understanding these aspects helps us to grasp the sometimes elusive concept of improvement culture and enables us to shed some light on possible problem areas. The following section describes these factors and their contribution to improvement culture.

9.2.2 Aspects of Improvement Culture

Six principal factors are identified in this section. Each of them can have a decisive influence on the culture of improvement and, ultimately, whether test process improvements will be successful.

Management Culture

How does management manage? The answer to this varies from company to company and even from country to country. Many organizations apply a “command and control” style of management where decisions are made at the top and handed down through the hierarchy of management until they reach the people who are instructed to perform specific tasks. In some countries, there could be several layers of management, with each layer having the freedom to decide on areas within their responsibility before handing over decisions to the next level down the hierarchy. In other countries and organizations, we may find a patriarchal style of management in place. The top boss decides what to do and the rest follow without changing or refining the decisions.

Management influences improvement culture by affecting the buy-in that people have with regard to suggested improvements. Achieving good buy-in generally means involving people at various stages in the decision making process and adopting a more consultative approach. Companies that practice strict command-and-control management styles often suffer from low levels of buy-in from the very people required to implement the changes. What could go wrong here? In our experience, the following situations can arise:

People implement measures that they know are going to be ineffective “just because the boss says so.”

People become reluctant to show initiative.

People avoid giving feedback to management, especially if it is not well received (“just do what I say”).

Management can negatively influence improvement culture by introducing too many changes too quickly. Referring back to the Satir model we described in section 8.2.2, we find in some organizations that management sometimes has no real appreciation of the chaos phase that all people go through when affected by a change. Management wants to see results quickly when they instigate changes. If productivity falls temporarily as a result of a change (and we know it will), management may see this as sign of failure. Frequently, all that is needed is to give people some more time and support to help them out of their chaos (see chapter 8). Instead, some managers will introduce further changes to correct the previous “mistakes,” which unfortunately only leads to an extension of the chaos and the potential for making bad “corrective” decisions.

A management culture that demands instant results is highly likely to create a negative improvement culture within an organization. It will be focused on short-term quick wins instead of long-term organization improvement and often sets targets that cannot realistically be achieved by staff within the time and budget made available. As test process improvers, we can sometimes recognize the signs of this when people say they are “tired of constant changes” or asked to “achieve the impossible.” Recognizing this will help us understand situations (and maybe explain that shrug of the shoulders), but actually changing the situation may require management changes that are frequently beyond the scope of the test process improver. We should tactfully point out these issues to management stakeholders and ask them to consider their impact on the organization’s culture of improvement.

Geographical Location

Sometimes geographical location can have an influence on the specific decisions made within a test improvement program. As test process improvers, we may hear statements like, “This is how we do things here” when, for example, referring to a particular analytical approach. The “here” part of such statements often refers not simply to the organization or project but to geographical location. For example, some software process improvement models are favored in certain countries (e.g., CMMI is used in the United States and Asian countries more than ISO/IEC 15504 is). If we ignore these aspects, we may be making proposals that go against the improvement culture in that country.

Attitude Toward Improvement

Test process improvement experiences within organizations and projects shape the goals, policies, and strategies that become ingrained into an improvement culture. This applies not only to positive experiences (a successful approach to rolling out changes to the test team) but also to negative experiences (inability to define realistic goals using a particular metric). Experiences gained by others (e.g., in a different department) may also influence attitudes toward improvement by way of anecdotes, tweets, and experience reports.

Our experience is, unfortunately, that the negative experiences tend to dominate the positive ones in shaping attitudes (bad news travels fast). It is also common to find that the views of one or two respected individuals can influence the attitudes adopted by whole project teams or even organizations. These are important people to identify if you want to understand or modify attitides (the roles identified by Malcom Gladwell [Gladwell 2000] can be particularly useful in helping to identify them).

Organizational Relationships

Relationships between different parts of an organization can significantly influence improvement culture. These relationships may be formed either by spreading experiences via, for example, company-wide user groups or communities or by merging different companies into a larger whole. Generally speaking, the exchange of different experiences can (and should) have a positive influence on improvement culture by challenging old established practices and opening up different parts of an organization to fresh views and approaches. Skilled test process improvers can harness this positive cultural effect by encouraging cooperation between different organizational units.

Organizational relationships can also exert a negative influence on improvement culture if the so-called “not invented here” mentality is present. This kind of mentality can block good practices and ideas from being adopted from outside a project or department. The basic assumption behind this commonly found behavior is “if we didn’t think of it, then it can’t be any good.” We have experienced this kind of protectionist behavior a number of times. Without wishing to take the role of psychologists, it would seem from our experiences that the effect is rooted in one or more of the following beliefs:

Concern that another organization may be taking over

Reluctance to admit that another organization has a better practice

Belief that your own organization is so special that only the ideas developed within it can possibly work

The negative effects on improvement culture resulting from organizational relationships can be very subtle. Test process improvers must be aware that their recommended organizational process improvements may fail to spread due to these influences and, where possible, suggest measures to break down organizational barriers (e.g., establishing communities to consider organizational issues and exchange ideas).

Life Cycle Model

The life cycle model used within an organization or project can have a decisive influence on the way in which test process improvement is approached.

Take, for example, iterative or agile life cycle models (see chapter 10). The improvement culture here is closely aligned to the iterations and can be characterized as follows:

Improvement is considered at frequent intervals (e.g., at the end of a sprint

when using SCRUM).

The scope of the improvement is often limited to the cycle (e.g., a sprint) that has just taken place, the aim being to improve a little but often.

Improvements are closely coupled to the problem, and waiting times for improvements to be implemented are minimized.

Compare this with the improvement culture that develops when using a sequential life cycle model such as the V-model. This shows the following characteristics:

Improvements generally are considered only occasionally, such as at the end of a software release cycle or at the end of the project itself.

The scope of improvement is broader and there is more opportunity to address organizational improvements.

The principal benefit is not focused on resolving issues in the current release or project, but for the next release (often with a different release manager) or the next project (often with a different project leader). So the culture is more attuned to making things better for your successors rather than for yourself (although it’s of course possible that the same leader and team members will be in the next project).

Test Strategy

The test approach being used will influence the overall culture of improvement by making some kinds of improvements more acceptable than others. If your testing approach is strongly based on the use of tools and automation, for example, you are more likely to get improvements accepted that have a technical emphasis and improve the automation strategy (e.g., introducing data-driven approaches) than by improvements of a more procedural nature that affect mostly manual testing (e.g., a new checklist for reviewing requirements specifications). If the testing approach adopted tends to be repeated from project to project, an improvement culture will develop within the organization that reflects that type of approach. This development will be reinforced if project retrospectives are routinely practiced.

Project retrospective A structured way to capture lessons learned and to create specific action plans for improving on the next project or next project phase.

This is, of course, entirely logical, but inexperienced test process improvers who are unaware of the strong link between test approach and improvement culture must appreciate this to avoid making unacceptable proposals.

Summary

To appreciate the culture of improvement in a project or organization, you must understand people’s attitudes in the context of that project or organization. Considering the preceding six factors is a key success factor in establishing good improvement recommendations and, particularly, in making them stick.

9.2.3 Test Process Improvement Manifesto

An example of an approach is the test process improvement manifesto [van Veenendaal 2008], which echoes the agile manifesto [URL: agile manifesto].

In addition to the improvement process as described in the IDEAL framework and the critical success factors that one needs to take into account during the implementation process, the test process improvement manifesto provides interesting recommendations and identifies several principles that can make a difference in a test process improvement project. These principles and recommendations are derived from an analysis of successful test improvement projects in various domains.

Test process improvement manifesto A statement that echoes the agile manifesto and defines the following values for improving the test process:

–flexibility over detailed processes

–best practices over templates

–deployment orientation over process orientation

–peer reviews over quality assurance (departments)

–business-driven over model-driven. [van Veenendaal 2008]

Flexibility over Detailed Processes

In general, having defined processes supports an organization. Only something that is defined can be improved. It guides new engineers and acts as corporate memory. However, building processes that are too rigorous takes away the “people values.” Good testers have the skills to act based on the context of a problem, and they perceive testing to be a challenging job. Supporting processes are needed, but the employed processes should provide enough flexibility and freedom to testers to allow them to think for themselves and find the best way forward. The ideal is “just enough process.”

Best Practices over Templates

Templates are great, but it is even better to provide examples of how they should be used. What provides more support, a test plan template or three test plan best practices? Experienced testers will choose the latter. When doing test process improvement, it’s important to focus on getting a library of best practices set up as soon as possible instead of overspending on defining templates. The best practices may not be the best in the industry, but they may be the best for your organization. If something better comes along, they can be replaced. This is what supports testing and makes process improvement work.

Deployment Orientation over Process Orientation

Building process is easy; it’s been done many times and there are numerous examples to be found. However, getting the processes deployed and thereby changing someone’s behavior is the hard part. Process improvement is all about change management. Test improvement plans sometimes erroneously focus almost entirely on defining the testing processes. In successful improvement projects, at least 70 percent of the improvement effort is spent on deployment—“getting the job done.” Defining the processes is the easy part and should account for only a small percentage of the effort and focus.

Peer Reviews over Quality Assurance (Departments)

Communicating and providing feedback are essential to project success. It is exactly this that peer reviews, if applied well, do. In principle, quality assurance officers also evaluate documents and provide feedback to engineers, but they tend to focus on conformance to templates and defined processes, partly because they are somewhat distanced from the testing profession. This reduces the value they contribute. Peer reviews, when done by qualified peers, provide pertinent feedback and advice for the given application, which is generally more beneficial than just adherence to a template.

Business-Driven over Model-Driven

Just trying to get to a higher maturity level defined by a model (e.g., from TMMi level 2 to 3) without understanding the business context will sooner or later always fail. The improvement team must understand the business problem in order to determine how to address the improvements. Whatever you do, make sure you know why you are doing it. What is the business problem you are trying to address? What is the test policy supported by management? When addressing a certain practice from an improvement model, there are most often many different ways to comply. The business problem (poor product quality, long test execution lead time, costs, etc.) will determine which one to choose. Process improvement must be constantly reviewed against the business drivers and test policy to ensure compliance.

9.3 Exercises

The following multiple-choice questions will give you some feedback on your understanding of this chapter. Appendix F lists the answers.

9-1 Which key success factor can be achieved by a test policy?

A: The test improvement is organized as a formal project.

B: The ambitions of the improvement are mapped to the maturity of the development organization.

C: Clear, measurable, and realistic objectives for the improvement process are set.

D: Clear time frames for improvements and length of feedback cycles are defined.

9-2 Which activity can help to ensure that people involved in an improvement program have sufficient time scheduled for participation?

A: Discuss within the team and allocate time to those who are not 100 percent scheduled to project work.

B: Inform management of the choices that need to be made regarding more resources or reallocating tasks.

C: Split scheduled work into regular, small chunks of time to ensure that other tasks can run in parallel.

D: Ask for volunteers to work extra time.

9-3 Why is it recommended to use existing practices if already available?

A: The team members may be offended.

B: It is unlikely that everything needs improving.

C: Employees may have had good ideas but have not succeeded in deploying them for one reason or another.

D: Don’t change for the sake of changing.

E: B, C, and D are correct.

9-4 Which statement is false when considering management culture of improvement?

A: Management culture may vary from country to country.

B: Command-and-control styles involve decisions that are handed down through the hierarchy of management until they reach the people who perform specific tasks.

C: Achieving good buy-in generally means involving people at various stages in the decision making process.

D: Companies that practice strict command-and-control management styles usually benefit from high levels of buy-in.

9-5 Which of the following statements is true regarding the impact of external consultants on an improvement project?

A: They can give advice based on their experienec and knowledge.

B: They can make important decisions on the working procedures.

C: Consultants who are “indispensible” to the project can provide long-term benefits to their customers.

D: They must return to the project regularly to ensure that the momentum of improvements is maintained.

9-6 Which of the following factors is a critical success factor that is typically relevant while establishing improvements?

A: Working on both long-term and short-term goals

B: Management commitment

C: The maturity of the development organization

D: Organizing test improvement as a project

9-7 Which of the following factors is a critical success factor that is typically relevant while getting started on improvements?

A: Length of feedback cycles are defined; these are not too large so that momentum can be maintained.

B: Change management.

C: Existing practices used if already available.

D: Sufficient time is scheduled for those involved.

9-8 Which of the following statements is not part of the test process improvement manifesto?

A: Flexibility over detailed processes

B: Deployment orientation over process orientation

C: Responding to change over following a plan

D: Best practices over template

9-9 What is a major benefit for test improvement when SCRUM is being applied?

A: The scope of the improvement is usually the entire organization.

B: Improvements are considered at frequent intervals.

C: Improvements are decoupled from the problem.

D: Process orientation is strong.

9-10 Which of the following factors is not a principal factor that can have a decisive influence on the culture of improvement?

A: Test policy

B: Attitude toward improvement

C: Life cycle model

D: Geographical location