Improving the Test Process: Implementing Improvement and Change - A Study Guide for the ISTQB Expert Level Module (2014)

Chapter 5. Selecting Improvement Approaches

In the two previous chapters, a number of approaches, models, methods, and techniques were introduced for test process improvement. In fact, a very large number are available to choose from. Within these preceding chapters, some comparisons between the various approaches and models were made. This chapter discusses a set of guidelines for selecting the appropriate approach for test process improvement within a project or organization. References will be made to the comparisons and comments made regarding applicability of approaches and models in the previous chapters. Indeed, selecting the appropriate approach is not an easy and straightforward task for a test process improver. However, failure to select the correct approach may imply that the test process improvement project will fail before it has even begun. It is indeed a critical success factor with huge impact. In this chapter, we discuss each of the reasons for applying process models, content models, and analytical approaches and, finally, reasons for applying a mixed (or hybrid) approach. To conclude the chapter, we’ll look at how analytical approaches are incorporated into the models we covered in chapter 3.

5.1 Selecting Test Process Improvement Approaches

Syllabus Learning Objectives

As stated, this chapter is all about selecting the appropriate test process improvement approach for a project or organization. The guidelines described are provided to support this decision process. In addition to these guidelines, the critical success factors described in chapter 9 will influence the decision for a certain test process improvement approach. The decision about the approach is typically one that is made by a Test Process Group (TPG), steering committee, or other management group (see section 7.1.). The test process improver often prepares a document describing recommendations on the approach to be applied as an input for the decision. If the result of the decision taken (i.e., the test process improvement approach) can be generally applied to the organization, it will typically be documented in a quality or test policy.

The guidelines presented in this chapter should by no means be taken as a set of mandatory requirements or unbreakable rules. They are only guidelines and should therefore be combined with knowledge about the specific project or organizational situation and, of course, with common sense.

5.2 Process Models

Reference models in general provide a body of information and best practices that form the core of the model. In the case of process (reference) models, a predefined scale of process maturity is mapped onto the core body of process information. In chapter 3, we discussed the software process improvement models CMMI and ISO/IEC 15504 and test process improvement models TMMi and TPI NEXT. In general, these (test) process models are best applied when the conditions in the following sections are met.

A Test Process Already Exists

With a test process already in place, a test process improvement model is often used to identify strengths and weaknesses. Based on an assessment, test process improvement models typically provide a list of recommendations where the current test process can or should be improved. The test process model is thus used as a process reference. Note that the test process models typically are not only useful for providing recommendations on where improvements can be made, they are often also useful for establishing test processes. To a certain level of detail, most process models also provide sample processes.

Comparisons or Benchmarking is Required Between Similar Projects

Since process models are basically reference process models, they can be used to assess a project. Process models typically provide an assessment framework with checklists and scoring rules. Using this assessment framework, you can assess projects and compare them to each another. This is typically not done to decide which project is more or less mature but much more to profit from mutual learning. If, for instance, one project has a higher level of maturity in test planning than another, the less mature project may benefit from the other project’s test planning practices. When you work in this way, internal cross-project learning is stimulated, which is often a highly practical way to establish improvements. An additional benefit is that the level of resistance will typically be low because these practices have been shown to work in your projects.

Compatibility with Software Process Improvement Models is Required

Many development organizations use a software (or system) process improvement model such as CMMI. Launching a totally independent and different test improvement program in such environments is usually not the best option. Most often, especially when a software process improvement program is well established, it is best to not reinvent the wheel. Managers who finally understand what software process improvement is all about like to see how a test process improvement model compares to the model they are familiar with. There are many instances where TMMi was selected as the test improvement model because it is highly compatible with the popular CMMI. When trying to achieve CMMI level 3, many organizations discover that testing is only marginally addressed in the model and end up looking for ways to improve testing while still being compliant to the requirements stated by the process areas Verification and Validation. This is where TMMi is strong; achieving TMMi level 2 will automatically fulfill the requirements for the CMMI process areas Validation and Verification (with the exception of peer reviews).

Company Policy is to Attain a Specific Maturity Level (e.g., TMMi Level 3)

Especially in large multinational organizations, having a meaningful quality or test policy across all development sites and test organizations is difficult to achieve. The concept of a common maturity level to be achieved is very tangible and concrete. Driven by various business objectives, many large organizations have used this concept. An example is Philips Consumer Electronics (nowadays called Philips Lifestyle), where a number of years ago all development sites across the globe were required to be at least CMM or CMMI level 3. This was documented in a company-wide quality policy. If situations like this exist, then a specific project or organization has, of course, no other option than to follow this directive.

A Well-Defined Starting Point with a Predefined Path of Improvement is Desired

This is a very common reason for selecting and using process improvement models. In fact, we believe it is probably the most important reason. Most organizations like process improvement models because they make it easier to do process improvement when, based on an assessment, they can find out where they are and a road map is more or less provided for the next steps to take. The progression from one maturity level to another is an integral feature of process improvement models, which gives them their predefined “prescribed” character that many users like. As an example, the TMap approach [Pol, Teunnissen, and van Veenendaal 2002], being another example of a test content-based model, was initially developed and documented in a book of over 500 pages. Many testers liked it, classes were popular, and over 10,000 copies were sold. However, the main comment the authors received was, “I don’t know where to start.” TPI [Koomen and Pol 1999] (nowadays superseded by TPI NEXT) was established primarily for this reason, and it supported organizations in prioritizing all these best practices within TMap (now superceded by TMap NEXT). The organizations that were already using TMap were provided with an assessment tool to define a starting point and thereafter a predefined path of improvements (and therefore TMap best practices) to follow. Many organizations that do not use TMap have since adopted TPI NEXT as their process improvement model.

A Measure of Test Maturity is Needed (e.g., for Marketing Purposes)

Although there are other reasons for the need to have a measure of test maturity, the most common one is marketing. There are many organizations who want a maturity “label” for their organization to show to the external world how well they are doing things. Typically, companies that are offering services such as (test) outsourcing are interested in this. However, in practice this has often led to so-called level scoring. The content and improvements are not (that) important, but achieving the maturity level with minimum effort is. Therefore, if an organization is advocating a particular CMMI or TMMi maturity, it is always recommended to ask the question, Who has determined their maturity level? This question is especially relevant where a high level has been determined (e.g., 4 or 5). In general, only when the assessment has been performed by an accredited lead assessor following a formal assessment process can the claim for a certain maturity level be substantiated. Unfortunately, there are many so-called level scoring organizations out there, some of which give process models a bad name. If you need a measure of test process maturity, look for professional and objective support, either from a reputable external source (e.g., a consultant) or an experienced testing services company. Avoid self-assessment for determining your maturity level; it is biased and unreliable.

Process Models are Respected and Accepted in the Organization

This factor can go both ways. We have been in organizations that had bad experiences using models such as CMMI and therefore were reluctant to use a formal test improvement model. A commonly heard comment was, “We do not want another CMMI!” On the contrary, in organizations that were already successfully applying a software process improvement model, introducing a model specifically for testing is often much easier (although the statement “yet another model” is occasionally also heard). Therefore, timing the introduction of a new model is critical. Generally speaking, in organizations where process models are respected and accepted, the introduction of test process improvement models is much easier and often successful.

Once a decision has been made to select a process model (among others, based on the conditions described in this section), the next question that comes to mind is, Which process model should be used? In such circumstances a list of characteristics that a thorough test process improvement model should adhere to is highly useful. Fundamentally, a test process improvement model should provide three essential benefits:

Clearly show the current status of the test process in a thorough, structured, and understandable manner.

Help to identify where improvements to that test process should take place.

Provide guidance on how to progress from the current to the desired state.

Section 3.1.1 describes a number of specific characteristics that can help us understand the aspects of a test process improvement model that can generally be considered as “desirable.” The areas are covered under the headings “Model Content,” “Model Design,” and “Formal Considerations.”

Another decision that needs to made is whether to use a generic software process improvement model (that also addresses testing) or a specific test improvement model. Using SPI models can be advantageous if only a relatively high-level consideration of the testing process is desired. If details are important (and they often are), a test process improvement model is more appropriate. A strong mover behind the development of test process improvement models was the low level of testing detail provided by software development models (e.g., CMMI). This was often considered inadequate for thorough and practical test process improvement. However, there may still be reasons for an organization to limit itself to using a software process improvement model also for test process improvement (e.g., using a single model reduces effort for learning the model).

Section 3.2.3 provides a comparison between the leading software process improvement models CMMI and ISO/IEC 15504. Section 3.3.3 provides a comparison between the leading test process improvement models TMMi and TPI NEXT. Both sections may provide support when you’re trying to make the decision regarding which improvement model should be used.

5.3 Content Models

In content (reference) models, no concept of different process maturity levels is provided, and they do not prescribe the path to be taken for improving test processes. The principal emphasis is placed on the judgment of the user to decide where the test process is and where it should be improved. Inchapter 3, the content-based models CTP and STEP were discussed. In general, these (test) content model are best applied when the conditions in the following sections are met.

A Test Process Needs to be Established

When an organization is immature and does not yet have a test process, it doesn’t much benefit from doing an assessment against a process model. The result of the assessment—e.g., “there is no test process”—is already known by the organization. The organization is much better off using a detailed content-based model that will help them to establish a test process. Content-based models will typically provide detailed processes and templates that can be used to define a standard process for the organization (or project).

An Assessment to Identify Costs and Risks Associated with the Current Test Process is Needed

Both CTP and STEP state that metrics are an essential part of their model. Having these metrics in place, and assuming they relate to the costs and benefits of testing, will make it possible to quantify costs and risks associated with the current test process. In practice, however, we have not seen many organizations applying CTP and STEP if they also have a full set of metrics. As a result, when metrics are not available, it may also be beneficial to use a test process model for assessing the current test process. Stating the costs and risks will be difficult in a quantitative way for both types of models if metrics are not available. Most likely the answer will be to state the costs and risks in a qualitative way.

Improvements Need to be Implemented in the Order Determined by Business Needs

Where improvements do not need to be implemented in the order specified by TMMi or TPI NEXT, but rather in the order determined by business needs, a content-based model may be the best choice. However, we believe that the test process models TMMi and TPI Next also have the possibility to be (or even should be) applied in a business-driven way. The criterion may be especially applicable when an analytical approach is used—for example, where GQM is applied, starting from business goals and identifying improvement areas in testing based on metrics. In such situations, a content-based model can be used to look for the solution to the identified test process issues.

Tailoring is Required to Ensure That the Test Process Fits the Company’s Specific Context

When using a test process model, this is often perceived as being too strict, leaving the organization with less freedom when it wants to comply with the requirements. Organizations that want to have a higher sense of freedom may prefer to choose a content-based model where they can tailor whatever they want. This may be driven by the organization’s culture (e.g., “mavericks who do not like standards”), but it can also be driven by the specific context of the organization. There may be circumstances where the standard test process models such as TPI NEXT and TMMi are very difficult to apply—for example, with process automation or crowd testing.

Discontinuous, Rapid Improvements and Changes to the Existing Test Process are Desired

When there is no long-term commitment or strategy for test process improvement, adopting a road map based on a process model doesn’t make much sense. Typically, these types of organizations are looking for quick solutions based on day-to-day problems. For example, if defect management is problematic, the organization will benefit from looking at CTP and studying the best practices on defect management. On this basis, the organization can build a tailored defect management process. Some time later another issue may arise, and again the organization looks for a rapid improvement by considering what content-based models have to offer.

The set of desirable model attributes described section 3.1.1 can of course also be used to evaluate the content-based models. In section 3.4., a comparison is made between the test process improvement models (TMMi and TPI NEXT) and the content reference models (CTP and STEP). This comparison can also be of use during the selection process for a test improvement approach.

5.4 Analytical Approaches

Analytical approaches are used to identify problem areas in test processes and set project-specific improvement goals. Section 4.2 introduced Goal-Question-Metric (GQM) and analysis using measures, metrics, and indicators as analytical approaches. These analytical approaches are best applied when the conditions in the following sections are met.

Specific Problems Need to be Targeted

Causal analysis is a perfect mechanism to target specific problems that keep reoccurring or a major failure that occurred in production. For these types of problems, using a model (either process or content based) is typically not the way to go. Doing a detailed causal analysis, identifying root causes and possibly common causes where multiple problems are at stake, is usually what is needed. The specific problems are often context dependent and this is where analytical approaches are strong; they focus on the specifics of the project or organization.

Measures and Metrics are Available or can be Established and Taken

Sometimes a lot of data is already available—for example, from tools such as defect management, test management, or project management that have been in use for a while. With all this information lying around for free, setting up a measurement program and starting to analyze the available data is an easy way forward. The data can be transformed into measures and analyzed to enable recommendations for (test) process improvement to be established. The measurement program will quickly show results, and based on these initial results, it can grow.

Evidence for the Need of a Test Process is Required

In many organizations, testers are convinced the test process needs to be improved and a test process improvement program is required. Often, however, management doesn’t see the need for this and is not ready to give their commitment. In such cases, a small-scale measurement program or causal analysis based on some major problems can be set up. The results from these analytical activities can then be used to gather evidence that a test process improvement program would be beneficial to the organization. Once management has been convinced and is committed, you may convert to a more model-based improvement approach.

Agreement is Needed About the Reasons for Change

This resembles the previous criterion but is different. In this case, management is already convinced that something needs to be done about testing. However, there is discussion about the main business objectives and the problems that are related to them. Where can a test improvement program make a significant contribution, and where is this contribution needed most? To be able to answer these and other questions, an analytical measurement or causal analysis approach can be used. By performing an analysis, the most important reasons for change can be substantiated and agreement between stakeholders can be established. Based on this agreement, test improvement goals can be defined.

The Root Cause of the Problem is not Necessarily Within the Control of the Test Process Owner

There are many problems that we as testers suffer from that we cannot solve ourselves. Examples can be found in the areas of configuration management, requirements management, and project management. Using a test (improvement) model doesn’t help to solve these types of problem. Doing causal analysis that shows that the root cause is somewhere other than the test process can help to raise awareness and make the problem area more visible (e.g., that test design rework is often the result of poor requirements management, or many regres sion tests are needed because of poor configuration management). Of course, such analysis can also be done by means of a measurement program.

A Small-Scale Evaluation or Improvement Pilot is Required/Budgeted For

If there are few resources available for improvements, then using full-blown reference models is often not really helpful. You can focus on only one or two critical issues. That’s all there is time for. Using analytical approaches can reveal these one or two critical issues (in terms of both impact and/or frequency of occurrence), and test improvement recommendations can be identified based on the analysis. The highly focused effort with limited resources can subsequently address the identified recommendation, after which the test improvement program stops (at least for a while).

A Pilot is Required to See Whether a Larger-Scale Investigation is Needed

This resembles strongly the criterion “Evidence for the need of a test process is required” mentioned earlier. Clearly, there is a need to convince management and get their commitment on a test improvement program. A causal analysis could be used to quickly identify a suitable improvement area (one that matters). By doing a pilot on a project and then measuring the results (costs/benefits) of the pilot, you can make a decision with stakeholders on whether a larger-scale investigation or improvement program is needed and can be budgeted for.

There is a Need to Test Hypotheses about the Causes, Symptoms, and Effects of Problems

After having studied the Goal-Question-Metric (GQM) approach in the previous chapter (see section 4.3.), it should be clear to you that analysis is one of the essential steps with the GQM approach. Abstraction sheets can be used to define a baseline hypothesis and identify variation factors and their impact on the baseline hypothesis. So, when there is a need to test hypotheses and gather evidence about the causes, symptoms, and effects of problems and of proposed improvements, clearly an analytical approach (e.g., GQM) is the way to go.

The Organizational Culture Values/Trusts Internally Developed Analysis Based on Local Evidence

This is a selection criterion that is hard to argue with. Sometimes it all depends on the organizational culture as to what is “possible” and what can be used. If there is a large “not invented here” syndrome, then one may face huge difficulties in introducing an external model. Despite taking change management into account, it sometimes just doesn’t work due to the organizational culture. The organizational culture values and trusts internally developed analysis based on local evidence more than externally built models (reference or content). Section 9.2 discusses issues concerning improvement culture in more detail.

Stakeholders from Many Areas are to be Involved Personally in the Analysis

When the context of testing involves many different stakeholders, which is often the case in situations such as testing a system-of-systems, testing multidisciplinary systems, having multiple (possibly remote) development groups, and testing safety-critical systems, then the analysis of testing issues often needs the involvement of many stakeholders (at least initially). In such complex situations, models are often an abstraction of reality that is too simple. Therefore, in addition to these models, brainstorming sessions and other techniques need to be used in a test improvement program. Different stakeholders representing many areas within (or even outside) the organization need to be involved in the analysis, making it appropriate to (also) use analytical approaches.

The Team has Control of the Analysis

Having control but also having the resources, knowledge, and skills to perform an analysis is another reason to choose an analytical approach. Clearly, this is almost like a precondition for being able to select and use analytical approaches. Having control means testing has the authority to organize causal analysis sessions (assuming this is accepted by stakeholders) and has access to the measurement database needed to perform data analysis. From chapter 4 you can see that performing causal analysis and executing a measurement program are not straightforward tasks. Both require specific knowledge and skills, and the possession of these skills is needed by testing to be able to use analytical approaches effectively and efficiently.

5.5 Mixed Approaches

Most widely used test improvement models are used with a top-down approach. However, it is also recommended to apply bottom-up, analytical-based approaches. Top-down approaches have proven to be valuable in establishing improvements. They provide the means to set up a solid test engineering working environment and to establish product quality targets. Experiences in practice have shown significant benefits of this approach. However, organizations also need to apply a bottom-up approach, such as causal analysis of day-to-day problems and the gathering and analysis of process and product metrics. By setting up a measurement-based approach in their organizations, testing will become better understood. Remember, knowledge is an important basis for improvement. The models can be used to define the strategic vision for the next year(s) and provide long-term improvement goals. Bottom-up analytical approaches may well be used to identify some critical issues that need to be solved today rather than tomorrow and thus provide short-term improvement goals.

In practice, we almost never find just a model-based, content-based, or analytical approach being applied in isolation. It may be the case that one of these approaches is leading, but it will most likely be blended with another one. Therefore, in practice, we often have a mixed approach, even if a process model approach is chosen. Here are a couple of examples of mixed approaches, such as the use of analytical approaches within a process model or content model:

Usage of causal analysis during a TMMi test improvement program

Usage of metrics during a STEP test improvement program

More and detailed examples of this are provided in the next section.

5.6 Analytical Approaches and Improvement Models

Applying analytical approaches to achieve test process improvement is considered a valuable practice by all of the models we covered in chapter 3. If you choose to use one of these models, you may be guided to implement analytical measures to achieve improvement, or you may choose to use the model to verify the results of your analysis. In the following sections, we provide some examples of how the models described in chapter 3 are blended with analytical approaches.

5.6.1 Analytical-Based Improvement with CMMI

Software process improvement models, such as CMMI, place considerable emphasis on the ability to perform causal analysis for improving the maturity of the software process. In fact, Causal Analysis and Resolution (CAR) is described in CMMI as a separate process area consisting of the following Specific Goals (SG) and Specific Practices (SP):

Specific Goal SG1: Determine causes of defects

SP 1.1 Select data for analysis

SP 1.2 Analyze causes

Specific Goal SG2: Address causes of defects

SP 2.1 Implement the action proposals

SP 2.2 Evaluate the effect of changes SP 2.3 Record data

Also, measurement is an important aspect in all software process improvement models. CMMI has a dedicated process area on measurement within the Support category. This process area, called Measurement and Analysis, consists of the following Specific Goals (SG) and Specific Practices (SP):

Specific Goal SG1: Align measurement and analysis activities

SP 1.1 Establish measurement objectives

SP 1.2 Specify measures

SP 1.3 Specify data collection and storage procedure

SP 1.4 Specify analysis procedures

Specific Goal SG2: Provide measurement results

SP 2.1 Collect measurement data

SP 2.2 Analyze measurement data

SP 2.3 Store data and results

SP 2.4 Communicate results

In addition to the process area Measurement and Analysis, CMMI defines the Generic Practice “Collect Improvement Information” (GP3.2). Being a generic practice, this must be applied by each process area and deals with collecting measures, measurement results, and improvement information derived from planning and performing the specific process to support future use and improvement of the organization’s process and process assets.

5.6.2 Analytical-Based Improvement with TPI NEXT

The TPI NEXT model considers some aspects of causal analysis at both the “efficient” and “optimizing” levels of test process maturity. Table 5–1 shows the three checkpoints to be achieved that relate to causal analysis.

Table 5–1 Causal analysis checkpoints TPI NEXT

TPI NEXT has a dedicated key area for metrics; especially at the efficient and optimizing levels, the metrics are used for the purpose of improving the (testing) process. TPI NEXT also provides some examples of metrics typically used at the “efficient” and “optimizing” levels. Table 5–2 shows some of the metrics proposed by the TPI NEXT process model.

Table 5–2 Example Metrics TPI NEXT

5.6.3 Analytical-Based Improvement with TMMi

TMMi considers causal analysis as part of process maturity level 5, “optimizing.” At this highest level of test process maturity, TMMi defines the process area Defect Prevention (PA 5.1) and two Specific Goals, one of which (SG1) fully covers the determination of root and common causes of defects. The specific goal is achieved by performing the following three specific practices:

SP1.1 Define defect selection parameters and defect classification scheme

SP1.2 Select defects for analysis

SP1.3 Analyze causes of selected defects

TMMi has many instances where it addresses measurement practices. It has a dedicated process area called Test Measurement that has the same specific goals and specific practices as the CMMI process area Measurement and Analysis. However, note that its subpractices are most often different. Within this process area TMMi provides many examples of commonly used test metrics. In addition, TMMi also has the same Generic Practice Collect Improvement Information (GP3.2) as the CMMI.

However, TMMi also has a process area called Advanced Reviews where peer review measurement results are used to adjust the test approach. Finally, already at TMMi level 2, within the process area Test Policy and Strategy, the specific goal “Establish test performance indicators” (SG3) is introduced to establish a set of goal-oriented test process performance indicators to measure the quality of the test process. The specific goal is achieved by performing the following two specific practices:

SP3.1 Define test performance indicators

SP3.2 Deploy test performance indicators

If you look back at the discussion in section 3.1.3 about the differences between continuous and staged representations in test process reference models, you will recall that models using the staged representation can suffer from the “all or nothing” syndrome. If, for example, all but one of the maturity level 2 goals are satisfied, the model still places the test process at maturity level 1. This can result in an unrealistically negative impression of the achieved test process maturity and could demotivate those striving to make improvements. Having a set of test performance indicators in TMMi helps considerably against this “all or nothing” syndrome. Statements like “We worked all year and achieved a lot, but (according to the model) we’re still where we started!” may result in frustration in the test improvement team and could lead to incorrect management perceptions (“What? Still at maturity level 1 after all that investment? You promised me improvements!”). Hence, we find here another reason to combine model-based approaches with analytical-based appproaches. The defined and established test performance indicators should provide measurable and tangible results, although possibly still being at the same maturity level. TMMi provides us with the following examples of test performance indicators:

Test effort and cost

Test lead time

Number of defects found

Defect detection percentage

Test coverage

Test maturity level

Refer to section 4.4. for an explanation and definition of these and other test performance indicators.

5.6.4 Analytical-Based Improvement with CTP and STEP

Both CTP and STEP are limited when it comes to causal analysis, as described in chapter 4. There are some elements of causal analysis present in both models, but the core functionality of causal analysis is not really addressed. When one looks at both CTP and STEP (and, in fact, most content-based models), their focus is very much on providing thorough test content for their users and not so much on the improvement process itself. The latter is addressed, but only marginally.

However, both models do emphasize the importance of metrics. The STEP model explicitly identifies measurement as one of the activities within its “measure behavior” phase. However, no specific details are provided on this. CTP describes a number of metrics relevant to the critical testing processes (see table 3–27). These may be applied to improving the test process. In its high-level approach to test process improvement, CTP proposes to do the following:

Gather detailed metrics on these critical testing processes.

Recommend improvements based on these metrics and develop an improvement plan.

In summary, one can conclude that content-based models are less blended with analytical improvement approaches than process improvement models such as CMMI, TMMi, and TPI NEXT, partly due to the limited focus of content-based models on the improvement process. The three process improvement models mentioned earlier encompass many of the analytical approaches described in chapter 4.

5.7 Exercises

The following multiple-choice questions will give you some feedback on your understanding of this chapter. Appendix F lists the answers.

5-1 What is considered a valid reason for using a process model to improve testing?

A: A test process already exists.

B: A test process needs to be established.

C: Specific problems need to be targeted.

D: Evidence for the need of a test process is required.

5-2 What is considered a valid reason for using a content model to improve testing?

A: Comparisons or benchmarking is required between similar projects.

B: Compatibility with software process improvement models is required.

C: Improvements need to be implemented in the order determined by business needs.

D: Agreement is needed about the reasons for change.

5-3 What is considered a valid reason for using an analytical approach to improve testing?

A: A measure of test maturity is needed (e.g., for marketing purposes).

B: Process models are respected and accepted in the organization.

C: Tailoring is required to ensure that the test process fits the company’s specific context.

D: The root cause of the problem is not necessarily within the control of the test process owner.

5-4 The usage of causal analysis during a TMMi test improvement program is an example of which of the following?

A: Model-based approach

B: Content-based approach

C: Analytical approach

D: Mixed approach

5-5 Which of the following TMMi process areas is not also related to an analytical approach ?

A: Test Measurement

B: Advanced Reviews

C: Test Policy and Strategy

D: Defect Prevention

E: None of the above