Improving the Test Process: Implementing Improvement and Change - A Study Guide for the ISTQB Expert Level Module (2014)

Chapter 10. Adapting to Different Life Cycle Models

This chapter considers the principal issues relating to different life cycle models and their influence on test process improvement. Failing to understand these differences may result in inappropriate improvement recommendations being proposed and, ultimately, will damage the improvement culture within the project or organization. Those involved will simply lose faith in the value of improving the test process if proposals are “out of touch.”

10.1 Test Process Improvement with Different Life Cycles

Syllabus Learning Objectives

In section 2.5.4 we looked at the way test process improvement is considered within different life cycles and how these models generally include activities related to learning and improvement as part of their life cycle definition. In section 9.2.2 we identified the life cycle model in use as one of the six factors that influence the culture of improvement within a project or organization.

Life cycle model A partitioning of the life of a product or project into phases. [Chrissis, Konrad, and Shrum 2004].

In the following sections, we draw on some of this material to show that the life cycle model used is a significant factor affecting the context within which test improvement takes place.

Influence of Iterative and Agile Life Cycle Models on Improvement Context

The principal aspects to consider here are as follows:

Improvement of cycle frequency

Organizational aspects

Scope of improvement

Source of improvements

Level of (test) documentation

Improvement methods

Support from test process improvement models

Within projects that use these life cycle models, improvements generally take place in frequent feedback loops that enable test process improvements to be considered frequently. This will be the case, for example, where the iterative Rational Unified Process (RUP) is used or, when applying SCRUM, at the end of a sprint or even as part of a daily stand-up meeting.

Because the scope is often limited to the previous cycle or sprint, small but frequent improvements are made that focus mainly on solving specific project problems (see section 2.5.4 for an explanation of how retrospectives are used in agile development life cycle models such as SCRUM). The focus of these improvements is often not on cross-project learning and institutionalization of improvements.

Looking at the organization of test improvement, we find that there is likely to be less focus on Test Process Groups at an organizational level and more emphasis on the self-management of teams within the project. These teams generally have the mandate to change the testing process within the project according to their needs, resulting in highly tailored processes. However, some organizations also use weekly test stand-up meetings to bring things to a higher and cross-project level.

Rational Unified Process (RUP) A proprietary adaptable iterative software development process framework consisting of four project life cycle phases: inception, elaboration, construction, and transition.

SCRUM An iterative incremental framework for managing projects commonly used with agile software development.

Since there is a more project-specific focus on (test) process improvement, less emphasis is likely to be placed on broader issues affecting testing across the organization. This could mean, for example, that fundamental testing problems may not be fully addressed because they are beyond this project-centric context. A typical example here is the approach taken to testing certain quality attributes, such as performance and reliabity. These issues may become deferred from iteration to iteration because they often require more skills and resources than the project team has available. It is hard in these areas to make a substantial next step without major investments. Solving problems only on a project level could also easily lead to suboptimization and losing touch with the bigger picture.

In agile and iterative contexts, the range and number of alternative improvement ideas to be considered may be considerably more than compared to the sequential life cycle models. Since most members have a part-time testing role within the project, these ideas can come from any project member. This places a stronger emphasis on evaluating and prioritizing improvement suggestions, which may be more of a team effort than a task assigned to a test process improver. Since this may require the specific testing knowledge of a test process improver, they can also act as a consultant to the team if requested to do so.

In projects using the agile methodology and practicing agile testing techniques such as extreme programming (XP), don’t expect to find the level of test documentation you would expect from projects using a sequential life cycle.

There may be a single combined “test document” covering the essential elements of a test policy, test strategy, and even high-level test plan. Test process improvers should avoid making “improvement” suggestions that call for more rigorous and thorough test documentation. Like it or not, this isn’t part of the life cycle’s approach. One of the main agile principles is that documentation is created only when there is a clear unambiguous need for it [URL: agile manifesto], [van Veenendaal 2008].

Agile testing Testing practice for a project using agile software development methodologies, incorporating techniques and methods such as extreme programming (XP), treating development as the customer of testing and emphasizing the test-first design paradigm.

Extreme programming (XP) A software engineering methodology used within agile software development whereby core practices are programming in pairs, doing extensive code review, unit testing of all code, and simplicity and clarity in code.

The methods used to propose test process improvements when using an agile or iterative life cycle will tend to be analytical methods for evaluating the root causes of problems, such as cause-effect diagrams. These are particularly useful methods for the problem-solving mindset that prevails at the end of an iteration or sprint. Note, however, that the life cycle used does not dictate the improvement method used.

As mentioned in chapter 5, analytical approaches go hand-in-hand with model-based approaches to test process improvement, and this is also true for projects that use an iterative or agile life cycle. However, more tailoring of the models is required. If a content-based model such as STEP or CTP is used (see sections 3.3.4 and 3.3.5), the amount of tailoring required will be substantial. The principal references to these models ([Black 2003], [Craig and Jaskiel 2002]) do not provide any explicit support for this tailoring and the model user is effectively on their own regarding the tailoring required. Basically, both STEP and CTP were developed in a period of time when agile and iterative development life cyles were almost nonexistant, and neither model has been updated since to adapt to today’s context. Nevertheless, techniques and other tools that are very useful in agile and iterative environments can be found in both STEP and CTP.

When using a process improvement model such as TPI NEXT or TMMi, more help is available to make the necessary adjustments for agile and iterative life cycles.

The official TPI NEXT book [van Ewijk, et al. 2009] includes chapters that show how to use the model in agile and iterative projects. This includes, for example, a list of the principal key areas to be considered and how their checkpoints should be best tailored and interpreted. In addition, the TMap NEXT content-based methodology (which forms the methodological foundation for the TPI NEXT model) is tailored for SCRUM projects in TMap NEXT in Scrum [van der Aalst and Davis 2013] so that TMap can also be applied in agile and SCRUM contexts.

The TMMi website [URL: TMMi] provides case studies and other material on using TMMi in agile projects. One of the authors has provided consulting services to a small financial institution while achieving TMMi level 2 and to a medium-sized embedded software company while achieving TMMi level 3, both employing the agile (SCRUM) life cycle using the “standard” TMMi model. Note that within TMMi, only the goals are mandatory, not the practices.

With TMMi, a special project has been launched to develop a special derivate that focuses on TMMi in agile environments. The main underlying principle is that TMMi is a generic model applicable to various life cycle models and various environments. Most (specific) goals and (specific) practices as defined by TMMi have shown to be also applicable in agile environments. Remember, testing still needs to be done in a professional way. However, many of the sub-practices and examples, and their interpretation, are (very) different. As a result, the TMMi Foundation is not developing a new maturity model but will document the way TMMi can be applied in an agile environment. It will be determined whether each “standard” TMMi goal and practice is also applicable for testing in an agile life cycle. Some goals (or practices) may just not be. For each goal and practice that is applicable, typical lightweight agile sub-practices and examples will be defined. The latest updates and results of this project can be found on the TMMi website. [URL: TMMi]

Influence of Sequential Life Cycle Models on Improvement Context

When sequential life cycle models such as the V-model are used, test process improvement is generally considered at the end of a software release cycle, at the end of a phase, or at the end of the project itself. This is where, for example, milestone reviews can be a particularly useful method for performing causal analysis and identifying improvements (see section 4.2.6). The feedback loops for process improvement contained in sequential life cycles enable product and process conformance and suitability (verification and validation) to be checked at predefined points in the project.

Using sequential life cycle models, the scope of improvement is broader and there is more opportunity to address organizational improvements compared to the more project-centric approach in iterative and agile life cycles. However, the benefits of performing test process improvement often accrue only for other projects and may not have a direct impact on the current project.

Many of the improvement methods described in this book have been developed to support projects that apply sequential life cycles, projects for which “home-grown” approaches have been developed, or even projects for which no test process is defined. The process maturity concepts implemented in process reference models such as TMMi and TPI NEXT are well suited to sequential life cycles, and the stepwise approaches in content reference models such as CTP and STEP are also well aligned to projects using sequential life cycles. In general, however, the process reference models TPI NEXT and TMMi provide more support for both sequential and iterative/agile life cycle models and would need less tailoring if applied to the latter.

Other Factors That Influence the Context of Improvement

Apart from the life cycle model used, two other factors, management culture and test approach, have an influence on the acceptability of a suggested improvement approach.

Management Culture

As noted in section 9.2.2, management culture can have a decisive influence on the culture of improvement and, ultimately, on whether test process improvements will be successful. Management culture includes the following categories:

Command and control: Decisions are made at the top and handed down through the hierarchy of management until they reach the people who are instructed to perform specific tasks.

Consultative: Management involves people at various stages in the decision making process and adopts a more consultative approach to get good buy-in.

Team driven: The team decides collectively on what is best in their project. This is typically found in projects using SCRUM.

Test Strategies Used and Their Influence on the Context of Improvement

Testing strategies may include one or more of the following:

Automated (see section 2.5.4), often referred to as the regression-averse strategy

Manual, including scripted (part of a methodical strategy), unscripted, and exploratory approaches (part of a dynamic strategy)

Clearly, the blend of approaches that are used within a project will influence the overall acceptability of the improvements proposed. Manual testing approaches may benefit from some automation, and the efficiency of automated approaches may be improved by introducing different techniques, such as keyword-driven. An overview and description of the various test strategies is provided in Foundations of Software Testing – ISTQB Certification [Black, van Veenendaal, and Graham 2012].

Concluding Remarks

Adapting for different context factors is an essential element in creating acceptable improvement proposals. The test process improver must evaluate whether proposals “fit” the chosen life cycle model and propose only those recommendations that are appropriate.

10.2 Exercises

The following multiple-choice questions will give you some feedback on your understanding of this chapter. Appendix F lists the answers.

10-1 Which of the following aspects has an influence on improvement context when using an iterative or agile life cycle model?

A: Test effectiveness metrics

B: Improvement cycle frequency

C: Time frames for reaching improvement milestones

D: Availability of process improvement skills

10-2 What statement best reflects the involvement of the test process improver in projects using an agile life cycle?

A: The test process improver has no direct involvement in agile projects. The team members take on testing tasks, including improvement.

B: The role of test process improver is no different than in projects using sequential life cycle models.

C: The test process improver may act as a consultant to the team on request.

D: The test process improver performs mostly analytical tasks because test process improvement models are rarely used in agile projects.

10-3 Which of the following statements correctly reflects the influence of sequential life cycle models on improvement context?

A: Test process improvement models are intended for use with sequential life cycle models only.

B: The main opportunity for test process improvement is at the end of the project.

C: Improvement activities focus mostly on process conformance issues (not limited to just process conformance—e.g., product conformance, usability).

D: There is more opportunity to address organizational improvements.

10-4 If you are doing test improvement in a organization that is mainly applying agile development, which of the following methods would you most likely not use?

A: STEP

B: Causal analysis

C: TMMi

D: TPI NEXT

10-5 Which of the following management cultures would best fit the situation where agile (SCRUM) is being practiced?

A: Command and control

B: Exploratory

C: Team driven

D: Consultative

10-6 Which test strategy would support a test improvement approach centered around tools?

A: Dynamic

B: Methodical

C: Model-based

D: Regression averse