Improving the Test Process: Implementing Improvement and Change - A Study Guide for the ISTQB Expert Level Module (2014)

Chapter 1. Introduction

In this chapter we introduce ourselves and the Expert Level syllabus “Improving the Test Process”, which forms the basis for this book.

Concerning the Expert Level in general, we will ask the fundamental question, What is an expert? and describe the expectations that can be placed on becoming an expert in test process improvement.

We show the overall ISTQB Certified Tester scheme and explain the importance of the Expert Level in enabling the definition of career paths for testers.

The “Improving the Test Process” syllabus is divided into two separately examinable parts. Each part is briefly described and issues concerning the certification exam are explained.

1.1 About the Authors

1.1.1 Erik van Veenendaal

Dr. Erik van Veenendaal, CISA, has been working as a practitioner and manager in the IT-industry since 1987. After a career in software development, he transferred to the area of software quality. As a test analyst, test manager and test consultant, Erik has over 20 years of practical testing experience. He has implemented structured testing, formal reviews and requirements processes and has carried out test process improvement activities based on Testing Maturity Model integration (TMMi) in a large number of organizations in different industries. Erik has also been a senior lecturer at the Eindhoven University of Technology, Faculty of Technology Management, for almost 10 years.

Erik founded Improve Quality Services BV (www.improveqs.nl) back in 1998 as an independent organization that focuses on advanced high-quality services. He has been the company director for over 12 years. Under his direction, Improve Quality Services became a leading testing company in The Nether lands. Customers are especially to be found in the area of embedded software (e.g., Philips, Océ en Assembléon) and in the finance domain (e.g., Rabobank, ING, and Triodos Bank). Improve Quality Services offers international consultancy and training services with respect to testing (e.g., test process improvement using the TMMi framework), quality management, and requirements engineering. Improve Quality Services BV was the second worldwide company to become accredited to perform TMMi assessments. It is a market leader for ISTQB Foundation Level and ISTQB Advanced Level training courses and a member of the International Requirements Engineering Board (IREB).

Erik is the (co-)author of numerous papers and a number of books on software quality and testing, including the bestsellers ISTQB Foundations of Software Testing, Test Maturity Model integration (TMMi), The Testing Practitioner, and Testing according to TMap. Erik was the first person to receive the ISEB Practitioner certificate with distinction and is also a Certified Information Systems Auditor (CISA) and accredited TMMi lead assessor. He is a regular speaker at both national and international testing conferences and a leading international trainer (ISTQB accredited) in the field of software testing. He holds the EuroSTAR record, winning the best tutorial award three times.

Since its foundation in 2002, Erik has been strongly involved in the International Software Testing Qualifications Board (ISTQB). From 2005 to 2009, he was the vice president of the ISTQB organization, and he is the founder of the local Belgium and The Netherlands board, the Belgium Netherlands Testing Qualifications Board (BNTQB). He is the editor of the ISTQB Standard Glossary of Terms Used in Software Testing. As a working party chair, he has been the lead of the ISTQB Expert Level working party for over 10 years. As a co-author, he is also involved in writing in the syllabi at the Foundation, Advanced, and Expert Levels. Erik is one of the founders of the TMMi Foundation and is currently its vice chair. He is the lead developer of the TMMi model. Erik is actively involved in various working parties of the International Requirements Engineering Board (IREB). For his outstanding contribution to the field of testing, Erik received the European Testing Excellence Award in December 2007.

After having provided leadership to Improve Quality Services BV for over 12 years, Erik stepped down from that role in July 2010. Since that time he has been living in Bonaire, where he is involved in international test consultancy, training, international organizations (e.g., ISTQB, TMMi, and IREB), publications, and presentations.

Erik can be contacted via email at eve@improveqs.nl and through his website, www.erikvanveenendaal.nl. You can also follow Erik on Twitter; his username is @ErikvVeenendaal.

1.1.2 Graham Bath

Graham Bath’s experience in testing spans over 30 years and has covered a wide array of domains and technologies. As a test manager, he has been responsible for the testing of mission-critical systems in spaceflight, telecommunications, and police incident control. Graham has designed tests to the highest levels of rigor within real-time aerospace systems such as the Tornado and Eurofighter military aircraft.

As a principal consultant for the T-Systems Global Delivery Center, Testing Services, he has mastered the quality improvement programs of several major companies, primarily in the financial, government, and automotive sectors. In his current position, Graham is responsible for developing the testing skills of T-Systems’ professional testing staff and for its range of test improvement offerings.

Graham is a regular speaker at testing conferences and has given tutorials on test improvement throughout the world. He is co-author (together with Judy McKay) of The Software Test Engineer’s Handbook, which is a study guide for the ISTQB Test Analyst and Technical Test Analyst Advanced Level certificates.

Graham chairs the ISTQB Expert Level Working Group and is co-author (together with Erik van Veenendaal and Isabel Evans) of the ISTQB Expert Level Certified Tester syllabus “Improving the Test Process.”

As a longstanding member of the German Testing Board, Graham chairs the GTB’s Advanced Level and Expert Level working groups.

Graham was born and raised in England but now lives and works in Munich, Germany. You can contact him via email at graham.bath@t-systems.com.

1.2 Purpose of the Book

We set ourselves some fairly tough requirements for this book. After all, it is intended for those aspiring to become “experts” in the field of test process improvement. Before we launch into the actual content, we’d like to give you a brief overview of the basic requirements we set ourselves. This will help you understand the general approach we have taken in structuring and writing the book.

First and foremost we, the authors, require that the book provides a thorough coverage of the subject matter and is readable.

Overview

Each chapter includes a brief introduction to summarize content.

Completeness and Structure

This book is based on the ISTQB Expert Level syllabus “Improving the Test Process” [ISTQB-CTEL-ITP] and covers everything you will need to know to successfully sit for the examinations. You can also use the information in this book to become a competent and employable test process improver.

We have maintained the structure of the syllabus throughout this book. Chapter and section numbers map closely to each other.

Readability

When writing a book that is based on a predefined syllabus, it’s all too easy to focus on syllabus coverage alone. Of course, syllabus coverage is essential, but the result is often a rather “dry” reading experience, which we don’t want. We want you to have a book that gives you syllabus coverage and is readable.

We intend to make this book readable by adopting a particular style and standardized approach to each chapter:

Learning objectives

These are the specific learning objectives provided in the syllabus and will be of particular use for those of you studying for the certification exam.

Technical content

Within each chapter, we deal with the actual technical content of a particular subject. The learning objectives of the ISTQB Expert Level syllabus are not just focused on learning and repeating, they are designed so that you can apply what you have learned and justify your choices. To help you achieve this we go beyond the information provided in the syllabus and add more descriptive material to give you a “well-rounded” level of knowledge.

Exercises

The exercises help develop your ability to apply the material provided in the book. These are not intended as formal multiple-choice exam practice questions written against learning objectives. Note that the ISTQB Expert Level exam also includes essay-type questions, which require a written answer. Appendix D provides a detailed overview of the exam that will help candidates to prepare.

Appendices

Useful information has been grouped together into the appendices listed in table 1–1.

Table 1–1 Overview of appendices

1.3 What Is an Expert?

Regrettably, the word expert is probably one of the most overused terms in our industry. In a competitive world it has become all too popular to label ourselves or others as “experts.” In fact, if you were to ask 10 people to define the term expert, you would likely get 10 different answers. When the ISTQB set up the Expert Level qualification, it therefore came as no surprise that there was some considerable debate about what expert should actually mean. Should it be some kind of “super guru,” should it be anyone with a bit more experience than average, or should it simply be left open to interpretation? The definition of a testing expert used by ISTQB is “a person with the special skills and knowledge representing mastery of a particular testing subject. Being an expert means possessing and displaying special skills and knowledge derived from training and experience.”

The approach taken by ISTQB is to consider “expert” as an integral part of a career path in testing (see section 1.5.). Each Expert Level module defines its own specific expectations and business outcomes (these are covered in section 1.4.).

The Expert Level program can be characterized by the following key attributes:

“Higher level” learning objectives are included that emphasize the ability to evaluate, make judgments, and bring together different elements to form a coherent whole (please refer to appendix E for details).

Subjects are given in-depth consideration.

High levels of experience are expected to achieve certification.

“Continuous learning” is expected. Unlike the Foundation and Advanced Level certificates, an Expert Level certificate is not for life; it is valid for five years. Certification renewal can be achieved either by retaking the exam or by collecting sufficient credits in the Certification Extension scheme [ISTQB-CEP] (see section 1.8.).

1.4 Expectations and Business Outcomes

It is not intended that candidates who qualify at the Expert Level should immediately be considered “world experts” in test process improvement. The expectation is that the qualified ISTQB CTEL in “Improving the Test Process” will be able to provide expert support within their organization or project to initiate, implement, and support improvements to testing in that organization or project.

Business outcomes describe the value obtained from acquiring the Expert Level certificate, principally from the perspective of an employer or sponsor. These are the people who ask the fundamental question, What can I expect an expert in test process improvement to do for me and my organization?

The Expert Level Overview document [ISTQB-EL-OVIEW] describes the business outcomes for each Expert Level module. The business outcomes for the “Improving the Test Process” syllabus are allocated to the two syllabus parts (see section 1.6. and appendix C) as follows:

Part 1: Assessing test processes

The expert test process improver is able to perform each of the following tasks:

Part 2: Implementing test process improvement

The expert test process improver is able to perform each of the following tasks:

1.5 Career Paths for Testers

Introduction of the Expert Level establishes the core structure of the ISTQB Certified Tester program, which starts with Foundation Level and progresses via the Advanced Level up to Expert Level. The syllabi within each level are carefully scoped to ensure that testing themes are introduced at an appropriate level and to ensure that specific subjects, such as test process improvement, are developed in increasing detail. The result is a structure that supports the development of career paths for professional testers. The arrows in the following diagram show the current paths within the ISTQB Certified Tester structure.

Career paths not only indicate an individual’s progression in terms of their experience and knowledge of a particular subject (e.g., test process improvement), they also indicate the certifications required to progress from one level to another. The Expert Level Overview document [ISTQB-EL-OVIEW] defines the amount of experience and the certifications required to achieve the different Expert Level certifications.

Figure 1–1 Structure of the ISTQB Certified Tester scheme

Figure 1–1 shows that the Expert Level syllabi are generally divided into parts. A candidate will be required to pass both examinations to achieve full certification and to formally become a Certified Expert Level Tester (CTEL) on the subject of improving the test process. Syllabus parts are discussed further in section 1.6. and in appendix C. Note that further career paths will be defined as more modules become available.

The career path for test process improvers is shown in table 1–2.

Table 1–2 Career path for test process improvers

Refer to appendix E for a brief description of the cognitive levels (K-levels) mentioned in table 1–2.

1.6 Syllabus Parts

The ISTQB Expert syllabus “Improving the Test Process” [ISTQB-CTEL-ITP] covers material that requires a minimum of 8.5 days of study. The overall syllabus is therefore divided into two parts, which allows separate courses to be provided, each with a specific focus, as shown in table 1–3:

Table 1–3 Parts of the “Improving the Test Process” module of the ISTQB Certified Tester Expert Level syllabus

Note that a candidate will be required to take both examinations in order to achieve full certification. Please refer to the following section and appendix D for further details of the exam.

1.7 The Certification Exam

The examination is introduced in this section. For further details, please refer to the document Expert Level Exam Structure and Rules issued by ISTQB [ISTQB-EL-EXAM] and appendix D.

A separate exam is provided for each part of the “Improving the Test Process” syllabus (see the preceding section). Each exam is made up of two components:

Multiple-choice questions

Essay questions

Before taking the exam, you must meet the following entry conditions:

A participant must have an ISTQB Advanced Level Test Manager certificate (or equivalent, such as, for example, ISEB Practitioner, Version 1.1 – September 4, 2001).

Examinees who wish to take a nonpublic exam, scheduled at the end of an accredited training course, must first produce evidence to the Examination Body that they have attended all days of that course (the training provider will normally provide this at the end of the course).

Before taking the exam, it is recommended that:

participants have at least seven years of practical testing experience

participants have attended a training course, although, as with other ISTQB levels, this is not formally required to take a (public) exam (i.e., an exam not provided at the end of a course).

Pass Mark

The pass mark for each exam is 65 %.

1.8 Certification

To receive full certification at the Expert Level (CTEL), you must pass both exams (see section 1.7.) and provide proof of the following practical working experience:

At least five years of general testing experience in the industry

At least two years of industry experience in test process improvement

Written proof of this experience and two verifiable references need to be submitted.

The Expert Level certificate is initially valid for five years. After the initial five years, individuals may extend their current level of certification for another five-year period. There is no limit to the number of times a certification can be extended.

Extension is achieved by retaking the exam for the Expert Level certificate or by completing activities to accumulate a minimum of 200 certification extension credits (CECs) before the current certification expires. The activities, renewal process, and CECs that may be awarded are defined in the ISTQB Certified Tester Expert Level, Certification Extension Process document [ISTQB-CEP].