Improving the Test Process: Implementing Improvement and Change - A Study Guide for the ISTQB Expert Level Module (2014)

Appendix D: The Exam

As mentioned in chapter 1, the “Improving the Test Process” syllabus consists of two parts, each with its own exam. Section 1.7. provided a general overview of the Expert Level exam. This appendix provides more details about the exam, including the general rules that govern it and the coverage of learning objectives in the exam for each syllabus part.

Essay questions have been introduced to the ISTQB Certified Tester Expert Level certification exams for the first time. We have provided some tips that will help candidates prepare for this part of the exam.

D.1 General Exam Aspects

Each exam is made up of two components:

Multiple-choice question component

Essay question component

Multiple-choice question component

The duration of this component is 45 minutes.

This component addresses the learning objectives shown in table D–3.

Table D-1 Multiple-choice question component Multiple-choice question component

Some questions may be more difficult than others. To reflect this, K3 questions may be set at 1, 2, or 3 points and K4 questions may be set at 2 or 3 points, at the discretion of the author of the question. The number of points allocated to a question is shown on the exam paper.

Each multiple-choice question is expected to be associated with at least one learning objective.

The allocation of the different K-levels of learning objectives (LOs) in the syllabus is reflected in the coverage of the learning objectives in the exam.

The multiple-choice exam has 25 questions and is allocated 35 points.

Essay question component

The duration is 90 minutes (see table D–2).

An exam contains three essay questions, of which two must be answered. No extra credit is given for answering the third essay question. (Don’t even consider it!)

Table D-2 Multiple-choice question component

Each essay question is scenario based. The question starts with a descriptive scenario, and on the basis of this, one or more questions are asked.

Each essay question covers at least two relevant business outcomes (see section 1.4. for a list).

Essay questions are distributed according to the coverage of the business outcomes and K5 and K6 learning objectives. The required distribution for each specific syllabus part is provided in sections 14.2 and 14.3.

It is not mandatory for exam participants to answer exam sections in a particular order. You are free to choose whether to answer the essay questions first and then the multiple-choice questions, to answer multi-choice questions first and then the essays, or even to mix them.

The scoring of a candidate’s answer to an essay question is marked according to a scoring guideline. This ensures the maximum possible objectivity in scoring questions.

Each essay is allocated 50 points.

Rules affecting time available

The total exam duration is 135 minutes, which is extended by 25 percent for those taking the exam in a language other than their own native language.

There is no mandatory time limit assigned to each question. The 45 minutes per question is just a guideline. If, for example, a candidate completes an essay question in 40 minutes, they will have 50 minutes to complete another question.

There is no mandatory time limit assigned to an exam component (multiple-choice and essay). If one component is completed in less time than indicated in the preceding tables, more time is available for the other component.

D.2 Part 1 Exam: “Assessing Test Processes”

The multiple-choice component

Coverage of learning objectives in the multiple-choice exam is shown in table D–3. Note that for certain learning objectives, a selection is given; for example, there will be a single question relating to learning objective 2.5.1 or 2.5.2.

Table D-3 Part 1 Exam multiple-choice questions

The essay component

The following K5 and K6 learning objectives are covered in the essay component of the part 1 exam:

LO 2.1.3 (K6)

LO 3.3.7 (K5)

LO 3.3.10 (K5)

LO 4.2.6 (K5)

LO 4.4.2 (K5)

LO 5.1.2 (K5)

LO 6.2.3 (K6)

LO 6.3.2 (K6)

LO 6.3.3 (K6)*

LO 6.3.5 (K5)*

* The following two learning objectives are considered “high priority.” It is recommended to include each of them as part of the essay questions:

Each essay question must cover a minimum of two business outcomes out of the four shown in section 1.4..

D.3 Part 2 Exam: “Implementing Test Process Improvement”

The multiple-choice component

Coverage of learning objectives in the multiple-choice exam is shown in table D–6. Note that certain learning objectives are grouped together for question selection. For example, there will be a single question which relates to either learning objective 6.4.1, 6.5.1 or 6.6.1.

Table D-4 Part 2 exam multiple-choice questions

The essay component

The following K5 and K6 learning objectives are covered in the essay component of the part 2 exam:

LO 6.1.2 (K6)

LO 6.4.5 (K6)*

LO 7.1.4 (K6)

LO 7.3.2 (K5)

LO 8.2.2 (K6)

LO 8.3.3 (K5)

LO 9.1.2 (K5)

LO 9.1.3 (K5)

LO 9.2.2 (K6)

* LO 6.4.5 (K6), “Create a test improvement plan,” is considered “high priority.” It is recommended to include this LO as part of the essay questions.

Each essay question must cover a minimum of two Business Outcomes taken from the list of four shown in section 1.4..

D.4 Tips

Essay questions are marked using a predefined marking scheme that allocates points when specific content is included in the answer. You can’t gain marks by showing knowledge in any area other than the one(s) asked for.

An essay question frequently consists of different tasks. Always check the number of points allocated to a particular task and allocate a similarly proportional amount of time for your answer.

Be sure that if the task asks for a specific number of items (e.g., “provide four reasons why <something> should be recommended”), you provide the requested number; no more and no less.

Try to make your answers clear and to the point. Unless otherwise stated in the question, there are generally no extra points allocated for long explanations.

Marks are allocated for content rather than writing style. It is, of course, important that you can be understood and make your point, but no extra marks are given for lengthy, verbose explanations (this should be an encouragement for anyone taking the exam in a language other than their own native language).

D.5 Common Problems

Candidates taking essay-type exams have tended to make similar mistakes, many of which are easily remedied with a little application. The most common problems are listed here for guidance:

Not answering the question set. Examination questions usually incorporate a simple scenario that sets the context for the question. Candidates who do not pay enough attention to this information tend to answer the question in the style “write down all you know about…” rather than addressing the specific issue(s) raised by the question. While some of the candidate’s answer may be relevant, there is a serious risk that much of it will not be and will earn no marks.

Not thinking through answers. Candidates need to structure their response to a question to ensure that they impart the required information with minimum effort. With long and rambling answers, you run the risk of overlooking key information, and the relative importance of your ideas may be undermined by the lack of structure. Long answers also take longer to write down than a well structured but shorter response.

Answers are too superficial. Questions are worded to indicate the level of response required. Sometimes a word count is included. If a question asks for a description of something, this will require more than simply identifying the “something” by name. Similarly, justify means provide some kind of evidence that your response is correct or appropriate. All information in an answer must be relevant; any irrelevant information will not earn marks. The candidate therefore wastes valuable time in writing down irrelevant information.

Addressing the scenario. Where a scenario is provided (especially in double-length questions), there is an expectation that candidates will refer to the scenario in their answer, using it to identify specific examples of points raised or to illustrate issues. In an ISTQB Expert Level exam, it is important to demonstrate practical validity of answers wherever possible.

Running out of time. The problems mentioned previously all contribute to wasted time. There is time to construct and write down good answers to the questions on the paper if time is not wasted. A good time plan is essential. It is a good idea to set out the key points of an answer in a bulleted list before starting to draft the text of the answer. At the end of the exam, further time should be allowed for checking through your answers to ensure that no major mistakes have been left uncorrected.

Illegible answers. It is difficult to write both quickly and neatly. However, an illegible scrawl makes it very difficult for a script marker to read a candidate’s response and important information may be unintelligible. Time planning should reduce the need to rush. Practice in writing at more than the usual speed would be sensible preparation for the examination, especially for candidates who do not normally communicate with pen and paper. You may find it helpful to highlight key points in your answer by underlining or writing in block capitals.