Comprehensive and Detailed Model Answer
Task 1(a): Identify two major test improvement models (2 points)
The two principal test process improvement models applicable to this scenario are:
TPI Next – Test Process Improvement Next
A business-driven process improvement model that evaluates testing capabilities across sixteen key areas using a form of continuous maturity representation.
TMMi – Test Maturity Model integration
A staged test process improvement model that establishes progressively increasing organizational testing maturity through predefined process areas, goals and practices.
Task 1(b): Identify four comparison criteria (4 points)
The following four criteria provide an appropriate basis for comparing the models:
1. Type of model and maturity representation.
2. Relationship with software process improvement models.
3. Scope and focus of assessment.
4. Overall improvement approach.
These criteria are particularly relevant because the bank requires immediate project-specific improvements rather than a long-term organizational maturity program.
Task 1(c): Evaluate both models against the criteria (16 points)
Comparison criteria
TPI Next
TMMi
1. Type of model
Continuous representation allows improvement efforts to concentrate on selected key areas. This suits the ATM project's tight schedule and limited improvement budget.
Staged maturity representation requires attention to defined process areas associated with successive maturity levels. This is more suitable for sustained organizational improvement.
2. Relationship with software process improvement
Independent of a particular software process improvement model. This provides flexibility because the bank has no established improvement framework.
Closely aligned with the CMMI structure, although it can be used independently. The structural relationship with CMMI offers limited immediate benefit because the bank has no established software process improvement initiative.
3. Scope and focus
Provides a broad overview of testing across sixteen key areas, allowing detailed investigation of the most significant weaknesses affecting the ATM project.
Provides detailed assessment of process areas at specified maturity levels. A maturity-level-focused assessment may be disproportionately broad or exclude higher-level capabilities relevant to the immediate problem.
4. Overall improvement approach
Business-driven and focused on testing engineering capabilities. Improvement recommendations can be linked directly to ATM reliability, functionality, usability and performance.
Emphasizes management commitment, structured process institutionalization and sustained development of testing maturity. These are valuable but may require investment beyond the immediate project.
Model suitability for the ATM project: conceptual comparison, not measured assessment scores.
TPI Next
TMMi
Flexible improvement across selected key areas, subject to checkpoint dependencies.
Structured progression through predefined organizational maturity levels.
Task 1(d): Management recommendation (3 points)
Recommended model: TPI Next.
TPI Next is the more suitable model because the bank requires targeted, short-term improvements within a single strategically important project. The organization does not currently have a long-term improvement strategy, an established software process improvement framework or sufficient resources for an extensive maturity program.
A focused TPI Next assessment should identify weaknesses in risk-based testing, regression coverage, test design, test planning and progress monitoring. Improvements should be implemented incrementally and evaluated during successive development iterations.
The bank should avoid committing to an organization-wide maturity target at this stage. Broader process improvement may be considered after the ATM project has demonstrated measurable benefits.
Task 2(a): Four reasons for using analytical-based improvement (16 points)
Reason
Application to the bank and ATM project
1. Specific recurring problems require investigation.
The savings and credit card projects experienced inadequate product quality, and many post-release defects appear related. Analytical improvement can investigate common failure patterns and identify preventive measures relevant to the ATM project.
2. Historical defect information is available.
The bank maintains a defect management system containing well-documented data from previous projects. Existing information provides an economical foundation for causal analysis without requiring extensive new data collection.
3. The root causes may exist outside testing.
Management has requested test improvements, but the actual deficiencies remain unknown. Problems may originate from requirements, architecture, implementation, integration or release management. Analytical investigation allows improvement activities to target the actual causes.
4. Improvement resources are limited.
The ATM project operates under tight timescales and a small improvement budget. A focused analysis of significant recurring defects enables the organization to prioritize improvements with the greatest potential business benefit.
Analytical improvement is particularly appropriate because it uses organization-specific evidence. Rather than assuming that adopting additional testing practices will resolve the quality problems, the bank can first determine why previous projects failed and then introduce measures designed to prevent recurrence.
Task 2(b): Select and explain an analytical-based approach (5 points)
Recommended approach: Root Cause Analysis (RCA).
Root Cause Analysis is the most appropriate analytical approach because the bank has historical defect information and evidence suggesting that multiple production defects may share common underlying causes.
A comprehensive measurement initiative using GQM could provide additional value, but establishing an extensive measurement program may require more resources and organizational commitment than the immediate project allows.
Root Cause Analysis can be conducted through the following process:
Step
Root Cause Analysis Process
1. Select defects for analysis
Use defect severity, frequency, recurrence and Pareto analysis.
2. Collect and organize evidence
Examine defect records, requirements, test results and development practices.
3. Identify underlying causes
Apply cause-effect diagrams and validate causal relationships.
4. Develop improvement recommendations
Introduce preventive actions addressing the verified root causes.
For example, analysis may establish that previous banking applications experienced recurring defects because transaction interruption and recovery scenarios were inadequately tested.
The ATM project could then introduce targeted tests for network interruption, cash-dispensing errors, transaction reversal, account balance consistency and recovery following hardware or software failures.
The effectiveness of the corrective actions should subsequently be evaluated during iterative development cycles.
Task 2(c): Identify appropriate improvement metrics (4 points)
The primary business objective is to improve product quality by reducing defects that escape testing and become observable during production use. Therefore, test effectiveness metrics should be selected.
Metric 1 – Defect Detection Percentage (DDP)
DDP measures the proportion of known defects identified during testing rather than following release.
DDP = (Defects found during testing / Total known defects) × 100
For example, if testing finds 180 defects and a further 20 defects are identified following release:
DDP = (180 / 200) × 100 = 90%
An increasing DDP indicates that a greater proportion of known defects is being detected before the software enters production.
Metric 2 – Post-release defect rate
This metric measures defects reported following production release, normalized against software size.
Post-release defect rate = Post-release defects / KLOC
KLOC represents thousands of lines of code. Measurements should be taken over a consistent post-release observation period, with appropriate differentiation between severity categories.
A decreasing post-release defect rate provides evidence that product quality is improving and fewer defects are escaping the testing process.
Defect Detection Percentage – Illustrative example
Value
Defects found during testing
180
Post-release defects
20
Defect Detection Percentage
90%
Static worked example of the chat calculator.
Requirements coverage may additionally be monitored to ensure that existing ATM functionality, usability and performance requirements are adequately tested. However, coverage is an indirect indicator and should not replace direct measures of defect detection effectiveness.
Overall assessment conclusion
The recommended strategy is to combine a focused TPI Next assessment with targeted root cause analysis of previous production defects. TPI Next provides a structure for identifying testing process weaknesses, while causal analysis establishes which weaknesses are most relevant to the bank's recurring product quality problems.
Improvements should be introduced incrementally during the ATM project's iterative development lifecycle and evaluated using appropriate defect detection and product quality indicators.
Management's aspiration to detect all defects before release should be translated into realistic, measurable objectives because testing cannot demonstrate the complete absence of defects.
Official References: ISTQB CTEL-ITP Syllabus 2011, Section 3.1.2 – Continuous and Staged Representations; Section 3.3.3 – Comparing TPI Next to TMMi; Section 4.2 – Causal Analysis; Sections 4.4.2.1 and 4.4.2.2 – Test Effectiveness Metrics; Section 5.1 – Selecting Test Process Improvement Approaches; Section 10.1 – Adapting to Different Life Cycle Models. ISTQB Sample Exam A – Answers v1.1, pages 32–35. Learning Objectives EITP-2.3.7 (K5) and EITP-4.1.2 (K6).
Complete Essay and Assessment Summary
Question
Assessment topic
Maximum points
Cognitive level
1
TMMi assessment, process areas, interviews and motivation
50
K5, K6
2
TPI Next matrix assessment, root cause analysis and management recommendations
50
K5, K6
3
Comparison of improvement models, analytical approaches and metrics
50
K5, K6
Total
150
All three essay questions have been extracted from pages 19–27 of the uploaded examination document, with their original scenarios, individual assessment tasks, point allocations, reproduced exhibits, comprehensive model answers and official references.
The answers are intentionally more extensive than 150–250 words per essay because each question contains several independently graded tasks worth a combined 50 points.
Source acknowledgement: © International Software Testing Qualifications Board (ISTQB), Sample Exam Set A, Version 1.1. If this material is intended for commercial publication, the usage restrictions in the original document must be observed.