CT-AI ISTQB Certified Tester AI Testing Exam Questions and Answers

Decision trees are a foundational algorithm used in supervised machine learning. The syllabus describes:

"A decision tree is a tree-like ML model whose nodes represent decisions and whose branches represent possible outcomes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.4)

The syllabus describes exploratory testing as especially useful when there are poor specifications or test oracle problems:

“Exploratory testing is especially useful when there are poor specifications or test oracle problems, which is often the case for AI-based systems.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.6, page 70 of 99)

The syllabus mentions a drawback of AI-generated test cases:

“AI-based test generation tools can generate test cases… However, unless a test model that defines required behaviors is used as the basis of the tests, this form of test generation generally suffers from a test oracle problem because the AI-based tool does not know what the expected results should be.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.3, page 78 of 99)

The ISTQB CT-AI syllabus (Section 4.3 – Test Design for AI-Based Systems ) highlights pairwise testing as an effective test-case reduction technique for systems with many input parameters. Lane-keeping assist systems typically include environmental, sensor, and vehicle-dynamic parameters, making exhaustive testing infeasible. Pairwise testing significantly reduces the number of test cases while still capturing all 2-way interactions , which are responsible for a large proportion of software defects.

Option B aligns with this syllabus description: pairwise testing reduces otherwise extremely large parameter combinations, making test effort manageable.

Option A overstates feasibility guarantees; the syllabus never claims pairwise testing always makes testing “typically feasible.” Option C is unsupported and incorrect because pairwise testing does not restrict parameters to driver-controlled ones. Option D is incorrect because, although some defects arise from higher-order interactions, pairwise testing captures many relevant defects and is widely recognized as a pragmatic compromise.

Thus, Option B is the correct statement.

According to the ISTQB CT-AI syllabus, metamorphic testing works by applying metamorphic relations (MRs) : predictable input transformations that should lead to predictable output changes. From the initial test data, clear relations emerge for defensive driving scoring. The score increases when:

v_mean decreases ,

a_pos decreases ,

a_neg becomes less negative , and decreases when the opposite occurs.

A valid metamorphic follow-up test must modify inputs in a direction consistent with at least one MR while keeping the expected output direction predictable.

Test A lowers v_mean compared to Test 1, with similar acceleration values. This directly satisfies the MR that lower speed → higher score.

Test C increases both acceleration and braking intensity compared to Test 2, making the reduced score range (30–70) consistent with more aggressive driving.

Test D modifies acceleration and braking magnitudes in ways consistent with Test 3’s defensive-driving scoring boundaries.

Test B , however, changes multiple variables in contradictory directions:

v_mean increases (worse)

a_pos increases (worse)

a_neg becomes less negative (better)

Because these changes conflict, the expected score trend becomes unpredictable , violating the premise of a metamorphic follow-up test.

Thus Test B cannot be considered a metamorphic follow-up , which makes Option C correct.

Section 1.7 – Neural Networks of the ISTQB CT-AI syllabus explains that neural networks consist of neurons connected by weighted links. During training, learning occurs by adjusting the weights on these connections . This is the essence of gradient descent and backpropagation. Option B correctly states this behavior: only the weights are modified, not the activation functions, neuron counts, or architectural structure.

Option A is incorrect because a neuron’s bias is not determined by previous activations; it is an independent trainable parameter added to the weighted input sum. Option C is incorrect because the syllabus states that a single-layer perceptron is a valid type of neural network , although limited to linearly separable problems. Option D is incorrect because no rule requires the number of input neurons to exceed or equal the number of output neurons. Instead, input neurons correspond to the number of features , while output neurons correspond to tasks or classes .

Therefore, Option B precisely reflects the syllabus definition of what changes during neural network training.

 Factors Affecting ML Functional Performance: The data pipeline, quality of the labeling, and biased data are all factors that significantly affect the performance of machine learning models. The number of classes, while relevant for the model structure, is not a direct factor affecting the performance metrics such as accuracy or bias.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Quality and its Effect on the ML Model and ML Functional Performance Metrics​​​​.

The syllabus defines explainability as the ability to understand how the AI-based system comes up with a particular result:

"Explainability is considered to be the ease with which users can determine how the AI-based system comes up with a particular result."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.7)

 Type of Testing for Stock-Price Prediction Models: Concept drift refers to the change in the statistical properties of the target variable over time. Severe weather storms causing massive losses in coffee production and affecting stock prices would require testing for concept drift to ensure that the model adapts to new patterns in data over time.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Section 7.6 Testing for Concept Drift, which explains the need to test for concept drift in models that might be affected by changing external factors​​.

According to Section 4.6 – AI Behaviour Testing (AIB Testing) of the ISTQB CT-AI syllabus, AIB testing is used to evaluate changes in the functional behavior of self-learning systems. The core principle is comparing pre-change and post-change model behavior using the same test inputs , so that any difference in outputs can be attributed to the model’s learning and not to differences in input data. This directly corresponds to Option C .

Option A is incorrect because the absence of a test oracle does not justify generating new test cases; AIB relies on reusing identical inputs to detect behavioral drift. Option B is invalid because using different inputs prevents meaningful comparison. Option D is incorrect because comparing with an unrelated non-self-learning system does not allow evaluation of the same model’s behavioral evolution.

Thus, Option C accurately represents the correct application of AIB testing: assessing model behavior changes by running identical test inputs before and after learning updates.

A. The difficulty of defining criteria for improvement before the model can be accepted.

Defining criteria for improvement is a challenge in the acceptance of AI models, but it is not directly related to the performance drop in real-world scenarios. It relates more to the evaluation and deployment phase rather than affecting the model's real-time performance post-deployment.

B. The fast pace of change did not allow sufficient time for testing.

This can significantly affect the model's performance. If the system is self-learning, it needs to adapt quickly, and insufficient testing time can lead to incomplete learning and poor performance.

C. The unknown nature and insufficient specification of the operating environment might have caused the poor performance.

This is highly likely to affect performance. Self-learning AI systems require detailed specifications of the operating environment to adapt and learn effectively. If the environment is insufficiently specified, the model may fail to perform accurately in real-world scenarios.

D. There was an algorithmic bias in the AI system.

Algorithmic bias can significantly impact the performance of AI systems. If the model has biases, it will not perform well across different scenarios and data distributions.

Given the context of the self-learning nature and the need for real-time adaptability, option A is least likely to describe the fall in performance because it deals with acceptance criteria rather than real-time performance issues​​.

The syllabus states that while pairwise testing is effective at finding defects by reducing the number of test cases needed, the resulting test suite can still be extensive and require automation:

"Even the use of pairwise testing can result in extensive test suites… automation and virtual test environments often become necessary to allow the required tests to be run."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.2, Page 67 of 99)

Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.

Testing the distribution shift in the training data for inappropriate bias (A) : This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.

Test the model during model evaluation for data bias (B) : This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.

Testing the data pipeline for any sources for algorithmic bias (C) : This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.

Check the input test data for potential sample bias (D) : While this is an important step, it focuses more on the input data and less on the overall data pipeline.

Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline is B. Test the model during model evaluation for data bias .

The ISTQB CT-AI syllabus explains in Section 1.6 – Machine Learning Approaches that supervised learning is appropriate when labeled data exists and the goal is to predict an output based on known historical examples. Predicting a customer’s purchase probability is a classification task when the output corresponds to discrete categories such as “likely to purchase” vs. “not likely to purchase.” The syllabus gives similar examples in describing classification as the process of assigning instances to predefined classes based on learned patterns in labeled data. Because the retail company wants to determine whether a customer will make a purchase based on marketing actions, classification is the most appropriate choice .

Option A (regression) predicts continuous numeric values and is less suitable because the task centers on categorical likelihood , not estimating exact monetary values. Option C (unsupervised learning) is used when labels are not available—here, the company has labeled purchase histories. Option D (reinforcement learning) requires an interactive environment with reward-driven behavior, which is not applicable to this scenario.

Thus, supervised learning (classification) is the most suitable approach according to the syllabus.

The syllabus explains that input changes that are in the same domain as what was used for training are expected to be handled with adaptability:

"Adaptability refers to the ability of a system to adjust its behavior in response to changes in its environment or inputs. This includes changes to the inputs which are still within the expected operational range of the system, such as resolution changes in images or sensor data."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 7.6 and 8.2)

The syllabus states:

“Supervised learning… divides problems into two categories: classification and regression. Regression is used when the problem requires the ML model to predict a numeric output, for example predicting the age of a person based on their habits.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.1, Page 26 of 99)

The ISTQB CT-AI syllabus describes the ML data preparation workflow in Section 2.2 – Data Preparation . Data preparation consists of data gathering , cleaning , transformation , and sampling . The syllabus emphasizes that one significant challenge during data gathering is combining data from multiple heterogeneous sources , which often differ in structure, quality, and format. Ensuring the resulting dataset is accurate, complete, and representative can be complex, making this a critical challenge in the ML workflow. This aligns directly with Option C .

Option A is incorrect because erroneous data correction is part of cleaning , not transformation. Option B contradicts the syllabus: while automation can help, not all steps should be automated due to the need for expert oversight, especially in detecting subtle data quality issues. Option D is incorrect because sampling continues to involve risk—particularly around representativeness—and the syllabus emphasizes caution, not complacency.

Thus, Option C is the only statement that accurately reflects the syllabus.

The syllabus states that non-determinism is one of the key challenges for ensuring safety in AI-based systems:

"The characteristics of AI-based systems that make it more difficult to ensure they are safe... include: complexity, non-determinism, probabilistic nature, self-learning, lack of transparency, interpretability and explainability, and lack of robustness."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.8, page 25 of 99)

The syllabus mentions that AI can help optimize regression test suites:

"An AI-based tool can perform optimization of the regression test suite by analyzing… the information from previous test results, associated defects, and the latest changes that have been made, such as features which are broken more frequently and which tests exercise code impacted by recent changes."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.4, page 79 of 99)

A. This is an example of expert system bias.

Expert system bias refers to bias introduced by the rules or logic defined by experts in the system, not by the data distribution.

B. This is an example of sample bias.

Sample bias occurs when the training data is not representative of the overall population that the model will encounter in practice. In this case, the over-representation of ethnicity A (70%) compared to B, C, and D (30%) creates a sample bias, as the model may become biased towards better performance on ethnicity A.

C. This is an example of hyperparameter bias.

Hyperparameter bias relates to the settings and configurations used during the training process, not the data distribution itself.

D. This is an example of algorithmic bias.

Algorithmic bias refers to biases introduced by the algorithmic processes and decision-making rules, not directly by the distribution of training data.

Based on the provided information, option B (sample bias) best describes the situation because the training data is skewed towards ethnicity A, potentially leading to biased model performance​​.

The syllabus explains:

"Where the internal structure of an AI-based system is too complex for humans to understand, the system can only be tested as a black box. Even when the internal structure is visible, this provides no additional useful information to help with testing."

This confirms that black-box testing is chosen because the tester does not need to understand the system’s internal structure.

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.5, page 61 of 99)

The syllabus defines clustering as:

“Clustering: This is when the problem requires the identification of similarities in input data points that allows them to be grouped based on common characteristics or attributes. For example, clustering is used to categorize different types of customers for the purpose of marketing.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.2, page 26 of 99)

The syllabus explains that EDA can be used to analyze data to identify outliers and unusual patterns, which can indicate adversarial attacks like data poisoning:

"Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.1.2, page 67 of 99)

Back-to-back testing is a method where the same set of tests are run on multiple implementations of the system to compare their outputs. This type of testing is typically used to ensure consistency and correctness by comparing the outputs of different implementations under identical conditions. Let's analyze the options given:

A. Comparison of the results of a current neural network model ML model implemented in platform A (for example Pytorch) with a similar neural network model ML model implemented in platform B (for example Tensorflow), for the same data.

This option describes a scenario where two different implementations of the same type of model are being compared using the same dataset. This is a typical back-to-back testing situation.

B. Comparison of the results of a home-grown neural network model ML model with results in a neural network model implemented in a standard implementation (for example Pytorch) for the same data.

This option involves comparing a custom implementation with a standard implementation, which is also a typical back-to-back testing scenario to validate the custom model against a known benchmark.

C. Comparison of the results of a neural network ML model with a current decision tree ML model for the same data.

This option involves comparing two different types of models (a neural network and a decision tree). This is not a typical scenario for back-to-back testing because the models are inherently different and would not be expected to produce identical results even on the same data.

D. Comparison of the results of the current neural network ML model on the current data set with a slightly modified data set.

This option involves comparing the outputs of the same model on slightly different datasets. This could be seen as a form of robustness testing or sensitivity analysis, but not typical back-to-back testing as it doesn’t involve comparing multiple implementations.

Based on this analysis, option C is the one that describes a situation of back-to-back testing the least because it compares two fundamentally different models, which is not the intent of back-to-back testing.

The ISTQB CT-AI syllabus explains that reusing pre-trained models is strongly related to similarity between the original task and the new task . Section 1.8 – Pre-trained Models and Transfer Learning states that reuse is effective when the new task is similar to the original one, such as adapting a cat-classifier to classify dog breeds. The syllabus warns about risks related to input differences , data preparation inconsistencies , inherited shortcomings , and explainability issues . These are legitimate objections (matching options A, B, and C) because large differences in image inputs or patterns can undermine transfer learning; misclassification risk can increase; and explainability often decreases when reusing pre-trained models .

However, output differences are NOT a valid concern here. Both the leaf-based and flower-based ML models classify the same plant species , meaning their outputs are identical . The syllabus does not identify output mismatch as a transfer-learning risk. Real risks concern inputs , bias inheritance , model transparency , and training differences —not output labels. Therefore, Option D describes an objection that can be safely neglected , because output classes are the same and do not hinder reuse.

In the syllabus, the evolution characteristic for AI-based systems means the ability of the system to evolve and adapt its behavior in response to changes in the environment or in its own performance:

"Evolution is the system’s ability to change itself to adapt to new situations, different hardware, or a changing operational environment."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.3)

 AI-Based Autonomous Functions: An AI-based autonomous system is one that can respond to its environment without human intervention. The other options either involve human decisions or do not use AI at all.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Autonomy and Testing Autonomous AI-Based Systems​​​​.

The ISTQB CT-AI syllabus (Section 5.2 – AI for Testing ) explains that various AI approaches can support testing activities. Probabilistic methods—one of the three major AI technique groups—are used to predict system failures , especially when dealing with uncertainty, likelihood estimation, and reliability analysis. This aligns precisely with Option B .

Option A is incorrect because regression test optimization is typically performed using search-based optimization , not classification. Option C is incorrect because fuzzy logic is more suited to reasoning under vagueness, not generating test cases. Option D is incorrect: defect prediction relies on statistical learning or classification models, not computational optimization.

Thus, Option B is the most syllabus-consistent mapping of AI techniques to testing tasks.

The syllabus details that threshold coverage requires each neuron to achieve an activation value greater than a specified threshold:

"Threshold coverage: Full threshold coverage requires that each neuron in the neural network achieves an activation value greater than a specified threshold."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 6.2, page 48 of 99)

Section 1.6.3 – Reinforcement Learning of the ISTQB CT-AI syllabus states that reinforcement learning (RL) is based on an agent interacting with an environment , performing actions, and receiving rewards or penalties . The core concept is the reward function , which guides the agent’s learning process. The syllabus emphasizes that training in RL is driven by rewards , and the agent aims to maximize cumulative reward over time. Therefore, Option C directly reflects the correct description: the agent learns by being rewarded for successful actions .

Option A is incorrect because RL does not use labeled data; that applies to supervised learning. Option B contradicts the syllabus definition: RL fundamentally requires interaction with the environment. Option D is incorrect because the reward function is defined by humans , not learned by the agent; the agent learns a policy , not the reward function itself.

Thus, Option C is the only statement consistent with RL as defined in the syllabus.

The ISTQB CT-AI syllabus describes several data quality aspects that affect ML performance. In Section 2.2 – Data Preparation , it explains that datasets may suffer from issues such as incomplete data , irrelevant data , incorrect data , unbalanced data , or data lacking preprocessing. “Incomplete data” means that portions of the required data are missing , often because some time periods, records, or sources were not captured. This aligns exactly with Option A, which correctly identifies missing intervals as incomplete data.

Option B is incorrect: “data not preprocessed” refers to data that has not undergone normalization, cleaning, or transformation—not data recorded incorrectly. Option C is wrong because irrelevant data does negatively affect ML models by introducing noise and unnecessary features. The syllabus explicitly states that including irrelevant features can degrade model learning. Option D is incorrect: “unbalanced data” relates to disproportionate class distribution , not recency or freshness of data.

Thus, Option A is the only statement that correctly matches the syllabus definition of this data quality aspect.

The syllabus states:

“When testing probabilistic and non-deterministic systems, the same input may produce different outputs. Tests need to be run several times to produce statistically valid test results, ensuring that an appropriate number of answers are accurate.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.4, page 58 of 99)

According to the syllabus, pre-trained models often inherit biases and limitations from the data and processes used in their original training, which may not align with the new use case. Specifically, the syllabus states:

"When using a pre-trained model, the training data and process cannot be fully controlled or known by the user of the model. As a result, the model can inherit biases or inaccuracies that were part of its original development and training process."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 1.8.3)