CT-AI_(v1.0)_World ISTQB Certified Tester AI Testing (v 1.0) Questions and Answers

The question asks which characteristic is least likely to cause safety-related issues for an AI system. Let's evaluate each option:

Non-determinism (A) : Non-deterministic systems can produce different outcomes even with the same inputs, which can lead to unpredictable behavior and potential safety issues.

Robustness (B) : Robustness refers to the ability of the system to handle errors, anomalies, and unexpected inputs gracefully. A robust system is less likely to cause safety issues because it can maintain functionality under varied conditions.

High complexity (C) : High complexity in AI systems can lead to difficulties in understanding, predicting, and managing the system's behavior, which can cause safety-related issues.

Self-learning (D) : Self-learning systems adapt based on new data, which can lead to unexpected changes in behavior. If not properly monitored and controlled, this can result in safety issues.

References :

ISTQB CT-AI Syllabus Section 2.8 on Safety and AI discusses various factors affecting the safety of AI systems, emphasizing the importance of robustness in maintaining safe operation.

Flexibility in AI systems is crucial for various reasons, particularly because it allows for easier modification and adaptation of the system as a whole.

AI systems are inherently flexible (A) : This statement is not correct. While some AI systems may be designed to be flexible, they are not inherently flexible by nature. Flexibility depends on the system's design and implementation.

AI systems require changing operational environments; therefore, flexibility is required (B) : While it's true that AI systems may need to operate in changing environments, this statement does not directly address the importance of flexibility for the modification of the system.

Flexible AI systems allow for easier modification of the system as a whole (C) : This statement correctly describes the importance of flexibility. Being able to modify AI systems easily is critical for their maintenance, adaptation to new requirements, and improvement.

Self-learning systems are expected to deal with new situations without explicitly having to program for it (D) : This statement relates to the adaptability of self-learning systems rather than their overall flexibility for modification.

Hence, the correct answer is C. Flexible AI systems allow for easier modification of the system as a whole .

References :

ISTQB CT-AI Syllabus Section 2.1 on Flexibility and Adaptability discusses the importance of flexibility in AI systems and how it enables easier modification and adaptability to new situations.

Sample Exam Questions document, Question #30 highlights the importance of flexibility in AI systems.

 A. Technological singularity

Technological singularity refers to a hypothetical point in the future when AI surpasses human intelligence and can continuously improve itself without human intervention. This scenario involves capabilities far beyond those described in the question.

 B. Narrow AI

Narrow AI, also known as weak AI, is designed to perform a specific task or a narrow range of tasks. It does not have general cognitive abilities and cannot learn multiple tasks by itself without retraining.

 C. Super AI

Super AI refers to an AI that surpasses human intelligence and capabilities across all fields. This is an advanced concept and not aligned with the description of having cognitive abilities similar to a young child.

 D. General AI

General AI, or strong AI, has the ability to understand, learn, and apply knowledge across a wide range of tasks, similar to human cognitive abilities. It aligns with the description of AI that can learn multiple tasks and operate in new environments without needing retraining.

Recognizing digits from a scan of handwritten numbers using machine learning is an example of classification. Here’s a breakdown:

Classification : This type of machine learning involves categorizing input data into predefined classes. In this scenario, the input data (handwritten digits) are classified into one of the 10 digit classes (0-9).

Why Not Other Options :

Reinforcement Learning : This involves learning by interacting with an environment to achieve a goal, which does not fit the problem of recognizing digits.

Regression : This is used for predicting continuous values, not discrete categories like digit recognition.

Clustering : This involves grouping similar data points together without predefined classes, which is not the case here.

References: The explanation is based on the definitions of different machine learning types as outlined in the ISTQB CT-AI syllabus, specifically under supervised learning and classification​​.

In a neural network, the activation value of a neuron is determined by a combination of inputs from the previous layer, the weights of the connections, and the bias at the neuron level. Here’s a detailed breakdown:

Inputs for Activation Value:

Activation Values of Neurons in the Previous Layer: These are the outputs from neurons in the preceding layer that serve as inputs to the current neuron.

Weights Assigned to the Connections: Each connection between neurons has an associated weight, which determines the strength and direction of the input signal.

Individual Bias at the Neuron Level: Each neuron has a bias value that adjusts the input sum, allowing the activation function to be shifted.

Calculation:

The activation value is computed by summing the weighted inputs from the previous layer and adding the bias.

Formula: z=∑(wi ⋅ ai)+bz = \sum (w_i \cdot a_i) + bz=∑(wi​ ⋅ ai​)+b, where wiw_iwi​ are the weights, aia_iai​ are the activation values from the previous layer, and bbb is the bias.

The activation function (e.g., sigmoid, ReLU) is then applied to this sum to get the final activation value.

Why Option A is Correct:

Option A correctly identifies all components involved in computing the activation value: the individual bias, the activation values of the previous layer, and the weights of the connections.

Eliminating Other Options:

B. Activation values of neurons in the previous layer, and weights assigned to the connections between the neurons : This option misses the bias, which is crucial.

C. Individual bias at the neuron level, and weights assigned to the connections between the neurons : This option misses the activation values from the previous layer.

D. Individual bias at the neuron level, and activation values of neurons in the previous layer : This option misses the weights, which are essential.

References:

ISTQB CT-AI Syllabus, Section 6.1, Neural Networks, discusses the components and functioning of neurons in a neural network.

"Neural Network Activation Functions" (ISTQB CT-AI Syllabus, Section 6.1.1).

 Factors Affecting ML Functional Performance: The data pipeline, quality of the labeling, and biased data are all factors that significantly affect the performance of machine learning models. The number of classes, while relevant for the model structure, is not a direct factor affecting the performance metrics such as accuracy or bias.

 Reference: ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Quality and its Effect on the ML Model and ML Functional Performance Metrics​​​​.

Defect prediction models aim to identify parts of the software that are likely to contain defects by analyzing historical data and code quality metrics. The primary goal is to use this predictive information to allocate testing and maintenance resources effectively. Let's break down why option D is the correct choice:

Understanding Classification Models:

Classification models are a type of supervised learning algorithm used to categorize or classify data into predefined classes or labels. In the context of defect prediction, the classification model would classify parts of the code as either "defective" or "non-defective" based on the input features.

Input Data - Code Quality Metrics:

The input data for these classification models typically includes various code quality metrics such as cyclomatic complexity, lines of code, number of methods, depth of inheritance, coupling between objects, etc. These metrics help the model learn patterns associated with defects.

Historical Data:

Historical versions of the code along with their defect records provide the labeled data needed for training the classification model. By analyzing this historical data, the model can learn which metrics are indicative of defects.

Why Option D is Correct:

Option D specifies using a classification model to predict the presence of defects by using code quality metrics as input data. This accurately describes the process of defect prediction using historical bug data and quality metrics.

Eliminating Other Options:

A. Identifying the relationship between developers and the modules developed by them : This does not directly involve predicting defects based on code quality metrics and historical data.

B. Search of similar code based on natural language processing : While useful for other purposes, this method does not describe defect prediction using classification models and code metrics.

C. Clustering of similar code modules to predict based on similarity : Clustering is an unsupervised learning technique and does not directly align with the supervised learning approach typically used in defect prediction models.

References:

ISTQB CT-AI Syllabus, Section 9.5, Metamorphic Testing (MT), describes various testing techniques including classification models for defect prediction.

"Using AI for Defect Prediction" (ISTQB CT-AI Syllabus, Section 11.5.1).

A. Add 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Adding more data to the training set can affect the R-Square score, but it does not directly verify the correctness of the implementation.

B. Train various models by changing the order of input features and verify that the R-Square score of these models vary significantly.

Changing the order of input features should not significantly affect the R-Square score if the implementation is correct, but this approach is more about testing model robustness rather than correctness of the implementation.

C. Compare the R-Square score of the model obtained using two different implementations that utilize two different programming languages while using the same algorithm and the same training and testing data.

This approach directly compares the performance of two implementations of the same algorithm. If both implementations produce similar R-Square scores on the same training and testing data, it suggests that the new implementation "X" is correct.

D. Drop 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Dropping data can lead to variations in the R-Square score but does not directly verify the correctness of the implementation.

Therefore, option C is the most appropriate strategy because it directly compares the performance of the new implementation "X" with another implementation using the same algorithm and datasets, which helps in verifying the correctness of the implementation​​.

A/B testing, also known as split testing, is a method used to compare two versions of a product or system to determine which one performs better. It is widely used in web development, marketing, and machine learning to optimize user experiences and model performance. Here’s why option C is the least descriptive of an A/B testing scenario:

Understanding A/B Testing:

In A/B testing, two versions (A and B) of a system or feature are tested against each other. The objective is to measure which version performs better based on predefined metrics such as user engagement, conversion rates, or other performance indicators.

Application in Machine Learning:

In ML systems, A/B testing might involve comparing two different models, algorithms, or system configurations on the same set of data to observe which yields better results.

Why Option C is the Least Descriptive:

Option C describes comparing the performance of an ML system on two different input datasets. This scenario focuses on the input data variation rather than the comparison of system versions or features, which is the essence of A/B testing. A/B testing typically involves a controlled experiment with two versions being tested under the same conditions, not different datasets.

Clarifying the Other Options:

A. A comparison of two different websites for the same company to observe from a user acceptance perspective : This is a classic example of A/B testing where two versions of a website are compared.

B. A comparison of two different offers in a recommendation system to decide on the more effective offer for the same users : This is another example of A/B testing in a recommendation system.

D. A comparison of the performance of two different ML implementations on the same input data : This fits the A/B testing model where two implementations are compared under the same conditions.

References:

ISTQB CT-AI Syllabus, Section 9.4, A/B Testing, explains the methodology and application of A/B testing in various contexts.

"Understanding A/B Testing" (ISTQB CT-AI Syllabus).