CT-AI Questions and Answers

Understanding Supervised Learning - RegressionSupervised learning is a category of machine learning where the model is trained on labeled data. Within this category,regressionis used when the goal is to predict a continuous numeric value.

Regressiondeals with problems where the output variable is continuous in nature, meaning it can take any numerical value within a range.

Common examples include predicting prices, estimating demand, and analyzing production trends.

(A) Determining the optimal age for a chicken's egg-laying production using input data of the chicken's age and average daily egg production for one million chickens.✅(Correct)

This is a classicregression problembecause it involves predicting a continuous variable:daily egg productionbased on the input variablechicken's age.

The goal is to find a numerical relationship between age and egg production, which makesregression the appropriate supervised learning method.

(B) Recognizing a knife in carry-on luggage at a security checkpoint in an airport scanner.❌(Incorrect)

This is animage recognition task, which falls underclassification, not regression.

Classification problems involve assigning inputs to discrete categories (e.g., "knife detected" or "no knife detected").

(C) Determining if an animal is a pig or a cow based on image recognition.❌(Incorrect)

This is anotherclassification problemwhere the goal is to categorize an image into one of two labels (pig or cow).

(D) Predicting shopper purchasing behavior based on the category of shopper and the positioning of promotional displays within a store.❌(Incorrect)

This problem could involve a mix ofclassificationandassociation rule learning, but it does not explicitly predict a continuous variable in the way regression does.

Regression is used when predicting a numeric output."Predicting the age of a person based on input data about their habits or predicting the future prices of stocks are examples of problems that use regression."​

Supervised learning problems are divided into classification and regression."If the output is numeric and continuous in nature, it may be regression."​

Regression is commonly used for predicting numerical trends over time."Regression models result in a numerical or continuous output value for a given input."​

Analysis of Answer ChoicesReferences from ISTQB Certified Tester AI Testing Study GuideThus,option A is the correct answer, as it aligns with the principles of regression-based supervised learning.

Bias in AI models often originates fromincomplete or non-representative training data. In this case, if the training datasetpurposely excludes specific faulty conditions, the machine learning model willfail to learn and detectthese conditions in real-world scenarios.

This results in:

Sample bias, where the training data is not fully representative of all possible faulty conditions.

Algorithmic bias, where the model prioritizes certain defect types while ignoring others.

B. Selecting training data by purposely including all known faulty conditions→ This would help reduce bias by improving model generalization.

C. Selecting testing data from a different dataset than the training dataset→ This is a good practice to evaluate model generalization but does not inherently introduce bias.

D. Selecting testing data from a boat manufacturer's bolt longevity data→ While using unrelated data can createpoor model accuracy, it does not directly introduce bias unless systematic patterns in the incorrect dataset lead to unfair decision-making.

Section 8.3 - Testing for Algorithmic, Sample, and Inappropriate Biasstates thatsample bias can occur if the training dataset is not fully representative of the expected data space, leading to biased predictions​.

Why are the other options incorrect?Reference from ISTQB Certified Tester AI Testing Study Guide:

AI-based systems can sometimes behave likeblack boxes, where the internal decision-making process is unclear.Transparencyrefers to theability to inspect and understand the training data, algorithms, and decision-making processof the AI system.

Transparency ensures that testers and stakeholders can review how an AI system was trained.

Access totraining datais a key factor in transparency because it allows testers toanalyze biases, completeness, and representativenessof the dataset.

Transparency is an essential characteristic of explainable AI (XAI).

Having access to training data means that testers can investigate how data influences AI behavior.

Regulatory and ethical AI guidelines emphasize transparency.

Many AI ethics frameworks, such asGDPR and Trustworthy AI guidelines, recommend transparency to ensurefair and explainable AI decision-making.

(A) Autonomy❌

Autonomy refers to an AI system’s ability to make decisions independentlywithout human intervention. However,having access to training data does not relate to autonomy, which is more about self-learning and decision-making without human control.

(B) Explorability❌

Explorability refers to the ability to test AI systems interactivelyto understand their behavior, but it does not directly relate to accessing training data.

(D) Accessibility❌

Accessibility refers to the ease with which people can use the system, not the ability to inspect the training data.

Transparency is the ease with which the training data and algorithm used to generate a model can be understood."Transparency: This is considered to be the ease with which the algorithm and training data used to generate the model can be determined."​

Why is Option C Correct?Why Other Options are Incorrect?References from ISTQB Certified Tester AI Testing Study GuideThus,option C is the correct answer, astransparency involves access to training data, allowing testers to understand AI decision-making processes.

Adeep neural network (DNN)is a type of artificial neural network that consists of multiple layers between the input and output layers. TheISTQB Certified Tester AI Testing (CT-AI) Syllabusoutlines the following characteristics of a DNN:

Structure of a Deep Neural Network:

ADNN comprises at least three types of layers:

Input layer: Receives the input data.

Hidden layers: Perform complex feature extraction and transformations.

Output layer: Produces the final prediction or classification​.

Analysis of Answer Choices:

A (Only input and output layers)→ Incorrect, as a DNN must haveat least one hidden layer.

B (At least one internal hidden layer)→ Correct, as a neural network must havehidden layersto be considered deep.

C (Minimum of five layers required)→ Incorrect, asthere is no strict definitionthat requires at least five layers.

D (Output layer is not connected to other layers)→ Incorrect, asthe output layer must be connectedto the hidden layers​.

Thus,Option B is the correct answer, asa deep neural network must have at least one hidden layer.

Certified Tester AI Testing Study Guide References:

ISTQB CT-AI Syllabus v1.0, Section 6.1 (Neural Networks and Deep Neural Networks)

ISTQB CT-AI Syllabus v1.0, Section 6.2 (Structure of Deep Neural Networks)​.

Pairwise testing is a combinatorial testing technique that reduces the number of test cases by focusing on testing interactions between pairs of parameters rather than all possible combinations. It is widely used in AI-based systems, including autonomous vehicles, where the number of possible input parameter combinations can be extremely high​.

Option A:“The number of parameters to test can be reduced to less than a dozen.”

This is incorrect. While pairwise testing significantly reduces the number of test cases, it does not necessarily limit them to a fixed number like a dozen. The final number of tests depends on the number of parameters and their possible values​.

Option B:“All high priority defects will be identified using this method.”

This is incorrect. While pairwise testing is effective in detecting defects caused by interactions between two parameters, it may not uncover defects resulting from more complex interactions involving three or more parameters​.

Option C:“While the number of tests needed can be reduced, there may still be a large enough set of tests that automation will be required to execute all of them.”

This is the correct answer. Even though pairwise testing reduces the number of test cases, AI-based systems such as autonomous vehicles still have a large number of test scenarios. Therefore, automation is often necessary to execute all test cases within the available time​.

Option D:“Pairwise cannot be applied to this problem because there is AI involved, and the evolving values may result in unexpected results that cannot be verified.”

This is incorrect. Pairwise testing can still be applied to AI-based systems, including those that evolve over time. However, additional testing techniques may be required to verify evolving behavior​.

Pairwise Testing for AI Systems:"Pairwise testing is widely used because it effectively reduces the number of test cases while maintaining defect detection capability"​.

Automation Requirement:"In practice, even with pairwise testing, extensive test suites may still require automation"​.

Analysis of the Answer Options:ISTQB CT-AI Syllabus References:

 Factors Affecting ML Functional Performance:The data pipeline, quality of the labeling, and biased data are all factors that significantly affect the performance of machine learning models. The number of classes, while relevant for the model structure, is not a direct factor affecting the performance metrics such as accuracy or bias.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Sections on Data Quality and its Effect on the ML Model and ML Functional Performance Metrics​​​​.

AI-based tools can significantly optimize regression test suites by analyzing historical data, past test results, associated defects, and changes made to the software. These tools prioritize and select the most relevant test cases based on previous defect patterns and frequently failing features, which helps in reducing the test execution time while maintaining effectiveness.

The optimization process involves:

Prioritizing test cases:AI-based tools rank test cases based on past defect detection trends, ensuring that the most relevant tests are executed first.

Reducing redundant test cases:The tool can eliminate test cases that do not contribute significantly to defect detection, reducing overall test execution time.

Augmenting test cases:The AI can also suggest new test cases if certain features are more prone to defects.

This approach has been proven to reduce regression test suite sizes by up to 50% while maintaining fault detection capabilities​.

Section 11.4 - Using AI for the Optimization of Regression Test Suitesstates that AI-based tools can optimize regression test suites by analyzing past test data and defect occurrences, leading to significant reductions in test execution time​.

Reference from ISTQB Certified Tester AI Testing Study Guide:

A. This is an example of expert system bias.

Expert system bias refers to bias introduced by the rules or logic defined by experts in the system, not by the data distribution.

B. This is an example of sample bias.

Sample bias occurs when the training data is not representative of the overall population that the model will encounter in practice. In this case, the over-representation of ethnicity A (70%) compared to B, C, and D (30%) creates a sample bias, as the model may become biased towards better performance on ethnicity A.

C. This is an example of hyperparameter bias.

Hyperparameter bias relates to the settings and configurations used during the training process, not the data distribution itself.

D. This is an example of algorithmic bias.

Algorithmic bias refers to biases introduced by the algorithmic processes and decision-making rules, not directly by the distribution of training data.

Based on the provided information, optionB(sample bias) best describes the situation because the training data is skewed towards ethnicity A, potentially leading to biased model performance​​.

 Type of Testing for Stock-Price Prediction Models:Concept drift refers to the change in the statistical properties of the target variable over time. Severe weather storms causing massive losses in coffee production and affecting stock prices would require testing for concept drift to ensure that the model adapts to new patterns in data over time.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 7.6 Testing for Concept Drift, which explains the need to test for concept drift in models that might be affected by changing external factors​​.

AI-based systems have specific quality characteristics, includingevolution,autonomy, andadaptability. A test objective that evaluates whether an AI systemevolvesto improve performance over time directly aligns with AI quality characteristics​.

Explanation of Answer Choices:

Option A: The robot must evolve to optimize its routing.

Correct.Evolution is an AI quality characteristic that ensures the systemlearns from past experiencesand adapts to improve efficiency​.

Option B: The robot must recharge for no more than six hours a day.

Incorrect.This is an operational constraint rather than an AI-specific quality characteristic​.

Option C: The robot must record the time of each delivery which is compiled into a report.

Incorrect.Logging data does not relate to AI quality characteristics likeadaptability or autonomy​.

Option D: The robot must complete 99.99% of its deliveries each day.

Incorrect.This is a performance target rather than an AI quality characteristic​.

ISTQB CT-AI Syllabus References:

Evolution as an AI Quality Characteristic:"Check how well the system learns from its own experience. Check how well the system copes when the profile of data changes (i.e., concept drift)"​.

Thus,Option A is the best choice as it directly tests an AI quality characteristic (evolution) in the upgraded autonomous robot.

Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.

Testing the distribution shift in the training data for inappropriate bias (A): This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.

Test the model during model evaluation for data bias (B): This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.

Testing the data pipeline for any sources for algorithmic bias (C): This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.

Check the input test data for potential sample bias (D): While this is an important step, it focuses more on the input data and less on the overall data pipeline.

Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline isB. Test the model during model evaluation for data bias.

References:

ISTQB CT-AI Syllabus Section 8.3 on Testing for Algorithmic, Sample, and Inappropriate Bias discusses various tests that can be performed to detect biases at different stages of the ML pipeline.

Sample Exam Questions document, Question #32 highlights the importance of evaluating the model for biases.

The question refers to a problem where data used for an object detection ML system was labelled incorrectly. This issue is most closely related to "accuracy issues." Here's a detailed explanation:

Accuracy Issues: The primary goal of labeling data in machine learning is to ensure that the model can accurately learn and make predictions based on the given labels. Incorrectly labeled data directly impacts the model's accuracy, leading to poor performance because the model learns incorrect patterns.

Why Not Other Options:

Security Issues: This pertains to data breaches or unauthorized access, which is not relevant to the problem of incorrect data labeling.

Privacy Issues: This concerns the protection of personal data and is not related to the accuracy of data labeling.

Bias Issues: While bias in data can affect model performance, it specifically refers to systematic errors or prejudices in the data rather than outright incorrect labeling.

References:This explanation is consistent with the concepts covered in the ISTQB CT-AI syllabus under dataset quality issues and their impact on machine learning models​​.

A. Add 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Adding more data to the training set can affect the R-Square score, but it does not directly verify the correctness of the implementation.

B. Train various models by changing the order of input features and verify that the R-Square score of these models vary significantly.

Changing the order of input features should not significantly affect the R-Square score if the implementation is correct, but this approach is more about testing model robustness rather than correctness of the implementation.

C. Compare the R-Square score of the model obtained using two different implementations that utilize two different programming languages while using the same algorithm and the same training and testing data.

This approach directly compares the performance of two implementations of the same algorithm. If both implementations produce similar R-Square scores on the same training and testing data, it suggests that the new implementation "X" is correct.

D. Drop 10% of the rows randomly and create another model and compare the R-Square scores of both the models.

Dropping data can lead to variations in the R-Square score but does not directly verify the correctness of the implementation.

Therefore, optionCis the most appropriate strategy because it directly compares the performance of the new implementation "X" with another implementation using the same algorithm and datasets, which helps in verifying the correctness of the implementation​​.

In a neural network, the activation value of a neuron is determined by a combination of inputs from the previous layer, the weights of the connections, and the bias at the neuron level. Here’s a detailed breakdown:

Inputs for Activation Value:

Activation Values of Neurons in the Previous Layer:These are the outputs from neurons in the preceding layer that serve as inputs to the current neuron.

Weights Assigned to the Connections:Each connection between neurons has an associated weight, which determines the strength and direction of the input signal.

Individual Bias at the Neuron Level:Each neuron has a bias value that adjusts the input sum, allowing the activation function to be shifted.

Calculation:

The activation value is computed by summing the weighted inputs from the previous layer and adding the bias.

Formula: z=∑(wi⋅ai)+bz = \sum (w_i \cdot a_i) + bz=∑(wi​⋅ai​)+b, where wiw_iwi​ are the weights, aia_iai​ are the activation values from the previous layer, and bbb is the bias.

The activation function (e.g., sigmoid, ReLU) is then applied to this sum to get the final activation value.

Why Option A is Correct:

Option A correctly identifies all components involved in computing the activation value: the individual bias, the activation values of the previous layer, and the weights of the connections.

Eliminating Other Options:

B. Activation values of neurons in the previous layer, and weights assigned to the connections between the neurons: This option misses the bias, which is crucial.

C. Individual bias at the neuron level, and weights assigned to the connections between the neurons: This option misses the activation values from the previous layer.

D. Individual bias at the neuron level, and activation values of neurons in the previous layer: This option misses the weights, which are essential.

References:

ISTQB CT-AI Syllabus, Section 6.1, Neural Networks, discusses the components and functioning of neurons in a neural network.

"Neural Network Activation Functions" (ISTQB CT-AI Syllabus, Section 6.1.1).

The process of developing a machine learning model typically involves the use of three types of datasets:

Training Data:This is used to train the model, i.e., to learn the patterns and relationships in the data.

Validation Data:This is used to tune the model’s hyperparameters and to prevent overfitting during the training process.

Test Data:This is used to evaluate the final model’s performance and to estimate how it will perform on unseen data.

Let's analyze each option:

A. Training data - validation data - test data

This option correctly includes all three types of datasets used in the process of creating and validating a model. The training data is used for learning, validation data for tuning, and test data for final evaluation.

B. Training data - validation data

This option misses the test data, which is crucial for evaluating the model’s performance on unseen data after the training and validation phases.

C. Training data - test data

This option misses the validation data, which is important for tuning the model and preventing overfitting during training.

D. Validation data - test data

This option misses the training data, which is essential for the initial learning phase of the model.

Therefore, the correct answer isAbecause it includes all necessary datasets used during the process of learning and creating the model: training, validation, and test data​​.

A. Feature selection

Feature selection is the process of selecting the most relevant features from the data. While important, it is not directly about handling excess data.

B. Data sampling

Data sampling involves selecting a representative subset of the data for training. When there is more data than needed, sampling can be used to create a manageable dataset that maintains the statistical properties of the full dataset.

C. Data labeling

Data labeling involves annotating data for supervised learning. It is necessary for training models but does not address the issue of having excess data.

D. Data augmentation

Data augmentation is used to increase the size of the training dataset by creating modified versions of existing data. It is useful when there is insufficient data, not when there is excess data.

Therefore, the correct answer isBbecause data sampling is the most relevant activity when dealing with an excess amount of data for training​​.

According to the ISTQB Certified Tester AI Testing (CT-AI) syllabus, data acquisition, a critical step in data preparation for machine learning (ML) workflows, consists of three key activities:

Identification:This step involves determining the types of data required for training and prediction. For example, in a self-driving car application, data types such as radar, video, laser imaging, and LiDAR (Light Detection and Ranging) data may be identified as necessary sources.

Gathering:After identifying the required data types, the sources from which the data will be collected are determined, along with the appropriate collection methods. An example could be gathering financial data from the International Monetary Fund (IMF) and integrating it into an AI-based system.

Labeling:This process involves annotating or tagging the collected data to make it meaningful for supervised learning models. Labeling is an essential activity that helps machine learning algorithms differentiate between categories and make accurate predictions.

These activities ensure that the data is suitable for training and testing machine learning models, forming the foundation of data preparation​.

AI-based systems oftenexhibit non-deterministic behavior, meaning theydo not always produce the same output for the same input. This makesensuring safety more difficult, as the system's behavior can change based on new data, environmental factors, or updates​.

Why Non-determinism Affects Safety:

In traditional software, the same input always produces the same output.

In AI systems, outputsvary probabilisticallydepending on learned patterns and weights.

This unpredictability makes itharder to verify correctness, reliability, and safety, especially in critical domains likeautonomous vehicles, medical AI, and industrial automation.

A (Simplicity):AI-based systems are typicallycomplex, not simple, which contributes to safety challenges​.

B (Sustainability):While sustainability is an important AI consideration, it doesnot directly affect safety​.

D (Robustness):Lack of robustnesscan make AI systems unsafe, butnon-determinism is the primary issuethat complicates safety verification​.

ISTQB CT-AI Syllabus (Section 2.8: Safety and AI)

"The characteristics of AI-based systems that make it more difficult to ensure they are safe include: complexity, non-determinism, probabilistic nature, self-learning, lack of transparency, interpretability and explainability, lack of robustness"​.

Why Other Options Are Incorrect:Supporting References from ISTQB Certified Tester AI Testing Study Guide:Conclusion:Sincenon-determinism makes AI behavior unpredictable, complicating safety assurance, thecorrect answer is C.

 A. Using a pixel comparison of the GUI before and after the change to check the differences.

Pixel comparison is a traditional method and does not involve AI. It compares images at the pixel level, which can be effective but is not an intelligent approach. It is not considered an AI usage and is the least likely usage of AI for detecting GUI changes.

 B. Using computer vision to compare the GUI before and after the test object changes.

Computer vision involves using AI techniques to interpret and process images. It is a likely usage of AI for detecting changes in the GUI.

 C. Using vision-based detection of the GUI layout changes before and after test object changes.

Vision-based detection is another AI technique where the layout and structure of the GUI are analyzed to detect changes. This is a typical application of AI.

 D. Using a ML-based classifier to flag if changes in GUI are to be flagged for humans.

An ML-based classifier can intelligently determine significant changes and decide if they need human review, which is a sophisticated AI application.