CT-AI Online Practice Questions

Explanation:
The process of developing a machine learning model typically involves the use of three types of datasets:
Training Data: This is used to train the model, i.e., to learn the patterns and relationships in the data.
Validation Data: This is used to tune the model’s hyperparameters and to prevent overfitting during the training process.
Test Data: This is used to evaluate the final model’s performance and to estimate how it will perform on unseen data.
Let's analyze each option:
A. Training data - validation data - test data
This option correctly includes all three types of datasets used in the process of creating and validating a model. The training data is used for learning, validation data for tuning, and test data for final evaluation.
B. Training data - validation data
This option misses the test data, which is crucial for evaluating the model’s performance on unseen data after the training and validation phases.
C. Training data - test data
This option misses the validation data, which is important for tuning the model and preventing overfitting during training.
D. Validation data - test data
This option misses the training data, which is essential for the initial learning phase of the model.
Therefore, the correct answer is A because it includes all necessary datasets used during the process of learning and creating the model: training, validation, and test data​.

Explanation:
A/B testing, also known as split testing, is a method used to compare two versions of a product or system to determine which one performs better. It is widely used in web development, marketing, and machine learning to optimize user experiences and model performance.
Here’s why option C is the least descriptive of an A/B testing scenario:
Understanding A/B Testing:
In A/B testing, two versions (A and B) of a system or feature are tested against each other. The objective is to measure which version performs better based on predefined metrics such as user engagement, conversion rates, or other performance indicators.
Application in Machine Learning:
In ML systems, A/B testing might involve comparing two different models, algorithms, or system
configurations on the same set of data to observe which yields better results.
Why Option C is the Least Descriptive:
Option C describes comparing the performance of an ML system on two different input datasets. This scenario focuses on the input data variation rather than the comparison of system versions or features, which is the essence of A/B testing. A/B testing typically involves a controlled experiment with two versions being tested under the same conditions, not different datasets.
Clarifying the Other Options:
A. A comparison of two different websites for the same company to observe from a user acceptance perspective: This is a classic example of A/B testing where two versions of a website are compared.
B. A comparison of two different offers in a recommendation system to decide on the more effective offer for the same users: This is another example of A/B testing in a recommendation system.
D. A comparison of the performance of two different ML implementations on the same input data: This fits the A/B testing model where two implementations are compared under the same conditions.
Reference: ISTQB CT-AI Syllabus, Section 9.4, A/B Testing, explains the methodology and application of A/B testing in various contexts.
"Understanding A/B Testing" (ISTQB CT-AI Syllabus).

Explanation:
A. 64-bit CPUs.
While 64-bit CPUs are essential for handling large amounts of memory and performing complex computations, they are not specifically optimized for the types of operations commonly used in machine learning.
B. Hardware supporting fast matrix multiplication.
Matrix multiplication is a fundamental operation in many machine learning algorithms, especially in neural networks and deep learning. Hardware optimized for fast matrix multiplication, such as GPUs (Graphics Processing Units), is most suitable for implementing AI and ML because it can handle the parallel processing required for these operations efficiently.
C. High powered CPUs.
High powered CPUs are beneficial for general-purpose computing tasks and some aspects of ML, but they are not as efficient as specialized hardware like GPUs for matrix multiplication and other ML-specific tasks.
D. Hardware supporting high precision floating point operations.
High precision floating point operations are important for scientific computing and some specific AI tasks, but for many ML applications, fast matrix multiplication is more critical than high precision alone.
Therefore, the correct answer is B because hardware supporting fast matrix multiplication, such as GPUs, is most suitable for the parallel processing requirements of machine learning​.

Explanation:
Detecting biases in the ML pipeline involves various tests to ensure fairness and accuracy throughout the ML process.
Testing the distribution shift in the training data for inappropriate bias (A): This involves checking if there is any shift in the data distribution that could lead to bias in the model. It is an important test but not the most direct method for detecting biases.
Test the model during model evaluation for data bias (B): This is a critical stage where the model is evaluated to detect any biases in the data it was trained on. It directly addresses potential data biases in the model.
Testing the data pipeline for any sources for algorithmic bias (C): This test is crucial as it helps identify biases that may originate from the data processing and transformation stages within the pipeline. Detecting sources of algorithmic bias ensures that the model does not inherit biases from these processes.
Check the input test data for potential sample bias (D): While this is an important step, it focuses more on the input data and less on the overall data pipeline.
Hence, the most likely useful test to help detect different kinds of biases in the ML pipeline is
B. Test the model during model evaluation for data bias.
Reference: ISTQB CT-AI Syllabus Section 8.3 on Testing for Algorithmic, Sample, and Inappropriate Bias discusses various tests that can be performed to detect biases at different stages of the ML pipeline.
Sample Exam Questions document, Question #32 highlights the importance of evaluating the model for biases.

Explanation:
Coverage for Neuron Activation Values: Sign change coverage is used to ensure that test cases cause each neuron to achieve both positive and negative activation values. This type of coverage ensures that the neurons are thoroughly tested under different activation states.
Reference: ISTQB_CT-AI_Syllabus_v1.0, Section 6.2 Coverage Measures for Neural Networks, which details different types of coverage measures, including sign change coverage​.