Free CT-AI sample questions

Correct answer: The AI tool will not know the expected outcome for the generated tests, creating a significant test oracle problem.

This is the fundamental limitation of most AI-based test generation tools. While excellent at exploring the state space and generating novel inputs (the 'how to test'), they lack the domain-specific knowledge to determine what the correct output should be. This is known as the test oracle problem. The most common use for such tools is to find crashes or server errors (5xx), for which the oracle is simple, rather than to find subtle logical bugs.

Correct answer: Using a model-agnostic technique like LIME (Local Interpretable Model-Agnostic Explanations) to generate explanations for individual predictions by analyzing how small input perturbations affect the outcome.

For a complex black-box model like a DNN, direct inspection of weights is not human-interpretable. LIME is designed specifically for this use case: providing local, understandable explanations for individual predictions of any black-box model. This allows a clinician to ask 'Why did the model flag this specific patient?' and get an answer based on the most influential features (e.g., 'because of high heart rate and low blood pressure'), making it ideal for validating explainability in a clinical setting.

Correct answer: False

The statement is false. Neuron coverage is a very weak structural coverage criterion, analogous to statement coverage in traditional code. It only ensures that each neuron has produced an output above a certain threshold at least once. It does not test the complex interactions between neurons, the different activation ranges, or the combinatorial logic of the network. Therefore, it provides no guarantee that the model is free of logical flaws or defects.

Correct answer: Outsourcing the labeling to a crowdsourcing platform where each image is labeled by multiple (e.g., three) different workers, and using the majority vote as the final label.

This is a standard industry best practice for large-scale labeling tasks. Crowdsourcing is fast and cost-effective for a large dataset. The key to quality control in this approach is redundancy: having multiple independent workers label the same data item. A consensus mechanism, like a majority vote, is then used to establish a higher-confidence ground truth label, effectively filtering out random errors and individual worker biases. This balances speed, cost, and quality.

Correct answers: Introduce a small number of mislabeled images into the training dataset (e.g., images of 'Stop' signs labeled as 'Speed Limit 60') and observe if the trained model's accuracy degrades significantly or learns a backdoor.; Generate adversarial examples by applying subtle, almost imperceptible pixel-level noise to a valid image of a stop sign and checking if it causes a misclassification (e.g., to 'Green Light').

This activity directly simulates a data poisoning attack. By intentionally corrupting a portion of the training data, testers can evaluate the model's resilience and determine if it can be manipulated into making specific, incorrect classifications.

This describes the process of an adversarial attack. The goal is to create inputs that are visually indistinguishable from legitimate ones to a human but are specifically crafted to fool the model. Testing with such examples is crucial for assessing robustness against malicious attacks in production.

Correct answer: Unsupervised Learning (Association)

The scenario describes market basket analysis, which is a classic example of association rule mining. Since the goal is to discover hidden patterns and relationships in unlabeled transactional data, it falls under the category of Unsupervised Learning, and more specifically, Association. Classification and Regression are supervised techniques that require labeled data, and Reinforcement Learning involves an agent learning through trial and error with rewards, which is not applicable here.

Correct answer: It enables the safe, repeatable, and scalable testing of rare and dangerous edge-case scenarios (e.g., multiple drones converging, sudden obstacles).

The core value of simulation for autonomous systems is the ability to test high-risk scenarios without real-world consequences. A virtual environment allows testers to create and repeat dangerous edge cases (like near-misses or sensor failures) thousands of times to ensure robustness. This is impractical, unsafe, and prohibitively expensive to do in a physical environment. This makes simulation an indispensable tool for testing safety-critical AI systems.

Correct answer: Recall (Sensitivity)

The scenario explicitly states that the cost of a False Negative (missing a fraudulent transaction) is very high. Recall is calculated as TP / (TP + FN). To maximize Recall, the number of False Negatives must be minimized. Therefore, Recall is the most important metric to prioritize when the business cost of missing a positive case (in this case, fraud) is high. Precision would be prioritized if the cost of False Positives was the main concern.

Correct answer: Overfitting, caused by the model learning the noise and specific details of the training data too well.

A large performance gap where the model performs exceptionally well on the data it was trained on, but poorly on new, unseen data (the validation set), is the classic definition of overfitting. This occurs when the model has memorized the training examples, including their noise, instead of learning the general underlying patterns. This makes it unable to generalize to new data.

Correct answer: Back-to-Back Testing

Back-to-back testing (or differential testing) is a technique where the system under test is executed with the same inputs as a reference system (the pseudo-oracle), and their outputs are compared. Using a previous, trusted version of the model as the reference system is a classic application of this technique for regression testing of complex or non-deterministic systems where a precise expected result is hard to define.