Free AAIA sample questions

Correct answers: Reviewing the vendor's data processing agreement and terms of service.; Verifying that personally identifiable information (PII) is masked or removed before data is sent to the vendor.

The data processing agreement (DPA) and terms of service are legally binding documents that outline how the vendor will handle the organization's data, including data ownership, usage rights, security controls, and compliance with privacy regulations like GDPR. This is a primary source of evidence for assessing privacy risk.

A crucial internal control to mitigate third-party privacy risk is to minimize the data shared. Verifying that the organization has a process to scrub PII before sending data to the vendor's service directly reduces the risk of a data breach or misuse of customer PII by the third party.

Correct answer: The plan lacks specific procedures for containing and remediating AI model failures like severe drift or biased outputs.

AI systems introduce unique failure modes not found in traditional IT systems. These include model drift, performance degradation, generation of harmful or biased outputs, and adversarial attacks. An effective incident response plan must include specific playbooks for identifying, containing (e.g., taking the model offline, reverting to a previous version), and remediating these AI-specific incidents.

Correct answer: Understanding the business objectives and identifying the associated high-level risks.

According to standard audit practices, the first step in planning any audit is to understand the context: what the system is supposed to achieve (business objectives) and what could go wrong (high-level risks). For a generative AI chatbot, risks include reputational damage from inappropriate responses, data leakage, and inaccurate information. This understanding drives the entire audit scope and approach.

Correct answer: False

This statement is false. While statistical sampling allows for quantifiable measurement of sampling risk, it does not eliminate all bias. Furthermore, in AI auditing, judgmental (or risk-based) sampling is often necessary to target high-risk areas, such as testing the model's behavior on known edge cases or transactions involving sensitive data, which might be missed by a purely random statistical sample.

Correct answer: Performing root cause analysis after a machine failure occurs.

If the model predicts a failure but cannot explain *why* (e.g., which sensor reading or combination of factors led to the prediction), it is extremely difficult for maintenance engineers to perform targeted diagnostics and root cause analysis. They know the machine is likely to fail, but they don't know which component is the problem. This lack of insight hinders proactive repair and learning from potential failures.

Correct answer: The variance in average price offered for the same product across different customer demographic segments.

A key risk indicator must be a measurable metric that provides an early warning of increasing risk. Measuring the statistical variance in pricing across defined demographic segments (e.g., based on geography, inferred income, etc.) directly targets the risk of price discrimination. A rising variance would trigger an investigation, making it the most effective KRI for this specific risk.

Correct answer: The audit scope must be expanded to verify compliance with specific, stringent requirements for risk management, data governance, transparency, and human oversight.

The 'high-risk' classification under the EU AI Act mandates a comprehensive set of obligations. The audit scope must therefore be significantly expanded to include conformity assessments against these legal requirements, which cover the entire lifecycle of the AI system, including the quality of training data, technical documentation, transparency for users, human oversight mechanisms, and a robust risk management system.

Correct answer: Model Registry

The Model Registry is the central component for ensuring integrity and reproducibility. It should version and store not just the trained model artifact, but also the code used for training, the version of the data it was trained on, its evaluation metrics, and its dependencies. This allows an auditor or data scientist to precisely recreate the model and its training environment, which is the essence of reproducibility.

Correct answer: Implement a formal change management process that includes mandatory testing in a staging environment and stakeholder approval before deploying updates to production.

The core issue is the uncontrolled and untested changes being pushed directly to production. This creates significant operational risk, as evidenced by the system's periodic failures. Implementing a formal change management process with a pre-production testing (staging) environment is the most critical control to ensure that updates are vetted for performance and stability before they can impact live operations and cause financial loss.

Correct answer: Substantive testing.

Substantive testing is designed to detect material misstatements in financial data. By independently recalculating the forecast (an analytical procedure), the auditor is gathering evidence about the validity and accuracy of the financial numbers produced by the AI model. This is a classic example of a substantive test.