AI Control Library / CF-08: Testing & Robustness
Designation: Core | Control types represented: Operational and Technical
Control requirement
Analyze relevant adversarial attacks by attacker goals, access, lifecycle stage and observed effects, evaluating defenses and limits against the deployment’s actual threat model.
Why this matters
A published attack does not automatically apply to every model or integration. Analysis must distinguish a plausible path, a controlled observation and a demonstrated consequential failure.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
AI models and applications exposed to relevant poisoning, evasion, extraction, prompt manipulation or other adversarial behavior.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The AI security owner leads with model, data and application specialists.
Implementation guidance
- Select attack classes relevant to assets and attacker capabilities, recording necessary assumptions and prerequisites.
- Design controlled evaluation that separates attacker input from observed model, application and downstream effects.
- Compare defenses under realistic variations, including repeated attempts where appropriate, without treating a single refusal as complete protection.
- Document coverage and residual paths, feeding findings into hardening, detection and release decisions.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Inspect attack applicability | The attacker access and prerequisites match the assessed scenario. |
| Review an observed success | Evidence distinguishes generated text from actual unauthorized consequence. |
| Evaluate a defense | Relevant variations and residual limits are recorded. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Attack applicability analysis
- Controlled test cases and versions
- Observed effects and defense results
- Treatment decisions
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate demonstrated material exposure to security and system owners. Contain the relevant path and avoid broad assurance claims from narrowly scoped attack tests.
Example in practice
Hypothetical example: A prompt elicits a suggestion to reveal a file, but retrieval authorization blocks access. Analysis records a model-policy failure separately from the absence of demonstrated data disclosure.
Related controls
- CF08-002: AI Threat Analysis & Modelling
- CF08-005: AI Model Hardening
- CF13-007: Runtime Adversarial Input Detection
Framework mappings
- CSA AICM (MDS-06)
- NIST AI RMF (MEASURE 2.5)
- EU AI Act (Art. 9(6))
- ISO 42001 (Cl. 6.1)
- MITRE ATLAS
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0