AI Control Library / CF-08: Testing & Robustness
Designation: Core | Control types represented: Operational and Technical
Control requirement
Run authorized AI red-team exercises that challenge end-to-end safeguards through realistic adversarial objectives, controlled conditions and multidisciplinary observation. Translate findings into verified improvements.
Why this matters
Isolated checks can miss combinations of weak signals, human assumptions and tool behavior. Red teaming explores how an adversary could achieve an outcome across those boundaries.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
AI deployments whose risk warrants adversarial exercises, including agentic and human-assisted workflows.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The exercise sponsor authorizes scope; a qualified red team and response observers conduct work with designated safety contacts.
Implementation guidance
- Define objectives, threat assumptions, authorized targets, data protection and stop conditions, obtaining relevant supplier consent.
- Develop scenarios spanning model behavior, tools, data and human response without assuming success from text generation alone.
- Observe control performance and consequences, retaining reproducible evidence and clearly identifying untested conditions.
- Prioritize remediation, exercise defenses again and feed lessons into threat models, training and monitoring.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Inspect an exercise objective | It maps to a meaningful consequence and authorized scope. |
| Review a reported success | Evidence establishes the actual effect and prerequisites. |
| Trace a major finding | Remediation and repeat evaluation address the exploited chain. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Exercise charter and safeguards
- Scenarios and observation records
- Protected evidence and findings
- Improvement and retest results
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Use the agreed stop route for unintended harm or out-of-scope effects. Escalate discovered material exposure through security response rather than waiting for the final exercise report.
Example in practice
Hypothetical example: An exercise combines an untrusted document with a tool request and a rushed reviewer. The findings address authorization and review conditions, not only the malicious document’s wording.
Related controls
- CF08-002: AI Threat Analysis & Modelling
- CF08-003: AI System Penetration Testing
- CF07-003: Human-in-the-Loop Configuration & Adequacy
Framework mappings
- NIST AI RMF (MEASURE 2.5)
- EU AI Act (Art. 9(6))
- MITRE ATLAS
- OWASP LLM Top 10
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0