AI Control Library / CF-08: Testing & Robustness
Designation: Core | Control types represented: Operational
Control requirement
Threat-model the complete AI deployment, identifying assets, actors, trust boundaries, attacker capabilities, misuse paths and control assumptions. Use the analysis to prioritize safeguards and tests.
Why this matters
Model-level threats alone omit retrieval permissions, tool authority and administrative access. A useful threat model follows how data and authority cross the real system.
How the control types work together
Operational controls establish the assigned responsibilities, review processes and response decisions described here. Their effectiveness depends on performing the work and using evidence to guide action, rather than approving a document alone.
When it applies
New AI architectures, acquisitions and material changes involving data, models, tools, users or deployment boundaries.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The system owner coordinates security, engineering and domain participants; control owners address the resulting actions.
Implementation guidance
- Map components, data flows, identities and consequential actions, distinguishing trusted configuration from untrusted content.
- Identify plausible attacker goals, access and knowledge, including insiders, suppliers and indirect content sources.
- Analyze abuse paths and assumptions, selecting mitigations and tests proportionate to consequences.
- Version the analysis and reassess after architecture, permission or threat changes, tracking unresolved risks.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Trace a high-impact threat | The model identifies its path, assumptions and treatment. |
| Inspect a tool integration | Authority and returned-content boundaries are included. |
| Introduce a new data source | Relevant threats and evaluation requirements are reconsidered. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Architecture and threat model
- Assumptions and abuse cases
- Treatment and test links
- Review and change history
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate unmitigated material paths to the system and risk owners. Restrict the exposed capability until safeguards or a reviewed alternative support the intended use.
Example in practice
Hypothetical example: A document assistant reads externally supplied files. Threat modeling identifies indirect instructions as untrusted content and adds tests of retrieval and tool boundaries.
Related controls
- CF03-018: Third-Party Plugin & API Risk Management
- CF08-003: AI System Penetration Testing
- CF14-001: Agentic System Security Boundaries
Framework mappings
- CSA AICM (TVM-12)
- NIST AI RMF (MAP 1.5)
- EU AI Act (Art. 9(2))
- ISO 42001 (Cl. 6.1)
- MITRE ATLAS
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0