AI Control Library / CF-08: Testing & Robustness
Designation: Core | Control types represented: Operational and Technical
Control requirement
Conduct authorized penetration testing of AI deployments against relevant security boundaries and misuse paths, using defined scope, safe conditions, evidence handling and verified remediation.
Why this matters
A prompt-only test may miss conventional access flaws or downstream actions. Penetration testing should challenge the complete deployed boundary without creating uncontrolled exposure.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
AI applications, APIs, retrieval integrations, identities, tools and supporting infrastructure, with explicit permission for any supplier-hosted target.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The security testing owner establishes authorization; qualified testers perform work and system owners remediate findings.
Implementation guidance
- Agree targets, identities, allowed techniques, data handling, stop conditions and supplier permission before testing.
- Derive cases from the threat model, including authorization, isolation, prompt injection and tool-execution boundaries.
- Use safe fixtures and controlled accounts, capturing reproducible evidence and the tested configuration.
- Prioritize findings by actual exposure, apply remediation and retest relevant paths before closing.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Inspect test authorization | Targets and techniques remain within the agreed boundary. |
| Review a claimed exploit | Evidence establishes the observed security effect without overstating capability. |
| Trace a serious finding | Containment and retest support the final disposition. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Rules of engagement
- Test scope and version
- Protected findings and reproductions
- Remediation and retest records
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Stop testing on unintended exposure or unsafe effects and notify the authorized contacts. Contain discovered material weaknesses through the approved response process.
Example in practice
Hypothetical example: A tester demonstrates cross-account retrieval using synthetic records. The finding establishes unauthorized access without extracting real customer files to prove impact.
Related controls
- CF08-002: AI Threat Analysis & Modelling
- CF08-007: AI Application Security Testing
- CF08-012: AI Vulnerability Remediation
Framework mappings
- CSA AICM (TVM-06)
- NIST AI RMF (MEASURE 2.5)
- EU AI Act (Art. 9(6))
- ISO 42001 (Cl. 9.2)
- MITRE ATLAS
- OWASP LLM Top 10
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0