AI Control Library / CF-08: Testing & Robustness
Designation: Core | Control types represented: Operational and Technical
Control requirement
Test AI application security across interfaces, identities, data stores, tools and deployment configuration, combining conventional application testing with AI-specific misuse cases and regression coverage.
Why this matters
An AI system is still an application with access controls and dependencies. Good model behavior cannot correct broken tenant isolation or an insecure export endpoint.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
AI web, mobile, API and background applications, including retrieval and external tool integrations.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The application security owner defines tests; development teams integrate checks and system owners resolve risk.
Implementation guidance
- Map application entry points, authorization paths and data flows, linking tests to actual security requirements.
- Test authentication, authorization, input handling, output consumption, tenant isolation and relevant AI interaction paths.
- Integrate suitable automated checks and qualified manual testing, using controlled accounts and safe data.
- Track findings through remediation and regression tests, revisiting coverage after architecture or permission changes.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Exercise cross-user access | The application independently enforces the intended boundary. |
| Use unsafe generated output in a controlled downstream fixture | Consumers validate or constrain it appropriately. |
| Change a connector or endpoint | Relevant security regression tests run before release. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Application test plan and baseline
- Automated and manual results
- Finding and remediation records
- Regression and release decisions
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate exploitable access or execution flaws to security and release owners. Restrict exposed functionality until verified correction or approved containment is in place.
Example in practice
Hypothetical example: A model refuses to disclose another tenant’s data, but a direct export API allows access. Application testing finds the API flaw and does not credit the refusal as tenant isolation.
Related controls
- CF08-003: AI System Penetration Testing
- CF12-007: AI API Security
- CF12-011: AI Output Handling Security
Framework mappings
- CSA AICM (AIS-05)
- NIST AI RMF (MEASURE 2.5)
- EU AI Act (Art. 9(6))
- ISO 42001 (Cl. 8.2)
- MITRE ATLAS
- OWASP LLM Top 10
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0