AI Control Library / CF-08: Testing & Robustness
Designation: Supplemental | Control types represented: Operational and Technical
Control requirement
Define and verify a risk-appropriate security baseline for AI applications, specifying minimum configurations, access boundaries, dependency controls, logging and deployment protections. Record approved deviations and the exact assessed configuration.
Why this matters
A generic checklist can miss the model, retrieval and tool settings that determine exposure. A baseline needs verifiable requirements tied to the service rather than a product label.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
AI application environments, deployment templates and material configuration changes, including production and consequential pilots.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The application security owner defines baseline requirements; engineering implements and verifies them.
Implementation guidance
- Derive minimum controls from architecture, threats, data sensitivity and action authority.
- Specify testable requirements for identities, retrieval, tools, secrets, dependencies, logs and network boundaries.
- Implement through controlled configuration or deployment checks, with reviewed deviations and evidence.
- Compare live state with the approved baseline and reassess after relevant architecture or capability changes.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Inspect a baseline requirement | Its implementation and acceptance evidence are identifiable. |
| Attempt a release missing a required safeguard | The gate blocks or invokes the approved exception path. |
| Review live configuration | Material deviations are detected and treated. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Application security baseline
- Configuration and enforcement
- Deviation approvals
- Release and reconciliation tests
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate material baseline deviation to security and release owners. Hold new exposure or restore an approved safe configuration while the gap is resolved.
Example in practice
Hypothetical example: A deployment template requires scoped tool identities and disabled debug content logging. A production-bound configuration fails its gate when either setting diverges.
Related controls
- CF01-012: AI System Baseline Management
- CF08-S05: AI Application Secure Deployment
- CF08-007: AI Application Security Testing
Framework mappings
- CSA AICM (AIS-02)
- NIST AI RMF (MEASURE 2.5)
- ISO 42001 (Cl. 8.2)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0