AI Control Library / CF-08: Testing & Robustness
Designation: Supplemental | Control types represented: Operational and Technical
Control requirement
Deploy AI applications through controlled, authorized processes that preserve the approved artifact and security configuration, protect secrets and verify operating safeguards before consequential use.
Why this matters
A tested candidate may become unsafe through different permissions, debug settings or dependencies at deployment. Release must preserve what was assessed and verify the live boundary.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
AI application releases, rollouts, infrastructure changes and recovery deployments, including automated delivery pipelines.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The release owner is accountable; platform and security teams implement protected deployment and verification.
Implementation guidance
- Bind approved code, model, prompts, dependencies and configuration to a controlled release manifest.
- Restrict build and deployment authority, protect secrets and verify artifact integrity through the delivery path.
- Use staged rollout or equivalent proportionate verification, checking identities, retrieval, tools, logs and failure behavior.
- Monitor initial operation and define rollback or safe suspension, recording actual deployed versions and findings.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Compare tested and deployed candidates | Material artifacts and security settings match authorization. |
| Attempt an unauthorized substitution | Delivery controls detect or prevent it. |
| Exercise failed rollout checks | The service holds, restricts or restores through the approved path. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Release manifest and approval
- Pipeline and secret controls
- Deployment verification
- Rollout, monitoring and restoration records
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate mismatched artifacts or missing live safeguards to release and security owners. Halt expansion and contain the affected deployment while restoring approved conditions.
Example in practice
Hypothetical example: A model passes evaluation in a read-only environment. Deployment checks reject a production identity with write privileges that were never included in the approved release.
Related controls
- CF01-011: AI Change Authorization
- CF01-013: AI System Rollback & Restoration
- CF08-S02: AI Application Security Baseline Requirements
Framework mappings
- CSA AICM (AIS-06)
- NIST AI RMF (MEASURE 2.5)
- ISO 42001 (Cl. 8.2)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Version 1.0