AI Control Library / CF-04: Auditability & Evidence
Designation: Core | Control types represented: Operational and Technical
Control requirement
Detect and report failures of AI logging and monitoring capability, including silent collection loss, delayed delivery, dropped records, disabled rules and failed alert routing. Define the operating restrictions required when material visibility is lost.
Why this matters
A quiet dashboard can mean that nothing is wrong or that the monitoring pipeline has stopped. The organization needs independent evidence that required observation and notification paths remain functional.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Event emitters, collectors, queues, storage, analysis rules, dashboards and notification channels supporting AI oversight.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The monitoring service owner maintains health detection; security and AI system owners decide whether affected operation can continue.
Implementation guidance
- Map required monitoring paths and their health signals, defining expected cadence, delivery delay, capacity and acceptable evidence gaps.
- Use independent health checks, reconciliation or controlled synthetic events where appropriate to detect failures that sources cannot self-report.
- Alert through a route that does not rely solely on the failed component, identifying affected systems and uncertainty about missing activity.
- Apply approved degraded-operation rules, restore coverage and reconcile buffered or missing records without assuming delayed evidence was available in real time.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Disable a required collector in a controlled exercise | Health monitoring reports the affected scope within the approved tolerance. |
| Interrupt the normal alert channel | An independent escalation route reaches the responsible team. |
| Restore delivery after a backlog | Completeness, timing and unresolved gaps are assessed before normal assurance resumes. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Monitoring dependency and health map
- Failure thresholds and routing
- Failure-injection test results
- Coverage restoration and gap analysis
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Notify monitoring, security and system owners when required visibility fails. Restrict or suspend consequential activity where accepted conditions depend on unavailable observation, preserving recoverable records during restoration.
Example in practice
Hypothetical example: A collector stops forwarding tool events while the model service remains healthy. A separate end-to-end test event detects the gap and places the action-taking workflow into its approved restricted mode.
Related controls
- CF04-007: AI Logging & Monitoring Policy
- CF04-012: AI Audit Log Monitoring & Anomaly Response
- CF01-018: Post-Deployment Monitoring Plan
Framework mappings
- CSA AICM (LOG-13)
- NIST AI RMF (MANAGE 4.1)
- ISO 42001 (Cl. 9.1)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Auditability & Evidence
Version 1.0