AI Control Library / CF-09: Monitoring & Drift Detection
Designation: Core | Control types represented: Operational and Technical
Control requirement
Define and report AI monitoring metrics with clear purpose, scope, source quality, uncertainty and action ownership. Make material risks, coverage gaps and response outcomes visible to the people responsible for operating and accepting the service.
Why this matters
A dashboard can hide risk through aggregation or changing denominators. Reporting needs to distinguish measured health, unobserved outcomes and unresolved findings instead of presenting every green indicator as assurance.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Operational, quality, drift, fairness and security reporting for AI deployments and relevant oversight audiences.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The monitoring program owner coordinates metric owners, operations and business risk authorities.
Implementation guidance
- Define the decision supported by each measure, including data source, denominator, review period and limitations.
- Validate calculations and collection quality, preserving context for model versions, populations and scope changes.
- Set thresholds, review recipients and escalation, separating operational alerts from periodic risk reporting.
- Review decisions and corrective outcomes, revising metrics that create noise, false confidence or incentives to suppress findings.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Reproduce a report measure | Sources and calculation support the stated scope. |
| Change monitored coverage | The report exposes the change rather than claiming unqualified improvement. |
| Inspect a material threshold breach | The responsible authority records investigation or action. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Metric dictionary and ownership
- Calculation and source checks
- Contextualized reports
- Review, escalation and outcome records
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate missing or misleading material reporting to monitoring and risk owners. Qualify claims and restore reliable observation before using the report to authorize expansion.
Example in practice
Hypothetical example: A dashboard reports a lower error rate after hard cases are routed out. Reporting includes the changed population and routed workload so the number is not mistaken for model improvement.
Related controls
- CF01-018: Post-Deployment Monitoring Plan
- CF08-S03: AI Application Security Metrics
- CF04-016: AI Governance Evidence Management
Framework mappings
- CSA AICM (TVM-09, TVM-10)
- NIST AI RMF (MANAGE 4.1)
- EU AI Act (Art. 72)
- ISO 42001 (Cl. 9.1)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Monitoring & Drift Detection
Version 1.0