AI Control Library / CF-09: Monitoring & Drift Detection
Designation: Core | Control types represented: Operational and Technical
Control requirement
Monitor deployed AI performance against approved task and service criteria using appropriate outcome evidence, representative samples and version context. Investigate deterioration and uncertainty before continuing unsupported reliance.
Why this matters
Low latency and successful requests do not establish that answers or predictions remain correct. Performance monitoring must reflect the task and the consequences of different errors.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Deployed predictive, generative and agentic services with measurable task or service performance requirements.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The system owner is accountable; domain and monitoring teams assess results and response.
Implementation guidance
- Derive operational measures from release criteria, including consequential error types, latency and capacity where relevant.
- Collect appropriate outcomes or qualified review samples, recording delay, selection effects and missing ground truth.
- Compare with approved baselines across relevant workloads and versions, investigating meaningful changes and uncertainty.
- Apply defined response and re-evaluation, verifying improvement after correction rather than silently resetting the baseline.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Review a healthy-uptime period | Task-quality evidence is considered separately from infrastructure status. |
| Inspect delayed labels | Reports expose unobserved outcomes rather than count them as successes. |
| Trigger a performance decline | An owner investigates and records an appropriate operating decision. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Performance measures and baselines
- Outcome and sample records
- Versioned trend analysis
- Investigation and response decisions
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate material degradation or inadequate evidence to the business owner. Restrict affected tasks, increase review or suspend use according to approved risk conditions.
Example in practice
Hypothetical example: A classification service returns responses normally but begins missing important cases. Outcome sampling detects the error increase even though the availability dashboard remains green.
Related controls
- CF08-010: AI Reliability & Performance Benchmarking
- CF09-004: Concept Drift & Model Decay Detection
- CF09-007: AI Output Quality Monitoring
Framework mappings
- CSA AICM (MDS-10)
- NIST AI RMF (MANAGE 4.1)
- EU AI Act (Art. 72)
- ISO 42001 (Cl. 9.1)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Monitoring & Drift Detection
Version 1.0