AI Control Library / CF-09: Monitoring & Drift Detection
Designation: Core | Control types represented: Operational and Technical
Control requirement
Detect relevant changes in AI input data compared with approved operating references, including content, distribution, missingness and schema. Investigate whether detected changes affect model suitability or control assumptions.
Why this matters
A change in inputs is a signal for investigation, not proof of harmful performance or a reason to retrain automatically. Important shifts may involve relationships between fields that single-field checks miss.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Deployed AI input streams, retrieval context and upstream data transformations within the monitored workflow.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The data and system owners define reference conditions; monitoring teams detect and route changes.
Implementation guidance
- Define reference populations, periods and valid schema, recording seasonality and known operating variation.
- Select risk-relevant checks for distribution, missingness, categories, text or context changes and feature relationships where appropriate.
- Set justified observation windows and alert criteria, recording sample size, uncertainty and coverage limits.
- Investigate source and consequence, deciding on data correction, evaluation, restrictions or accepted variation through controlled review.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Introduce a relevant synthetic input shift | The selected method detects it or records its known limitation. |
| Exercise ordinary seasonal variation | Alerts are interpreted with context rather than assumed harmful. |
| Trace a drift alert | Source and performance implications are assessed before model changes. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Input references and coverage
- Detection methods and thresholds
- Shift and false-positive tests
- Investigation and treatment records
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate material out-of-scope data or unresolved impact to data and system owners. Restrict affected processing or increase review while suitability is assessed.
Example in practice
Hypothetical example: A service receives a new document layout. Drift detection identifies the change, and targeted evaluation determines whether extraction still works before retraining is proposed.
Related controls
- CF05-002: AI Data Inventory
- CF09-004: Concept Drift & Model Decay Detection
- CF09-012: Monitoring Re-calibration on System Change
Framework mappings
- NIST AI RMF (MANAGE 4.1)
- EU AI Act (Art. 72)
- ISO 42001 (Cl. 9.1)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Monitoring & Drift Detection
Version 1.0