AI Control Library / CF-09: Monitoring & Drift Detection
Designation: Core | Control types represented: Operational and Technical
Control requirement
Operate the collection, scheduling, alerting and response capability needed to detect relevant fairness and bias changes in deployed AI. Preserve the approved metric definitions and route limitations or adverse signals to qualified assessment owners.
Why this matters
A sound fairness methodology does not help if its data pipeline silently drops a group or alerts have no owner. This control addresses the operational mechanism that keeps fairness surveillance functioning.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Deployed AI with material people-related effects, including workflows with delayed labels and privacy-constrained group information.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The monitoring owner operates the capability with fairness assessment, privacy and business decision owners.
Implementation guidance
- Implement approved fairness metrics, baselines, group scope and review windows from the assessment program.
- Monitor data freshness, coverage, missingness and calculation quality, protecting sensitive groups and small samples.
- Route material changes and evidence gaps to qualified reviewers, retaining workflow and version context.
- Exercise pipeline failure, representative disparity and response scenarios, verifying corrective action and continued observation.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Interrupt a relevant data feed | The system reports lost fairness visibility instead of unchanged favorable results. |
| Introduce a controlled disparity signal | The approved calculation and review route identify it. |
| Inspect response closure | Assessment and business owners verify treatment rather than merely clearing the alert. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Fairness monitoring configuration
- Data and calculation health
- Alert and response tests
- Assessment and treatment handoffs
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Escalate material disparities or failed monitoring to fairness and business owners. Restrict unsupported claims or affected use where necessary while evidence and safeguards are restored.
Example in practice
Hypothetical example: A new intake channel omits group data used by the approved assessment. Pipeline monitoring identifies the coverage gap before dashboards report apparently stable fairness from a shrinking subset.
Related controls
- CF02-007: Bias Metrics Definition & Monitoring
- CF02-012: Equity & Fairness Drift Monitoring
- CF04-013: AI Monitoring System Failure Reporting
Framework mappings
- NIST AI RMF (MANAGE 4.1)
- EU AI Act (Art. 72)
- ISO 42001 (Cl. 9.1)
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Monitoring & Drift Detection
Version 1.0