AI Control Library / CF-02: Fairness, Bias & Rights Impact
Designation: Core | Control types represented: Operational and Technical
Control requirement
Define, justify and monitor fairness measures appropriate to the decision context, affected groups and harm scenarios. Record denominators, sampling, uncertainty, thresholds, tradeoffs and the response required when results fall outside accepted conditions.
Why this matters
No single number establishes fairness for every use. A favorable overall score can hide different error burdens, and an apparently equal rate can be misleading when samples or outcome labels are incomplete.
How the control types work together
Operational controls define the objectives, review duties and response decisions. Technical controls implement the data checks, measurement or system changes needed to test and enforce those decisions.
When it applies
AI assessment and operational evaluation where measurable differences can inform decisions; combine quantitative results with contextual or qualitative evidence when measurement is incomplete.
Define the assessment boundary and potential harm before selecting methods. Record evidence gaps and a reasoned response; unavailable demographic data does not establish that the system is fair.
Ownership
The assessment lead develops measures with domain and statistical expertise; the business risk owner approves their decision use and limits.
Implementation guidance
- Define the harm being measured and relevant comparison populations; choose measures such as error, access or outcome differences for stated reasons.
- Specify datasets, denominators, uncertainty estimation and minimum evidence needed to interpret results; treat insufficient samples as uncertainty rather than a pass.
- Document tradeoffs between fairness measures and task objectives, approving thresholds with context rather than importing a universal numerical rule.
- Implement reproducible calculation and reporting, route material results to an owner and revisit the metric design when the use or population changes.
How to verify it
Use an authorized evaluation or a representative sample of actual decisions and records. Establish the relevant population, evidence limits and acceptance criteria before interpreting a result.
| Test | Expected result |
|---|---|
| Recompute a reported metric | Dataset, denominator, version and method reproduce the result. |
| Inspect a small-group result | Uncertainty and privacy limits prevent a misleading confident conclusion. |
| Trigger a defined threshold | A documented response occurs rather than passive reporting. |
Record a finding when required evidence or action is absent. Treat insufficient evidence as an unresolved limitation rather than a favorable result. A statistically observed difference requires contextual investigation and does not by itself establish its cause or legal meaning.
Evidence to retain
- Metric specification and rationale
- Calculation and uncertainty records
- Approved thresholds and tradeoffs
- Monitoring and response history
Protect demographic and other sensitive information with appropriate access, minimization and retention. Use privacy-preserving reporting where suitable, and prevent published small-group results from exposing individuals.
If the control fails
Escalate material disparities, conflicting objectives or insufficient evidence to the risk owner and assessment authority. Restrict fairness claims and consequential expansion until the limits are understood and treated.
Example in practice
Hypothetical example: A triage model meets its overall accuracy goal but misses more urgent cases in a relevant subgroup. The team tracks that harmful error type explicitly and avoids declaring success based on accuracy or outcome parity alone.
Related controls
- CF02-003: Demographic Impact Assessment
- CF02-008: Bias Remediation Process
- CF02-012: Equity & Fairness Drift Monitoring
Framework mappings
- NIST AI RMF (MEASURE 2.11)
- EU AI Act (Art. 10(2)(f), Art. 15)
- ISO 42001 (Cl. 9.1, A5.4)
Review and maintenance
Review at the interval approved by the control owner and after material changes to purpose, population, data, decision rules or human workflow. Revisit relevant complaints, incidents and changed external obligations, and verify corrective action before closing findings.
Back to Fairness, Bias & Rights Impact
Version 1.0