CF10-008: AI Incident Metrics & Reporting

AI Control Library / CF-10: Recourse & Incident Response

Designation: Core | Control types represented: Operational and Technical

Control requirement

Measure and report AI incidents using defined categories, populations, timelines, impact and response outcomes. Preserve uncertainty and changes in detection coverage so reporting supports accountable improvement.

Why this matters

A falling incident count may reflect fewer reports rather than safer operation. Speed metrics can also encourage premature closure unless paired with effectiveness and harm-related evidence.

How the control types work together

Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.

When it applies

AI incident portfolios, operational response reporting and executive risk review.

Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.

Ownership

The incident metrics owner coordinates response, business and assurance functions.

Implementation guidance

  1. Define measures such as detection, triage, verified containment, recovery, recurrence and affected scope, with clear event definitions.
  2. Collect attributable timelines and outcomes, distinguishing proposed, blocked and executed activity and recording uncertainty.
  3. Explain changes in coverage, reporting behavior, severity and denominators before comparing periods.
  4. Review findings with accountable owners and verify that actions address recurring harm or response gaps.

How to verify it

Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.

Test Expected result
Reproduce a response metric Defined events and records support the result.
Inspect an apparent improvement Coverage and classification changes are considered.
Trace a recurring pattern A responsible owner initiates and verifies improvement.

Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.

Evidence to retain

  • Metric definitions and scope
  • Incident timelines and outcomes
  • Contextualized reports
  • Review and corrective-action records

Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.

If the control fails

Escalate misleading reporting or material unaddressed patterns to incident and business authorities. Correct the report and treatment before relying on it for risk decisions.

Example in practice

Hypothetical example: The team records containment when tool authority is verified revoked, not when a stop message is sent. The measure reveals a previously hidden delay in stopping child workers.

Related controls

Framework mappings

  • CSA AICM (SEF-05)
  • NIST AI RMF (MANAGE 3.1)
  • EU AI Act (Art. 72)
  • ISO 42001 (Cl. 9.1)

Review and maintenance

Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.

Back to Recourse & Incident Response

Version 1.0

Scroll to Top