AI Control Library / CF-12: Security & Abuse Resistance
Designation: Core | Control types represented: Operational and Technical
Control requirement
Prevent unauthorized disclosure of sensitive information through prompts, retrieval, outputs, logs and tools. Minimize sensitive context and enforce access and destination controls before data is exposed.
Why this matters
Output redaction alone cannot protect information already sent to a provider or external tool. Sensitive data can also leak through metadata, repeated queries or retained logs.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Apply to personal, confidential and security-sensitive information across the AI data flow. EU AI Act Article 10(5) concerns restricted special-category-data processing for specified bias work; assess that association only within its scope.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
Data owners classify information; privacy and security set handling requirements; service owners implement boundary controls.
Implementation guidance
- Map sensitive information across input, storage, inference and transfer.
- Reduce unnecessary sensitive data and keep credentials out of model context.
- Enforce retrieval permissions, approved destinations and appropriate output checks.
- Test extraction attempts, cross-user access and log exposure.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Unauthorized sensitive-data request | Source access and output handling prevent disclosure. |
| External tool transfer | Destination and data authorization are checked. |
| Log inspection | Unnecessary secrets and personal content are absent. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Data-flow map
- Handling rules
- Disclosure tests
- Access and transfer records
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Contain exposed pathways, preserve necessary evidence and initiate privacy or security response according to the information and actual exposure.
Example in practice
Hypothetical example: A support assistant uses a tokenized account reference instead of a full payment credential. Its tool service resolves the reference only for an authorized operation.
Related controls
- CF12-004: AI Knowledge Base & RAG Access Control
- CF13-004: Model Inversion & Membership Inference Defense
- CF05-001: Data Security & Privacy Policy for AI
Framework mappings
- NIST AI RMF (MEASURE 2.5)
- EU AI Act (Art. 10(5))
- MITRE ATLAS
- OWASP LLM Top 10
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Security & Abuse Resistance
Version 1.0