AI Control Library / CF-13: Model & Adversarial Security
Designation: Core | Control types represented: Operational and Technical
Control requirement
Detect and respond to inference API use that indicates extraction, privacy attacks, credential abuse or other prohibited activity. Correlate signals across identities and time without collecting unnecessary sensitive content.
Why this matters
Individual requests can appear legitimate while a coordinated sequence exposes model behavior or sensitive information. Volume alone is insufficient to distinguish abuse from genuine demand.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Apply to internal and external model APIs, batch interfaces and reseller pathways. Include service accounts and tenant-level aggregate use.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The API owner defines acceptable use and limits; security analyzes patterns; customer operations supports proportionate response.
Implementation guidance
- Define abuse scenarios, observables and privacy-conscious logging.
- Correlate identity, query patterns, error signals and resource use.
- Set proportionate throttling, investigation and suspension rules.
- Test distributed abuse and evaluate impact on legitimate high-volume users.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Coordinated probing simulation | Signals support cross-account investigation. |
| Abusive account restricted | Related sessions and tokens are considered. |
| Legitimate traffic burst | Handling follows approved capacity and abuse rules. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Abuse scenarios
- Detection configuration
- Investigation records
- Response decisions
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Constrain suspect access, preserve relevant telemetry and assess extraction or disclosure exposure before restoring privileges.
Example in practice
Hypothetical example: A service links unusual repeated boundary queries across several newly created accounts. Security investigates a possible extraction campaign rather than treating each account in isolation.
Related controls
- CF13-004: Model Inversion & Membership Inference Defense
- CF13-005: Model Extraction & Theft Prevention
- CF12-007: AI API Security
Framework mappings
- NIST AI RMF (MEASURE 2.5)
- MITRE ATLAS
- OWASP LLM Top 10
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Model & Adversarial Security
Version 1.0