AI Control Library / CF-13: Model & Adversarial Security
Designation: Core | Control types represented: Operational and Technical
Control requirement
Assess model and data supply paths for backdoors that cause targeted behavior under particular triggers. Combine provenance, controlled training and targeted behavioral evaluation before deployment and after suspect changes.
Why this matters
A backdoored model may perform well on ordinary benchmarks while failing only on selected inputs. A clean aggregate score or intact file hash cannot establish the absence of malicious behavior.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Apply to third-party models, adapters, fine-tunes and data contributions, especially when their development history is incomplete.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
The model owner approves candidate artifacts; security designs trigger-focused tests; supply-chain owners investigate provenance gaps.
Implementation guidance
- Collect development provenance and verify approved artifact sources.
- Review training changes and unusual targeted data patterns.
- Test plausible triggers, target behaviors and clean-task performance.
- Quarantine unexplained behavior and compare against trusted baselines.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Targeted trigger suite | Unexpected conditional behavior is investigated. |
| Unknown adapter provenance | Release requires explicit assessment. |
| Ordinary benchmark passes | Targeted security tests are still required. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Artifact provenance
- Trigger test cases
- Baseline comparisons
- Release decisions
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Withdraw suspect artifacts, identify affected deployments and rebuild or replace from trusted inputs before retesting.
Example in practice
Hypothetical example: A downloaded adapter behaves normally except when a particular phrase appears. The team quarantines it and checks whether any deployed workflows used that adapter.
Related controls
- CF13-003: Data Poisoning Attack Prevention & Detection
- CF13-011: Model Supply Chain Tampering Defense
- CF08-001: AI Testing & Robustness Policy
Framework mappings
- NIST AI RMF (MEASURE 2.5)
- MITRE ATLAS
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Model & Adversarial Security
Version 1.0