AI Control Library / CF-13: Model & Adversarial Security
Designation: Core | Control types represented: Operational and Technical
Control requirement
Prevent, detect and contain malicious manipulation of training, fine-tuning, feedback and retrieval data that could alter model behavior. Track provenance and restrict how untrusted contributions enter trusted datasets.
Why this matters
Poisoning can damage broad performance or selectively target a person, topic or trigger. Automatically accepting user feedback or scraped content creates a path from an external actor into future behavior.
How the control types work together
Operational controls define scope, authority and the actions required when conditions are not met. Technical controls produce evidence or enforce the relevant boundaries. Both must apply to the same deployment and operating conditions.
When it applies
Apply to initial training and continuous updates, including annotation, preference data, external corpora and retrieval indexes used by the application.
Define the assessed boundary, dependencies and evidence limitations. Tailor depth and review frequency to potential impact, and document why a particular safeguard or exclusion is appropriate to the actual deployment.
Ownership
Data owners approve sources; ML teams control ingestion; security investigates suspected manipulation.
Implementation guidance
- Record source provenance, permitted use and contributor access.
- Quarantine new data and check labels, anomalies and targeted concentrations.
- Separate user feedback from automatic model or index promotion.
- Test representative poisoning scenarios and preserve clean rollback datasets.
How to verify it
Use representative records and authorized controlled tests. Identify the relevant version, operating conditions and expected result before testing, and retain enough evidence to support the conclusion.
| Test | Expected result |
|---|---|
| Unauthorized dataset edit | Change is blocked or detected. |
| Suspicious contribution cluster | Data is quarantined for review. |
| Poisoned update simulation | Affected version can be identified and withdrawn. |
Record a finding when required results cannot be demonstrated or practice differs materially from the approved conditions. Missing evidence limits the conclusion; it should not be treated as a successful test.
Evidence to retain
- Data lineage
- Ingestion reviews
- Dataset versions
- Poisoning test results
Protect evidence according to its sensitivity, access needs and retention rules. Avoid unnecessary copies of personal information, secrets or confidential material, while preserving the records needed for authorized review.
If the control fails
Freeze ingestion or promotion, quarantine affected data and assess models and indexes already built from it.
Example in practice
Hypothetical example: A surge of similarly phrased feedback promotes one supplier. Review identifies coordinated submissions before the preference dataset is approved for fine-tuning.
Related controls
- CF13-001: AI Training Pipeline Security
- CF13-006: Backdoor Attack Detection & Prevention
- CF05-001: Data Security & Privacy Policy for AI
Framework mappings
- CSA AICM (DSP-21)
- NIST AI RMF (MEASURE 2.5)
- ISO 42001 (Cl. 8.2)
- MITRE ATLAS
Review and maintenance
Review at the interval approved by the control owner and after relevant changes, incidents or failed assessments. Update affected procedures and configuration, address evidence gaps and verify corrective action before closing findings.
Back to Model & Adversarial Security
Version 1.0