CF03-015: Open-Source & Open-Weight Model Risk Assessment

AI Control Library / CF-03: AI Supply Chain & Third-Party Risk

Designation: Core | Control types represented: Operational and Technical

Control requirement

Assess open-source and open-weight models before consequential use, covering provenance, licensing, dependencies, loading behavior, capabilities, misuse exposure, maintainability and the organization’s ability to operate safeguards without a managed provider.

Why this matters

Access to weights enables control and customization but also transfers more operational responsibility to the deployer. An open-weight label does not establish safe loading, suitable permissions, reliable behavior or unrestricted use rights.

How the control types work together

Operational controls define acceptance, responsibility and response. Technical controls verify artifacts, observe actual use or enforce data and action boundaries. The process must cover the same deployment that the technical mechanisms protect.

When it applies

Downloaded or self-hosted models and derivatives, including community checkpoints, adapters, fine-tunes and quantized versions.

Tailor review depth to information sensitivity, granted authority and business dependence. Record what the organization can test directly, what relies on provider evidence and how material visibility gaps affect the acceptance decision.

Ownership

The model owner is accountable; security, platform, domain and licensing reviewers assess the relevant evidence.

Implementation guidance

  1. Record the exact artifact and derivation, source credibility, license conditions and required dependencies; distinguish weight availability from source and data transparency.
  2. Assess deserialization and repository-code risks, required privileges, network behavior and supported runtime security.
  3. Evaluate capabilities and limitations on the intended use, including tool authority, information exposure and harmful misuse paths.
  4. Define patching, monitoring, resource limits, recovery and replacement responsibilities, approving only the evaluated configuration and permitted purpose.

How to verify it

Use representative deployment records and authorized controlled tests. Establish the assessed service, artifact version and acceptance conditions before interpreting results; do not extend evidence beyond the boundary it covers.

Test Expected result
Review model acquisition Provenance, license and executable dependencies receive explicit assessment.
Evaluate the deployment configuration Permissions and safeguards reflect actual capabilities and intended use.
Simulate loss of upstream support Maintenance, isolation or replacement responsibilities remain actionable.

Record a finding when a required result cannot be demonstrated or actual practice differs materially from the accepted conditions. Assign corrective action and verify the affected boundary before closing the issue.

Evidence to retain

  • Model risk and license assessment
  • Artifact and dependency records
  • Loading and use-case evaluations
  • Operating safeguards and ownership

Protect supplier-confidential information, credentials and personal data in these records. Preserve enough version and scope information to establish which service or artifact the evidence supports.

If the control fails

Escalate unacceptable provenance, loading or use restrictions to the model and risk owners. Isolate evaluation and withhold consequential deployment until the identified conditions are addressed.

Example in practice

Hypothetical example: A team selects an efficient community model but finds that its sample loader retrieves additional code. The assessment replaces that loading path, tests the intended task and assigns maintenance before deployment.

Related controls

Framework mappings

  • CSA AICM (MDS-12)
  • NIST AI RMF (MAP 1.6)
  • ISO 42001 (Cl. 6.1, A9.1)
  • MITRE ATLAS

Review and maintenance

Review at the interval approved by the control owner and after relevant supplier, component, contract, access or deployment changes. Reassess material incidents and new evidence, updating acceptance conditions and verifying corrective actions where required.

Back to AI Supply Chain & Third-Party Risk

Version 1.0

Scroll to Top