AI systems depend on many external components. Each is a potential route for attack.
Components at Risk
- Pre-trained models downloaded from hubs.
- Datasets used for training, fine-tuning and evaluation.
- Libraries and frameworks for ML and AI applications.
- Plugins, tools and MCP servers connected to assistants.
- Model APIs from providers.
Specific Threats
- Model files that execute code when loaded, through unsafe serialisation formats.
- Backdoored or poisoned models and datasets.
- Typosquatted packages with names similar to popular libraries.
- Compromised maintainers or accounts.
- Malicious tool integrations.
Controls
- Use trusted sources and verified publishers.
- Prefer safe model formats such as safetensors over formats that allow code execution.
- Pin versions and verify hashes.
- Scan models and packages.
- Maintain an inventory of models, datasets and dependencies — sometimes called an AI bill of materials.
- Review licences and terms.
- Test third-party components before production use.
Ongoing
Monitor for vulnerabilities and advisories affecting your components, and update deliberately.