AI supply chain security secures everything you didn't build: downloaded models, third-party datasets, ML libraries and serving tooling. Key risks: data poisoning (manipulated training data embedding errors or backdoors), model tampering & malicious model files (some formats execute code on load - insecure deserialization), compromised ML dependencies (ordinary software-supply-chain risk), and provenance gaps. Testing = inventory the AI dependencies, verify model/dataset provenance & integrity, check how models are loaded (can the format run code?), review ML dependencies, and assess poisoning exposure in data pipelines. It sits alongside app-layer AI testing (prompt injection, agent security, OWASP LLM Top 10). Details below.
// 01 The AI stack is assembled, not built
Almost no one trains a frontier model from scratch. Modern AI systems are assembled: a pre-trained model pulled from a public hub, fine-tuned on a mix of your data and third-party datasets, wired together with open-source ML libraries, and served through more third-party tooling. That's efficient - and it means the majority of your AI system is code and data you didn't write and can't fully see. Each imported component is a trust decision and a potential point of compromise. The security world learned this lesson the hard way with software dependencies; AI adds the same supply-chain risk plus new AI-specific ones around models and data. If your AI security effort focuses only on prompt injection at the app layer, you're guarding the front door while leaving the loading dock open.
// 02 The risk map
Data poisoning
Manipulated training or fine-tuning data that degrades the model or embeds a hidden backdoor trigger.
Model tampering
A downloaded model altered to behave maliciously, or whose weights hide a backdoor.
Malicious model files
Some model formats execute code when loaded - loading an untrusted model can equal running untrusted code.
Compromised dependencies
The ML libraries and frameworks carry the same software-supply-chain risk as any codebase.
Underlying all four is a provenance gap: if you can't verify where a model or dataset came from and that it hasn't been altered, you can't trust it - and in AI, “trust” means letting it shape decisions or run in your environment.
// 03 The under-appreciated one: models that run code
The most concrete and least-discussed AI supply chain risk is brutally simple: loading a model can execute code. Some model serialization formats can run arbitrary code when the model is loaded - so downloading an untrusted model file and loading it is equivalent to running untrusted code. It's insecure deserialization wearing an AI hat, and it turns “let's try this model from a public hub” into a potential remote-code-execution event on your infrastructure. The defences are practical: prefer safer, load-only formats for untrusted models; obtain models from trusted sources with verified integrity (hashes, signatures); and load third-party models in an isolated, low-privilege environment so a malicious file is contained. This single risk justifies treating model files with the same suspicion as any executable download.
// 04 How to test the AI supply chain
Testing blends classic software-supply-chain review with AI-specific checks. The workflow: (1) inventory the models, datasets, libraries and services the system depends on - you can't secure what you haven't listed; (2) verify provenance and integrity of models and datasets (trusted source, checksum/signature); (3) assess the loading path - can the model format or loading process execute code, and is loading isolated?; (4) review ML dependencies for known vulnerabilities and version pinning, exactly as in CI/CD supply-chain testing; and (5) evaluate poisoning exposure in any pipeline ingesting external or user-supplied data. This runs alongside testing the AI application itself - prompt injection, excessive agency in agents, RAG security and the broader OWASP LLM Top 10 - to cover the full AI attack surface.
// 05 How to defend it
The defensive playbook mirrors mature software supply-chain security, adapted for AI. Source control: obtain models and datasets only from trusted, verified sources, and pin and verify them. Integrity: record and check hashes/signatures; maintain an AI bill of materials so you know every model, dataset and library in play. Isolation: load and run third-party models in sandboxed, least-privilege environments. Format hygiene: prefer safe serialization formats. Pipeline defence: validate and monitor data feeding training/fine-tuning to limit poisoning, and treat user-supplied data as untrusted. Dependency management: patch ML libraries like any other. Combined, these turn a sprawling, opaque AI supply chain into a governed one - the same discipline we apply to CI/CD and cloud, extended to models and data. Engagements follow our methodology.
// 06 Frequently asked questions
What is AI supply chain security?
Securing everything that goes into an AI system you didn't build: pre-trained models from public hubs, third-party datasets, ML libraries and frameworks, and serving tooling. Because modern AI is assembled from many external components, each is a potential point of compromise - a poisoned dataset, tampered model file or malicious dependency can introduce hidden behaviour or a straightforward software vulnerability.
What are the main AI supply chain risks?
Data poisoning (manipulated training/fine-tuning data causing errors or a backdoor), model tampering and malicious model files (an altered model, or a format abused to execute code on load), compromised ML libraries and dependencies (ordinary software-supply-chain risk), and provenance gaps (you can't verify where a model or dataset came from). Some, like unsafe model deserialization, are straightforward RCE risks.
Can loading an AI model execute malicious code?
Yes, depending on the format. Some model serialization formats execute arbitrary code when loaded, so loading an untrusted model file can equal running untrusted code - a form of insecure deserialization. Prefer safer load-only formats for untrusted models, obtain models from trusted sources with verified integrity, and load third-party models in an isolated, low-privilege environment.
How do you test AI supply chain security?
Combine software-supply-chain review with AI-specific checks: inventory the models, datasets, libraries and services; verify provenance and integrity; assess how models are loaded and whether the format can execute code; review ML dependencies for known vulnerabilities and pinning; and evaluate data-poisoning exposure in pipelines ingesting external data. It sits alongside prompt injection, excessive agency and the OWASP LLM Top 10.
// 07 Related reading
- OWASP LLM Top 10 and AI penetration testing — the app-layer companion.
- Insecure deserialization — the mechanism behind malicious model files.
- CI/CD pipeline security and MCP & AI agent security.