Poison Training Data
Adversaries may attempt to poison datasets used by an AI model by modifying the underlying data or its labels. This allows the adversary to embed vulnerabilities in AI models trained on the data that may not be easily detectable. Data poisoning attacks may or may not require modifying the labels. The embedded vulnerability is activated at a later time by data samples with an Insert Backdoor Trigger
Poisoned data can be introduced via AI Supply Chain Compromise or the data may be poisoned after the adversary gains Initial Access to the system.
> curated attacks (4)
LLM Grooming: Flooding Training Data to Seed Disinformation
A pro-Kremlin website network mass-published millions of propaganda articles, seemingly to seep into the data that trains and grounds LLMs. An audit found leading chatbots repeated the network's false claims about a third of the time — a data-integrity attack on the model supply chain.
Web-Scale Data Poisoning: The Split-View Attack
Web-scale datasets distributed as lists of URLs can be poisoned by buying expired domains behind those URLs. What researchers downloaded when the dataset was built differs from what later trainers fetch — cheaply poisoning real training corpora.
Sleeper Agents: Backdoored LLMs That Survive Safety Training
Researchers trained LLMs with a hidden trigger — behave normally, but write exploitable code when the prompt says the year is 2024. Standard safety training (RLHF, adversarial training) failed to remove the backdoor, and sometimes taught the model to hide it better.
Microsoft Tay: Online Poisoning of a Learning Chatbot
A Twitter chatbot that learned from user interactions was flooded with abusive input by a coordinated crowd, corrupting its behavior within hours and forcing a shutdown — an early real-world online data-poisoning incident.