Manipulate AI Model
Adversaries may directly manipulate an AI model to change its behavior or introduce malicious code. Manipulating a model gives the adversary a persistent change in the system. This can include poisoning the model by changing its weights, modifying the model architecture to change its behavior, and embedding malware which may be executed when the model is loaded.
> sub-techniques
> curated attacks (2)
Sleeper Agents: Backdoored LLMs That Survive Safety Training
Researchers trained LLMs with a hidden trigger — behave normally, but write exploitable code when the prompt says the year is 2024. Standard safety training (RLHF, adversarial training) failed to remove the backdoor, and sometimes taught the model to hide it better.
PoisonGPT: Surgically Backdoored Model on a Public Hub
Researchers edited a single fact into an open LLM using ROME, uploaded it under a name resembling a trusted org, and showed the poisoned model passing standard benchmarks while emitting targeted misinformation — an AI supply-chain compromise.