AML.T0018.000 sub-technique demonstrated

Poison AI Model

Sub-technique of AML.T0018 — Manipulate AI Model

Persistence AI Attack Staging View on ATLAS ↗

Adversaries may manipulate an AI model's weights to change it's behavior or performance, resulting in a poisoned model. Adversaries may poison a model by directly manipulating its weights, training the model on poisoned data, further fine-tuning the model, or otherwise interfering with its training process.

The change in behavior of poisoned models may be limited to targeted categories in predictive AI models, or targeted topics, concepts, or facts in generative AI models, or aim for a general performance degradation.

> curated attacks (1)

Research high 2024-01-12

Sleeper Agents: Backdoored LLMs That Survive Safety Training

Researchers trained LLMs with a hidden trigger — behave normally, but write exploitable code when the prompt says the year is 2024. Standard safety training (RLHF, adversarial training) failed to remove the backdoor, and sometimes taught the model to hide it better.