AI Agent Context Poisoning

Adversaries may attempt to manipulate the context used by an AI agent's large language model (LLM) to influence the responses it generates or actions it takes. This allows an adversary to persistently change the behavior of the target agent and further their goals.

Context poisoning can be accomplished by prompting the an LLM to add instructions or preferences to memory (See Memory) or by simply prompting an LLM that uses prior messages in a thread as part of its context (See Thread).