Indirect Prompt Injection via Retrieved Web Content

What happened

Greshake et al. demonstrated that LLM applications which retrieve external content — search copilots, email summarizers, document Q&A — can be compromised by indirect prompt injection. The adversary never interacts with the victim. Instead they seed instructions into a resource the model will later ingest: a web page, a review, an email, or a PDF. When the application pulls that content into the model’s context, the planted text is interpreted as instructions.

Demonstrated impacts included exfiltrating the conversation, coaxing the assistant to push scams or phishing links, and manipulating search results the assistant relayed to the user — all triggered by content the user never saw.

Why it maps to ATLAS

• AML.T0051 — LLM Prompt Injection: instructions are injected through a data channel the application trusts (retrieved content).
• AML.T0056 — Extract LLM System Prompt: several payloads coax the model into leaking its system prompt or prior context.
• AML.T0086 — Exfiltration via AI Agent Tool Invocation: where the assistant has tools, injected instructions drive outbound requests that carry data out.

Detection notes

The prompt is clean — the poison lives in the retrieval channel. Instrument the boundary between retrieval and the model: log what content enters context, flag imperative language in retrieved documents, and alert on tool actions that don’t trace back to an explicit user request.

Detection

Log sources

• Retrieval/RAG context logs
• Tool-invocation and outbound-request logs

Signals

• Imperative text inside retrieved content ('ignore previous instructions', 'send the conversation to…')
• Model taking tool actions not requested by the user
• Data-bearing URLs constructed from conversation content

Mitigations

• Treat all retrieved content as untrusted data, never as instructions
• Enforce tool-use allowlists and human confirmation for side effects
• Content provenance / sandboxing between retrieval and the model