LLM Prompt Obfuscation
Adversaries may hide or otherwise obfuscate prompt injections or retrieval content to avoid detection from humans, large language model (LLM) guardrails, or other detection mechanisms.
For text inputs, this may include modifying how the instructions are rendered such as small text, text colored the same as the background, or hidden HTML elements. For multi-modal inputs, malicious instructions could be hidden in the data itself (e.g. in the pixels of an image) or in file metadata (e.g. EXIF for images, ID3 tags for audio, or document metadata).
Inputs can also be obscured via an encoding scheme such as base64 or rot13. This may bypass LLM guardrails that identify malicious content and may not be as easily identifiable as malicious to a human in the loop.
> curated attacks (2)
Rules File Backdoor: Poisoning AI Coding-Assistant Config
Hidden, obfuscated instructions planted in a shared 'rules file' silently steered AI coding assistants like Cursor and GitHub Copilot to inject malicious code — a supply-chain backdoor invisible in normal review.
Hacking ChatGPT's Long-Term Memory with Prompt Injection
A prompt injection hidden in a shared document wrote false, persistent 'memories' into ChatGPT's long-term memory. Because memories survive across sessions, the planted instructions could exfiltrate future conversations indefinitely.