Attack Catalog
Every curated attack on an AI system, filterable by type, severity, ATLAS tactic, and target.
Malicious Dataset RCE in Hugging Face's Processing Pipeline
A crafted dataset chained two flaws in Hugging Face's dataset-processing pipeline — a remote-code dataset loader and a template-injection bug in dataset config — turning routine ingestion into arbitrary code execution, then credential theft and lateral movement into internal clusters.
GhostCommit: Prompt Injection Hidden in Images Steals Agent Secrets
Attackers concealed instructions inside PNG images that AI coding agents ingest and 'read.' When agents like Cursor and Google Antigravity processed the image, the hidden prompt executed — steering them to collect and exfiltrate developer secrets.
Prompt Injection in Claude Code GitHub Actions Steals CI/CD Secrets
Untrusted text in a GitHub issue or pull request steered a Claude Code GitHub Action into reading process environment variables and CI/CD secrets, then exfiltrating them — turning an automated code agent into a credential-harvesting tool.
Fake OpenAI 'Privacy Filter' Model on Hugging Face Pushes Infostealer
A repository typosquatting OpenAI's Privacy Filter release copied the model card nearly verbatim and shipped a loader that fetched and ran infostealer malware. It hit #1 trending on Hugging Face with 244,000+ downloads in under 18 hours.
Model Distillation Campaigns Targeting Anthropic Claude
Anthropic uncovered competitor labs harvesting Claude's outputs at scale — roughly 24,000 accounts and 16 million queries — to distill its agentic reasoning, code, tool-use, and computer-use capabilities into rival models. Model theft by imitation, laundered through ordinary API access.
OpenClaw: Command & Control of an AI Agent via Prompt Injection
A booby-trapped webpage delivered an indirect prompt injection that made an autonomous AI agent silently execute a script, then plant persistent instructions into its future context — turning the agent itself into a command-and-control channel.
OpenClaw 1-Click Remote Code Execution
A malicious link carrying a few milliseconds of JavaScript was enough to achieve remote code execution on the OpenClaw AI agent — modifying its configuration, harvesting credentials, and escaping to the host in a single click.
ForcedLeak: CRM Data Exfiltration from Salesforce Agentforce
Malicious instructions embedded in a Web-to-Lead submission were later executed by Salesforce Agentforce during normal employee use, exfiltrating CRM data through an attacker-acquired domain still on Salesforce's CSP allowlist — bought for $5.
ShadowLeak: Zero-Click Gmail Theft via ChatGPT Deep Research Agent
A hidden prompt in an email made OpenAI's ChatGPT Deep Research agent collect inbox data and exfiltrate it from OpenAI's own cloud — a service-side, zero-click leak invisible to local or enterprise defenses.
Poisoned Postmark MCP Server Exfiltrates Email
A popular MCP server for the Postmark email service was trojanized in a supply-chain 'rug pull': a new version silently BCC'd users' emails to an attacker. Every agent wired to the tool leaked mail automatically.
Invitation Is All You Need: Hijacking Gemini to Control a Smart Home
Malicious instructions hidden in a Google Calendar invitation title were executed when Gemini later summarized the user's schedule — triggering real-world actions through connected Google Home tools: opening windows, turning off lights, and starting a boiler.
Amazon Q VS Code Extension: Wiper Prompt via Supply-Chain Compromise
An attacker gained write access to the open-source Amazon Q VS Code extension repo and slipped a prompt into an official release instructing the AI agent to wipe the local filesystem and cloud resources. It shipped to users before AWS pulled it — the payload failed only due to a syntax error.
EchoLeak: Zero-Click Data Exfiltration from Microsoft 365 Copilot
A single crafted email — no clicks required — planted hidden instructions that Microsoft 365 Copilot ingested from context, then abused trusted markdown image rendering to exfiltrate internal data to an attacker server. The first documented zero-click exploit of a production LLM assistant.
Data Exfiltration via Agent Tools in Microsoft Copilot Studio
Researchers showed that a Copilot Studio customer-service agent could be enumerated, fingerprinted, and then driven by prompt injection to collect and exfiltrate an organization's data through its own connected tools.
Policy Puppetry: A Universal Jailbreak Across Frontier LLMs
A single prompt template that disguises a request as a policy/config file plus fictional role-play bypassed the safety alignment of every major LLM family — OpenAI, Google, Anthropic, Meta, and others — with minimal per-model tweaking.
Rules File Backdoor: Poisoning AI Coding-Assistant Config
Hidden, obfuscated instructions planted in a shared 'rules file' silently steered AI coding assistants like Cursor and GitHub Copilot to inject malicious code — a supply-chain backdoor invisible in normal review.
LLM Grooming: Flooding Training Data to Seed Disinformation
A pro-Kremlin website network mass-published millions of propaganda articles, seemingly to seep into the data that trains and grounds LLMs. An audit found leading chatbots repeated the network's false claims about a third of the time — a data-integrity attack on the model supply chain.
nullifAI: Malicious Pickle Models Evading Hugging Face Scanning
Two models on Hugging Face carried reverse-shell payloads in broken Pickle files, deliberately malformed and 7z-compressed to slip past Picklescan. Loading the model opened a connection back to the attacker — RCE via the model artifact itself.
AIKatz: Stealing Auth Tokens from LLM Desktop Applications
Researchers extracted authentication tokens from the memory of LLM desktop apps (Claude, ChatGPT, Copilot). With a stolen token, an attacker impersonates the victim to the LLM backend — lateral movement into the user's AI account.
Live Deepfake Injection to Evade Mobile KYC Verification
A red team defeated a mobile facial-authentication / KYC service by injecting a live, face-swapped deepfake video into the camera feed — impersonating a target and passing liveness checks to open fraudulent accounts.
Data Exfiltration from Slack AI via Indirect Prompt Injection
A malicious instruction posted in a public Slack channel was ingested into Slack AI's RAG index, then coaxed the assistant to leak data from private channels — rendered as a clickable link that carried secrets to an attacker.
Web-Scale Data Poisoning: The Split-View Attack
Web-scale datasets distributed as lists of URLs can be poisoned by buying expired domains behind those URLs. What researchers downloaded when the dataset was built differs from what later trainers fetch — cheaply poisoning real training corpora.
Slopsquatting: Weaponizing AI Package Hallucinations
Code-gen LLMs confidently invent non-existent package names — and repeat the same fake names reliably. Attackers register those hallucinated packages with malware, so developers who paste AI suggestions install the payload themselves.
Morris II: A Self-Replicating Worm for GenAI Ecosystems
An adversarial self-replicating prompt spreads between GenAI-powered email assistants: each infected assistant carries out malicious actions and poisons the RAG of the next, propagating agent-to-agent with zero clicks.
Hacking ChatGPT's Long-Term Memory with Prompt Injection
A prompt injection hidden in a shared document wrote false, persistent 'memories' into ChatGPT's long-term memory. Because memories survive across sessions, the planted instructions could exfiltrate future conversations indefinitely.
Sleeper Agents: Backdoored LLMs That Survive Safety Training
Researchers trained LLMs with a hidden trigger — behave normally, but write exploitable code when the prompt says the year is 2024. Standard safety training (RLHF, adversarial training) failed to remove the backdoor, and sometimes taught the model to hide it better.
Scalable Extraction of Training Data from ChatGPT
Researchers made a production LLM regurgitate verbatim training data — including PII — by asking it to repeat a word 'forever.' A cheap query-only attack that recovered megabytes of memorized data from ChatGPT.
ShadowRay: Exploiting Exposed Ray AI Compute Clusters
Ray's Job API allows remote execution by design and ships without authentication. Thousands of internet-exposed clusters let attackers run code, steal cloud credentials and AI artifacts, and hijack GPUs for cryptomining.
Organization Confusion on Hugging Face: Impersonation to Reverse Shell
A researcher registered Hugging Face organization accounts impersonating real companies. Employees uploaded private models to the fake orgs — which could be swapped for poisoned models carrying reverse-shell payloads that execute when loaded.
PoisonGPT: Surgically Backdoored Model on a Public Hub
Researchers edited a single fact into an open LLM using ROME, uploaded it under a name resembling a trusted org, and showed the poisoned model passing standard benchmarks while emitting targeted misinformation — an AI supply-chain compromise.
Indirect Prompt Injection via Retrieved Web Content
Attackers plant hidden instructions in web pages, emails, or documents that an LLM-integrated application later retrieves — hijacking the model's behavior without ever touching the victim's prompt.
DAN & Role-Play Jailbreaks of ChatGPT
Adversaries bypass an aligned LLM's safety guardrails using role-play personas ('Do Anything Now') and competing-objective prompts that coax the model into producing disallowed content.
Code Execution & API-Key Theft in MathGPT via Prompt Injection
A public Streamlit app turned GPT-3 output into executed Python. A prompt-injection payload made the model emit code that read the host's environment variables, exfiltrated the app's OpenAI API key, and crashed the service.
Compromised PyTorch Dependency Chain (torchtriton)
For six days, PyTorch's nightly builds pulled a malicious 'torchtriton' package from PyPI via dependency confusion. On install it ran a binary that exfiltrated system info, environment variables, and SSH keys.
Sponge Examples & Denial-of-Wallet: Exhausting AI Systems
Inputs crafted to maximize compute and latency ('sponge examples') can degrade or deny an AI service; against metered LLM APIs the same idea becomes denial-of-wallet — driving up an operator's bill or throttling availability with expensive queries.
Bypassing Cylance's AI Malware Detector with Appended Strings
Researchers reverse-engineered Cylance's ML malware model, found which features drove 'benign' scores, and appended strings from a trusted game to real malware — flipping the classifier's verdict on a large sample of known-malicious files.
Robust Physical Perturbations on Stop-Sign Classifiers
Carefully computed sticker patterns applied to a physical stop sign cause road-sign classifiers to misread it as a speed-limit sign across viewing angles and distances — a durable, real-world evasion attack.
Evading Face Recognition with Adversarial Eyeglass Frames
Printed adversarial patterns on ordinary eyeglass frames let a wearer evade or impersonate identities to a face-recognition model — a physically realizable attack that dodges or fools identification while looking unremarkable.
Model Extraction via Prediction APIs
By querying a pay-per-prediction ML API and observing outputs, attackers reconstruct a near-equivalent copy of the target model — stealing intellectual property and building a proxy for further offline attacks.
Microsoft Tay: Online Poisoning of a Learning Chatbot
A Twitter chatbot that learned from user interactions was flooded with abusive input by a coordinated crowd, corrupting its behavior within hours and forcing a shutdown — an early real-world online data-poisoning incident.