Exfiltration via AI Inference API
Adversaries may exfiltrate private information via AI Model Inference API Access. AI Models have been shown leak private information about their training data (e.g. Infer Training Data Membership, Invert AI Model). The model itself may also be extracted (Extract AI Model) for the purposes of AI Intellectual Property Theft.
Exfiltration of information relating to private training data raises privacy concerns. Private training data may include personally identifiable information, or other protected data.
> sub-techniques
> curated attacks (3)
Model Distillation Campaigns Targeting Anthropic Claude
Anthropic uncovered competitor labs harvesting Claude's outputs at scale — roughly 24,000 accounts and 16 million queries — to distill its agentic reasoning, code, tool-use, and computer-use capabilities into rival models. Model theft by imitation, laundered through ordinary API access.
Scalable Extraction of Training Data from ChatGPT
Researchers made a production LLM regurgitate verbatim training data — including PII — by asking it to repeat a word 'forever.' A cheap query-only attack that recovered megabytes of memorized data from ChatGPT.
Model Extraction via Prediction APIs
By querying a pay-per-prediction ML API and observing outputs, attackers reconstruct a near-equivalent copy of the target model — stealing intellectual property and building a proxy for further offline attacks.