Craft Adversarial Data
Adversarial data are inputs to an AI model that have been modified such that they cause the adversary's desired effect in the target model. Effects can range from misclassification, to missed detections, to maximizing energy consumption. Typically, the modification is constrained in magnitude or location so that a human still perceives the data as if it were unmodified, but human perceptibility may not always be a concern depending on the adversary's intended effect. For example, an adversarial input for an image classification task is an image the AI model would misclassify, but a human would still recognize as containing the correct class.
Depending on the adversary's knowledge of and access to the target model, the adversary may use different classes of algorithms to develop the adversarial example such as White-Box Optimization, Black-Box Optimization, Black-Box Transfer, or Manual Modification.
The adversary may Verify Attack their approach works if they have white-box or inference API access to the model. This allows the adversary to gain confidence their attack is effective "live" environment where their attack may be noticed. They can then use the attack at a later time to accomplish their goals. An adversary may optimize adversarial examples for Evade AI Model, or to Erode AI Model Integrity.
> sub-techniques
> curated attacks (4)
Sleeper Agents: Backdoored LLMs That Survive Safety Training
Researchers trained LLMs with a hidden trigger — behave normally, but write exploitable code when the prompt says the year is 2024. Standard safety training (RLHF, adversarial training) failed to remove the backdoor, and sometimes taught the model to hide it better.
Bypassing Cylance's AI Malware Detector with Appended Strings
Researchers reverse-engineered Cylance's ML malware model, found which features drove 'benign' scores, and appended strings from a trusted game to real malware — flipping the classifier's verdict on a large sample of known-malicious files.
Robust Physical Perturbations on Stop-Sign Classifiers
Carefully computed sticker patterns applied to a physical stop sign cause road-sign classifiers to misread it as a speed-limit sign across viewing angles and distances — a durable, real-world evasion attack.
Evading Face Recognition with Adversarial Eyeglass Frames
Printed adversarial patterns on ordinary eyeglass frames let a wearer evade or impersonate identities to a face-recognition model — a physically realizable attack that dodges or fools identification while looking unremarkable.