An adversarial example is an input changed slightly — often imperceptibly — so a model gets it wrong.
Classic Examples
- Adding faint noise to a photo so a classifier sees a panda as a gibbon.
- Stickers on a stop sign that make a vision system misread it.
- Small wording changes that flip a text classifier's decision.
Why Models Are Vulnerable
Models rely on statistical patterns that don't always match human perception. Attackers can find directions in input space that change predictions dramatically.
Attack Settings
- White-box: the attacker knows the model and can compute effective changes.
- Black-box: the attacker only queries the model; attacks often still transfer from similar models.
Defences
- Adversarial training: include adversarial examples in training.
- Input preprocessing: transformations that disrupt perturbations, though adaptive attackers can often bypass them.
- Ensembles and detection: flag suspicious inputs.
- Certified robustness: mathematical guarantees for small perturbations, at a cost to accuracy.
Practical View
For high-stakes systems, assume adversarial inputs exist. Combine robust models with other checks and human oversight.