Outliers are values far from the rest of the data. Some are errors; some are the most important observations you have.
Finding Outliers
- Visual checks: box plots, histograms and scatter plots.
- Rules of thumb: values beyond 1.5 times the interquartile range from the quartiles, or more than three standard deviations from the mean (for roughly normal data).
- Domain rules: a human height of 3 metres or a negative age is impossible.
- Multivariate methods: a value may be normal on its own but unusual in combination, such as a high salary for a junior role.
Error or Real?
Investigate before acting. Typos, unit mix-ups (grams versus kilograms), sensor faults and default values are errors. Big purchases, extreme weather and fraud are real — and often exactly what you care about.
Treatment Options
- Correct errors when the true value can be recovered.
- Remove clear errors that can't be fixed.
- Cap (winsorise) extreme values at a chosen percentile.
- Transform long-tailed data, for example with a logarithm.
- Use robust methods: medians, robust scaling, tree-based models and absolute-error losses are less sensitive to outliers.
- Model them separately when extremes behave differently.
Document Decisions
Record which outliers were changed or removed and why. Treating outliers inconsistently between training and production causes silent errors.