A prediction without a measure of uncertainty can mislead. Knowing how confident a model is helps decide when to trust it.
Types of Uncertainty
- Aleatoric: inherent randomness in the data — some outcomes are unpredictable.
- Epistemic: uncertainty from limited knowledge — reducible with more data.
Methods
- Calibrated probabilities for classifiers.
- Prediction intervals for regression, from quantile regression or statistical models.
- Bayesian models, which produce distributions over outcomes.
- Ensembles: disagreement among models indicates uncertainty.
- Conformal prediction: produces prediction sets or intervals with statistical coverage guarantees under mild assumptions.
Uses
- Routing uncertain predictions to human review.
- Communicating forecast ranges to decision-makers.
- Detecting inputs unlike the training data.
- Risk-aware decisions.
Communicating Uncertainty
Show ranges and scenarios rather than single numbers, and explain what they mean in plain language.