An estimate from a sample — an average, a conversion rate, a difference between groups — is uncertain. A confidence interval shows how uncertain.
What It Is
A 95% confidence interval is a range calculated so that, if you repeated the sampling many times, about 95% of such intervals would contain the true value. For example: the conversion rate is 4.2%, 95% CI 3.8% to 4.6%.
Reading It
- Narrow interval: precise estimate.
- Wide interval: considerable uncertainty — often too little data.
- For a difference between groups, an interval that excludes zero indicates a statistically significant difference at that level.
What Makes Intervals Wider
- Smaller samples.
- More variable data.
- Higher confidence levels (99% intervals are wider than 95%).
Common Misreading
It's tempting to say "there's a 95% probability the true value is in this interval". Strictly, the 95% refers to the method's long-run success rate. In practice, treat it as a plausible range for the true value.
Why Use Them
Confidence intervals show both the size of an effect and its uncertainty, which p-values alone don't. "Revenue increased by $2 per user (95% CI: $0.10 to $3.90)" tells decision-makers far more than "p = 0.04".
Bootstrapping
When formulas are awkward, bootstrap intervals resample the data many times and use the spread of recalculated estimates. They work for medians, ratios and model metrics too.