Probability is the language of uncertainty, and underpins statistics and machine learning.
Events and Probabilities
A probability is a number between 0 (impossible) and 1 (certain). The probabilities of all possible outcomes add up to 1.
Conditional Probability
The probability of A given B: P(A | B). The chance that a customer churns given they contacted support twice is different from the overall churn rate.
Independence
Events are independent if one doesn't change the probability of the other. Many real events aren't independent, and assuming they are leads to errors — for example, treating repeat purchases by the same customer as unrelated.
Bayes' Theorem
Bayes' theorem updates a probability in light of evidence:
P(A | B) = P(B | A) × P(A) / P(B)
A classic example: a test that is 99% accurate for a condition affecting 1 in 1,000 people. Most positive results are still false positives, because the condition is so rare. Base rates matter.
Expected Value
The average outcome weighted by probabilities. Useful for decisions: expected cost of a fraud miss versus a false alarm.
Common Misconceptions
- Gambler's fallacy: believing past outcomes change independent future ones.
- Ignoring base rates.
- Confusing P(A | B) with P(B | A).
Why It Matters
Model outputs, confidence intervals, A/B tests and risk assessments all rely on these ideas.