Simple A/B tests don't suit every situation. Other designs fill the gaps.
Multivariate Tests
Test several changes and their combinations at once. They need much more traffic, but can reveal interactions.
Switchback Experiments
Alternate treatments over time periods rather than users — useful in marketplaces where users affect each other, such as delivery or ride-hailing.
Geo Experiments
Apply treatments in some regions and compare them with similar regions — useful for advertising and offline changes.
Cluster Randomisation
Randomise groups (stores, schools, teams) when individual randomisation isn't possible or treatments spill over.
Quasi-Experiments
When randomisation isn't possible, use difference-in-differences, synthetic controls or regression discontinuity.
Interference
If one user's treatment affects others, simple A/B test results are biased. Choose designs that account for it.
Choosing
Consider how treatments spread, available traffic and the decision at stake. Consult statisticians for high-stakes experiments.