AI bias occurs when a system produces systematically worse or unfair outcomes for some groups of people.
Sources of Bias
- Historical bias: data reflects past inequalities, such as pay gaps or discriminatory decisions.
- Representation bias: some groups are under-represented in training data, so the model performs worse for them.
- Measurement bias: features or labels are measured differently across groups.
- Label bias: labels encode human judgements, which may themselves be biased.
- Aggregation bias: one model applied to groups whose patterns differ.
- Deployment bias: a system used in a different context or population from the one it was designed for.
Why Removing Sensitive Attributes Isn't Enough
Other features often act as proxies: postcode, first name, school or shopping patterns can correlate strongly with ethnicity, gender or age. The model can learn the same bias indirectly. Sometimes you need sensitive attributes to measure fairness.
Detecting Bias
Break performance and outcomes down by group: error rates, selection rates, calibration. Look at small subgroups and intersections, where problems often hide.
Reducing Bias
- Improve data collection and representation.
- Revisit labels and objectives.
- Apply fairness constraints or reweighting during training.
- Adjust decision thresholds carefully and transparently.
- Add human review for affected decisions.
Keep Checking
Bias can emerge after deployment as populations and behaviour change. Monitor outcomes by group over time.