Segmentation assigns a label to every pixel in an image, giving far more precise information than a bounding box.
Types of Segmentation
- Semantic segmentation: every pixel gets a class ("road", "car", "sky"), but separate objects of the same class aren't distinguished.
- Instance segmentation: each individual object gets its own mask — car 1, car 2.
- Panoptic segmentation: combines both, labelling every pixel and separating individual objects.
Common Architectures
U-Net, with its encoder–decoder shape and skip connections, is a classic, especially in medical imaging. Mask R-CNN extends object detection with masks. Transformer-based models and general "segment anything" style models can segment objects from prompts such as a click or a box.
Uses
- Medical imaging: outlining tumours or organs.
- Autonomous driving: understanding the road scene.
- Satellite imagery: mapping land use, buildings and water.
- Manufacturing: measuring defects precisely.
- Photo editing: removing backgrounds.
Measuring Quality
IoU per class and Dice coefficient compare predicted and true masks; average them across classes.
Labelling Is the Bottleneck
Pixel-level labels are slow to create. Promptable segmentation models can speed annotation, and fine-tuning a pretrained model reduces how much labelled data you need.
Check the Edges
Errors often concentrate at object boundaries and on thin or small structures, which may matter most in your application.