An autoencoder is a neural network trained to reproduce its input after passing it through a narrow bottleneck.
Structure
- Encoder: compresses input into a smaller representation.
- Bottleneck: the compressed code.
- Decoder: reconstructs the input from the code.
Because the bottleneck is small, the network must learn the most important features of the data.
Uses
- Dimensionality reduction: a non-linear alternative to PCA.
- Anomaly detection: trained on normal data, the autoencoder reconstructs normal examples well and unusual ones poorly; high reconstruction error flags anomalies.
- Denoising: trained to reconstruct clean data from noisy inputs.
- Pretraining and feature learning.
Variants
- Denoising autoencoders.
- Variational autoencoders (VAEs): learn a smooth probabilistic latent space useful for generating new data, and a component of some image generation systems.
- Sparse autoencoders: encourage few active features — used in interpretability research to find meaningful features inside language models.
Limitations
Reconstructions can be blurry; anomaly thresholds need careful tuning.