An embedding is a list of numbers — a vector — that represents an item so that similar items have similar vectors.
The Idea
An embedding model maps text (or images, audio, products) into a space with hundreds or thousands of dimensions. Items with related meanings end up close together: "reset my password" sits near "I can't log in", far from "office opening hours".
Measuring Similarity
The usual measure is cosine similarity, the cosine of the angle between two vectors, from −1 to 1. If vectors are normalised to length 1, it equals their dot product, which is fast to compute.
What They're Used For
- Semantic search: find passages that match a question's meaning, not just its words.
- Retrieval for RAG.
- Clustering and topic discovery.
- Deduplication: find near-duplicate records.
- Recommendations: items similar to ones a user liked.
- Features for classifiers: a quick, strong baseline for text classification.
Choosing an Embedding Model
Consider the languages and domain you need, maximum input length, vector size (bigger vectors cost more storage), speed and licence. Small open models such as all-MiniLM-L6-v2 are a good starting point; benchmarks help compare options.
Practical Notes
- Use the same model for indexing and querying.
- Long documents must be split into chunks within the model's input limit.
- Re-embed everything if you change models.
- Embeddings capture meaning imperfectly; exact names and codes are often better found with keyword search.