Cosine Similarity vs Dot Product
People often think cosine similarity and the dot product always put results in the same order. They do not, and length is the reason. The dot product grows with how long the vectors are, not just their angle. Cosine similarity looks at the angle alone. So the two agree only when each vector has a length of 1. Many embedding models do that, but some do not.
Cosine Similarity
A score that says how alike two vectors are, by their angle.
Cosine similarity measures how alike two vectors are by the angle between them, not their length. The score runs from 1, pointing the same way, through 0, at right angles, to minus 1, pointing opposite ways. For embeddings, a higher score means closer meaning. It is the most common way to compare embeddings in search. When vectors are scaled to length 1, it gives the same result as the dot product.
Dot Product
Times each pair of numbers in two lists. Then add it all up.
The dot product turns two vectors of the same length into one number. You multiply the numbers in each matching place, then add up the results. For [1, 2] and [3, 4], that is 1 × 3 + 2 × 4, which is 11. A large result means the two vectors point the same way. Neurons use it to weigh their inputs, and search uses it to find items that are alike.
In one line
- Cosine Similarity
- A score that says how alike two vectors are, by their angle.
- Dot Product
- Times each pair of numbers in two lists. Then add it all up.
These live in different corners of the glossary (AI and General), which is exactly why they get confused.