Skip to content
BENIBIX Learn

Dense Vector vs Sparse Vector

People think a sparse vector is a weaker dense one, and it is not. A dense vector scores hidden traits of meaning, so it finds texts that say the same thing in other words. A sparse vector has one slot per word, so it finds exact words, names and codes. Each catches what the other misses, which is why hybrid search uses both.

AI🔎 Retrieval & Search

Dense Vector

A list of numbers with a value in each slot. It holds meaning, not exact words.

A dense vector is a list of numbers where almost every number has a value. Embedding models make dense vectors. The numbers are not words. Each one scores a hidden feature of the meaning. So two texts with the same meaning get close vectors, even with no words in common. Searching with them is called dense retrieval.

Example
A search for "cheap flights" finds a page titled "low-cost airfare". The words differ, but their dense vectors sit close together.
Why it matters
Dense vectors find matches by meaning. That is what makes semantic search work.
Full entry
AI🔎 Retrieval & Search

Sparse Vector

A long list of numbers, mostly zeros. Each slot stands for one word.

A sparse vector is a very long list of numbers where nearly all are zero. Each slot stands for one word in a big vocabulary. Only the words in the text get a value, often a weight for how much that word matters. Keyword search methods like BM25 work this way. Sparse vectors are great at exact words, names and codes. They miss matches that use different words.

Example
A support bot gets the code "E-404". A sparse vector finds the one manual page with that exact code, where a dense vector drifts to error pages in general.
Why it matters
Exact terms matter in real search. Sparse and dense vectors together are the base of hybrid search.
Full entry

In one line

Dense Vector
A list of numbers with a value in each slot. It holds meaning, not exact words.
Sparse Vector
A long list of numbers, mostly zeros. Each slot stands for one word.