Skip to content
BENIBIX Learn

Word Embedding

AI🔤 NLP & LanguageText Representation

Also known as Word Vector, Word2Vec

Each word as a list of numbers, so words with close meanings sit close.

Think of it like

Think of it like a map where words that mean close things live on the same street.

Example

Lucy's search tool uses word embeddings. A search for 'car' also finds pages that only say 'vehicle', because the two vectors are close.

Why it matters

Embeddings let a computer tell how alike two words are. The language models of today are built on this idea.

Definition

A word embedding gives each word a vector, a list of a few hundred numbers. The numbers are learned from a corpus, so words used in the same way end up with close vectors. 'Cat' sits near 'dog', and far from 'tax'. Directions in the space can carry meaning too: king minus man plus woman lands near queen. Word2Vec, from 2013, made the idea popular.
Words with close meanings sit together. The step from man to woman matches the step from king to queen.

Common mistake

Thinking the numbers are hand-picked meanings. Nobody chose them; they come from how words are used together.

Key takeaways

  1. 1
    A vector per word. A few hundred numbers.
  2. 2
    Close means alike. Cat sits near dog.
  3. 3
    Learned from text. Not chosen by hand.

Compare