# Positional Encoding

> It adds each token's place in the text, so word order is not lost.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Position Embedding, Rotary Position Embedding, RoPE

## Think of it like

Think of it like seat numbers on a bus. They tell you who sits where in the line.

## Definition

Positional encoding adds where each token sits in the text. Attention on its own treats text like a bag of tokens. Without this, 'dog bites man' and 'man bites dog' would look the same. The first transformer added a pattern of waves to each token's vector. Most models now use a method called rotary position embedding. It turns each vector by an angle based on where it sits.

## Example

Lucy's model reads 'not bad, really good' and 'not good, really bad'. Positional encoding is what lets it tell the two apart.

## Why it matters

Order carries meaning. Without position, a transformer could not tell who did what to whom.

## Common mistake

Thinking the model reads left to right like a person. Attention sees all tokens at once. The position signal is the only reason that order counts.

## Three principles

- **Adds position.** Where each token sits.
- **Keeps order.** Dog bites man is not man bites dog.
- **Waves or rotation.** Two common methods.

## Related terms

- [Transformer](https://learn.benibix.com/term/ai/transformer)
- [Self-Attention](https://learn.benibix.com/term/ai/self-attention)
- [Bag of Words](https://learn.benibix.com/term/ai/bag-of-words)
- [Token (AI)](https://learn.benibix.com/term/ai/token)

## Taught in

- [Transformer Architecture](https://learn.benibix.com/learn/ai-llms/transformer-architecture): LLMs, 5/15

![Two sentences with the same three words in a different order. Each word carries a position number, 1, 2 or 3, so the two sentences look different to the model.](https://learn.benibix.com/diagrams/llm-positional.svg)

---

Source: https://learn.benibix.com/term/ai/positional-encoding
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
