# Self-Attention

> Each word in a text looks at the other words in that same text.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Intra-Attention, Scaled Dot-Product Attention

## Think of it like

Think of it like each word in a line. It looks at all the other words in that same line.

## Definition

Self-attention is attention where a text looks at itself. Each token looks at the other tokens in the same text. Each token makes three vectors, called a query, a key and a value. The query of one token is checked against the key of each other token with a dot product. The scores decide how much of each value to mix in. In a model that writes text, each token may only look back, never ahead.

## Example

Take 'Lucy gave Jack her book'. Self-attention lets 'her' check itself against 'Lucy' and 'Jack'. It leans on 'Lucy'.

## Why it matters

It is the step that runs in every layer of a transformer. So it is where most of the work gets done.

## Common mistake

Thinking the model can peek ahead while it writes. In a writing model, each token only sees the tokens before it.

## Three principles

- **Looks at itself.** The same sequence.
- **Query, key, value.** Three vectors per token.
- **Only looks back.** When writing text.

## Related terms

- [Attention Mechanism](https://learn.benibix.com/term/ai/attention-mechanism)
- [Transformer](https://learn.benibix.com/term/ai/transformer)
- [Dot Product](https://learn.benibix.com/term/general/dot-product)
- [Vector](https://learn.benibix.com/term/general/vector)
- [Multi-Head Attention](https://learn.benibix.com/term/ai/multi-head-attention)

## Taught in

- [Self-Attention and Multi-Head](https://learn.benibix.com/learn/ai-llms/self-attention-and-multi-head): LLMs, 6/15

![A sentence about an animal that did not cross a street because it was tired. Lines run from the word it to the other words. The thickest line goes to animal.](https://learn.benibix.com/diagrams/llm-attention.svg)

---

Source: https://learn.benibix.com/term/ai/self-attention
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
