# Attention Mechanism

> How a model picks which other words matter for each word.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Attention, Attention Layer

## Think of it like

Think of it like a highlighter. For each word, you mark the other words that help explain it.

## Definition

The attention mechanism lets a model pick which other tokens matter most for each token. It gives each other token a weight, then mixes in their meaning by those weights. Take 'The animal did not cross the street because it was tired'. Attention helps the model link 'it' to 'animal'. A transformer runs many of these side by side in each layer.

## Example

Jack's model translates 'The bank by the river flooded'. Attention links 'bank' to 'river', so the model picks the word for a river bank, not a money bank.

## Why it matters

Attention is why language models can keep track of who 'it' is in a long text.

## Common mistake

Reading attention weights as the model's reasons. They show where information flowed, which is only part of why an answer came out.

## Three principles

- **Weights per token.** Which others matter.
- **Mixes in meaning.** By those weights.
- **Many heads, many layers.** Run side by side.

## Related terms

- [Self-Attention](https://learn.benibix.com/term/ai/self-attention)
- [Transformer](https://learn.benibix.com/term/ai/transformer)
- [Dot Product](https://learn.benibix.com/term/general/dot-product)
- [Multi-Head Attention](https://learn.benibix.com/term/ai/multi-head-attention)
- [FlashAttention](https://learn.benibix.com/term/ai/flash-attention)

## Taught in

- [Transformer Architecture](https://learn.benibix.com/learn/ai-llms/transformer-architecture): LLMs, 5/15

![A sentence about an animal that did not cross a street because it was tired. Lines run from the word it to the other words. The thickest line goes to animal.](https://learn.benibix.com/diagrams/llm-attention.svg)

---

Source: https://learn.benibix.com/term/ai/attention-mechanism
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
