# Transformer

> The design inside most big chat models. It is built on attention.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** Transformer Model, Transformer Architecture

## Think of it like

Think of it like a reader who sees all the words on a page at once. They do not read one by one.

## Definition

A transformer is a kind of neural network built for text. Its key part is attention, which lets each word look at all the others at once. Older networks read one word at a time. A transformer reads them all at once, so it trains fast on GPUs. It came out in 2017, and most big language models since are built on it.

## Example

Emma's assistant reads her 10-page report. Its transformer lets each word look back at the title and the key words, all in one pass.

## Why it matters

Without it, the big chat models of today could not have been trained.

## Common mistake

Thinking a transformer is a product. It is a design, like a type of engine. Many different models use it.

## Three principles

- **Built on attention.** Every token sees the rest.
- **Trains in parallel.** Fast on GPUs.
- **Since 2017.** Under nearly every LLM.

## Related terms

- [Attention Mechanism](https://learn.benibix.com/term/ai/attention-mechanism)
- [Neural Network](https://learn.benibix.com/term/ai/neural-network)
- [Large Language Model (LLM)](https://learn.benibix.com/term/ai/large-language-model)
- [Recurrent Neural Network (RNN)](https://learn.benibix.com/term/ai/recurrent-neural-network)

## Taught in

- [Transformer Architecture](https://learn.benibix.com/learn/ai-llms/transformer-architecture): LLMs, 5/15

![A tall stack. Tokens at the bottom become vectors with their positions added. They pass up through repeated blocks of attention and a small network, and the top gives a chance for each next token.](https://learn.benibix.com/diagrams/llm-transformer.svg)

---

Source: https://learn.benibix.com/term/ai/transformer
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
