Skip to content
BENIBIX Learn

Transformer

AI💬 LLMsTransformer Architecture

Also known as Transformer Model, Transformer Architecture

The design inside most big chat models. It is built on attention.

Think of it like

Think of it like a reader who sees all the words on a page at once. They do not read one by one.

Example

Emma's assistant reads her 10-page report. Its transformer lets each word look back at the title and the key words, all in one pass.

Why it matters

Without it, the big chat models of today could not have been trained.

Definition

A transformer is a kind of neural network built for text. Its key part is attention, which lets each word look at all the others at once. Older networks read one word at a time. A transformer reads them all at once, so it trains fast on GPUs. It came out in 2017, and most big language models since are built on it.
Tokens go in at the bottom and pass up through the same block many times. At the top, out come chances for the next token.

Common mistake

Thinking a transformer is a product. It is a design, like a type of engine. Many different models use it.

Key takeaways

  1. 1
    Built on attention. Every token sees the rest.
  2. 2
    Trains in parallel. Fast on GPUs.
  3. 3
    Since 2017. Under nearly every LLM.

Learn it properly