# BERT (Language Model)

> A language model from 2018 that reads text in both directions. It is built to understand, not to write.

**Field:** Artificial Intelligence  
**Category:** NLP & Language  
**Also known as:** BERT, Bidirectional Encoder Representations from Transformers, BERT Model

## Think of it like

It is like a reader who looks at both sides of a gap. Then she guesses the missing word.

## Definition

BERT is a language model that reads text both ways. It came out in 2018, and its name stands for Bidirectional Encoder Representations from Transformers. It looks at the words on both sides of each word at once. It learned by filling in hidden words in books and web pages. It came in two sizes, with 110 million and 340 million parameters. It does not write long text, but turns text into numbers that other tools use to sort and search.

## Example

In 2019, Google said BERT would help Search understand one in 10 searches in the US in English.

## Why it matters

It showed that one model trained on plain text could be tuned for many language tasks. Many search and labeling tools still run on models like it.

## Common mistake

Thinking BERT is a chatbot like the ones people talk to today. It reads and labels text, but it does not write replies.

## Three principles

- **Both directions.** It looks left and right of each word.
- **Encoder only.** Built to understand, not to write.
- **Tune for a task.** One base model, many uses.

## Related terms

- [Masked Language Model](https://learn.benibix.com/term/ai/masked-language-model)
- [Encoder (Transformer)](https://learn.benibix.com/term/ai/encoder)
- [Transformer](https://learn.benibix.com/term/ai/transformer)
- [Pretraining](https://learn.benibix.com/term/ai/pretraining)
- [Fine-Tuning](https://learn.benibix.com/term/ai/fine-tuning)
- [Semantic Search](https://learn.benibix.com/term/ai/semantic-search)

## Told apart from

- [BERT vs GPT](https://learn.benibix.com/compare/bert-vs-gpt): People think BERT is an early chat model. It is not. BERT reads a whole text both ways and hands back numbers that other tools use to sort and search. GPT reads only the words so far and writes the next one. One reads, and the other writes.

![The sentence 'The cat sat on the [MASK] by the door' with arrows reaching the mask from words on both sides. Below it, guesses with bars: mat is long, floor is shorter, roof is very short.](https://learn.benibix.com/diagrams/ai5-masked-word.svg)

---

Source: https://learn.benibix.com/term/ai/bert
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
