Skip to content
BENIBIX Learn

BERT (Language Model)

AI🔤 NLP & LanguageEncoder and Decoder Models

Also known as BERT, Bidirectional Encoder Representations from Transformers, BERT Model

A language model from 2018 that reads text in both directions. It is built to understand, not to write.

Think of it like

It is like a reader who looks at both sides of a gap. Then she guesses the missing word.

Example

In 2019, Google said BERT would help Search understand one in 10 searches in the US in English.

Why it matters

It showed that one model trained on plain text could be tuned for many language tasks. Many search and labeling tools still run on models like it.

Definition

BERT is a language model that reads text both ways. It came out in 2018, and its name stands for Bidirectional Encoder Representations from Transformers. It looks at the words on both sides of each word at once. It learned by filling in hidden words in books and web pages. It came in two sizes, with 110 million and 340 million parameters. It does not write long text, but turns text into numbers that other tools use to sort and search.
BERT reads the words on both sides of the gap. That is the 'bidirectional' in its name.

Common mistake

Thinking BERT is a chatbot like the ones people talk to today. It reads and labels text, but it does not write replies.

Key takeaways

  1. 1
    Both directions. It looks left and right of each word.
  2. 2
    Encoder only. Built to understand, not to write.
  3. 3
    Tune for a task. One base model, many uses.

Compare