Skip to content
BENIBIX Learn

Sequence-to-Sequence Model (Seq2Seq)

AI🔤 NLP & LanguageEncoder and Decoder Models

Also known as Seq2Seq, Encoder-Decoder Model, Sequence to Sequence

A model with two parts. One reads the input, and the other writes a new output from it.

Think of it like

It is like a friend who hears your whole sentence. Then she says it in French.

Example

A seq2seq model reads 'Where is the station?' in English. Its decoder writes 'Wo ist der Bahnhof?' in German, one word at a time.

Why it matters

It made it possible to turn text into other text of any length. Translation and summary tools grew from it.

Definition

A seq2seq model turns one string of words into a new one. It has two parts. The encoder reads the whole input and packs its meaning into numbers. The decoder then writes the output one piece at a time, using those numbers. The input and output can be different lengths, which suits translation and summaries. A paper in 2014 showed it working well for English to French.
The encoder reads it all, then the decoder writes the new sentence word by word.

Common mistake

Thinking every language model has these two parts. Most chat models now use just the writing half.

Key takeaways

  1. 1
    Encoder reads. The whole input comes first.
  2. 2
    Decoder writes. One piece at a time.
  3. 3
    Any length. Input and output can differ.