Skip to content
BENIBIX Learn

Post-Training

AI💬 LLMsLanguage Model Training

Also known as Posttraining, LLM Post-Training

The last stage of training. It turns a model that has read a lot into a helper.

Think of it like

It is like manners class. You already know lots of words. Now you learn to be helpful and kind.

Example

At first, the lab's model can only carry on text. After post-training, it answers questions, keeps to rules, and says no to bad asks.

Why it matters

Most of what makes a chat model feel helpful, safe and good at reasoning is added here.

Definition

Post-training is all the training a language model gets after pretraining. Pretraining teaches it to guess the next word from a huge pile of text. Post-training turns that into a helpful assistant. First it copies good example answers. Then it learns which answers people prefer, with methods like RLHF or DPO. Many labs also reward it for right answers on tasks a computer can check, such as maths and code.
This is the stage after pretraining. It turns the model into a helper.

Common mistake

Thinking it adds new facts. Most facts come from pretraining. Post-training mostly shapes how the model acts.

Key takeaways

  1. 1
    Copy good answers. Examples written to learn from.
  2. 2
    Learn preferences. Which answer people like more.
  3. 3
    Reward right work. On tasks a computer can check.

Learn it properly