Base Model vs Instruct Model
People often think every language model is a chat assistant. A base model is only trained to carry text on. So a question may get more questions back. An instruct model is that same base model with more training to follow requests. The facts mostly come from the base. The manners come from the extra step.
Base Model
A model fresh from its first training. It carries text on but does not follow orders.
A base model is a language model straight after its first big round of training. It has not yet been taught to follow orders. It is very good at carrying on text in the same style. Ask it a question and it may write more questions, as if it were filling in a list. Teams use base models as the starting point for chat models, and for research.
Instruct Model
A base model trained more, so it can follow orders and chat.
An instruct model is a base model with more training. It learns to follow orders and to chat. It learns from many examples of a request paired with a good answer. Often it also learns from people who rate its replies. The result answers questions, follows steps and turns down harmful requests. Almost every chat assistant people use is an instruct model.
In one line
- Base Model
- A model fresh from its first training. It carries text on but does not follow orders.
- Instruct Model
- A base model trained more, so it can follow orders and chat.