Skip to content
BENIBIX Learn

RLHF vs RLVR

People think RLVR is just a new name for RLHF. It is not. In RLHF, people rank answers and a reward model learns their taste, so it can reward any kind of reply. In RLVR, a program checks the answer against a key or runs tests, so it only works where a right answer exists. RLHF shapes tone and help. RLVR trains math and code.

AI💬 LLMs

Reinforcement Learning from Human Feedback (RLHF)

A model learns which answers people like best. Then it gives more of those.

Reinforcement learning from human feedback trains a model to give the kind of answers people prefer. First, people compare pairs of answers and pick the better one. A reward model learns from those picks to score any answer. Then the language model is trained to write answers that score high. It makes replies more helpful and polite, and less likely to cause harm.

Example
Workers see two replies to 'How do I fix a flat tyre?' and pick the clearer one, 100,000 times over. The model then learns to write the clearer kind.
Why it matters
It is a big reason chat helpers are kind and useful, not odd or rude.
Full entry
AI💬 LLMs

RL with Verifiable Rewards (RLVR)

Points for a model each time a check shows it got the right answer.

Reinforcement learning with verifiable rewards, or RLVR, trains a model on tasks a program can check. The model tries a problem, such as a sum or a coding task. A checker compares the final answer with the key, or runs the code's tests. It gives a reward only for a pass, so no person or reward model has to judge. The name comes from a late 2024 paper. The method is used to train reasoning models on math and code.

Example
A model works through 1,000 math problems. A short script checks each final number against the answer key, and only the right ones earn a reward.
Why it matters
A check by a program is cheap and hard to argue with. That makes long training on math and code possible.
Full entry

In one line

Reinforcement Learning from Human Feedback (RLHF)
A model learns which answers people like best. Then it gives more of those.
RL with Verifiable Rewards (RLVR)
Points for a model each time a check shows it got the right answer.