Skip to content
BENIBIX Learn

Fine-Tuning and LoRA

Level 3 · How a model is trained

How to train a finished model for one job. Full fine-tuning against LoRA and QLoRA, and why tuning is a poor way to add facts.

Course contentsLesson 9 of 15

After this lesson you can

  • Say when fine-tuning is the right tool
  • Explain what LoRA trains and what it locks
  • Avoid catastrophic forgetting

13 mincovers 5 glossary entries

Worth reading first: Post-Training and RLHF

How to train a model for one job

Fine-tuning means training a finished model a little more, on your own examples, so it gets better at one job. A few hundred to a few thousand good examples is often enough. It changes how the model writes and what it focuses on. It is a poor way to teach it new facts.

Full fine-tuning changes every parameter. For a model with 70 billion of them, that needs a rack of chips and a whole copy of the model for each job you tune it for. That cost is why most teams now use a lighter method.

PEFT trains only a small part of the model. The letters stand for parameter-efficient fine-tuning. The big model stays locked, and a few new pieces are trained beside it. The result is a small add-on file, not a whole new model. One base model can then carry many add-ons, one for each task.

LoRA is the most common PEFT method. Its full name is low-rank adaptation. Next to each big table of weights, it trains two thin ones, and adds their product on top. Its 2021 paper cut the trained numbers for GPT-3 by 10,000 times. It cut the chip memory by 3 times, and the quality held up.

QLoRA goes further. It stores the locked model in a squeezed 4-bit form, about a quarter of the usual size. Then it trains LoRA pieces on top. Its makers tuned a model with 65 billion parameters on one chip. That chip had 48 gigabytes of memory, for a job that used to need many.

Training on a narrow set has a cost. A model tuned hard on legal letters can get worse at things it used to do well, like plain chat or sums. That is catastrophic forgetting. Mixing some general examples into the set, and training gently, keeps it in check. LoRA helps too, since the original weights never change.

Teams often fine-tune when they need the model to know their own files. That rarely works well. The model learns the style of the files, not a true copy of their facts. Putting the right file into the prompt when the question is asked usually does the fact part better, and for less.

Full fine-tuning, LoRA and QLoRA

MethodWhat is trainedWhat you need
Full fine-tuningEvery parameterMany chips, and a full copy per task
LoRATwo thin tables beside each big oneOne strong chip, and a small add-on file
QLoRALoRA on a squeezed 4-bit modelOne chip, even for a big model
LoRA leaves the big grid alone. Only the two thin strips learn, so the add-on stays small.

Worked example

Emma tunes replies for her shop

Emma has 800 past emails her team wrote to customers. She trains a LoRA add-on for a model with 7 billion parameters.

The add-on file is under 100 megabytes, next to about 14 gigabytes for the model. Replies now sound like her team, and the base model is untouched.

The words for this lesson5

Check yourself

What does LoRA train?

Finished this lesson?

Marking it complete also marks its 5 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.