Skip to content
BENIBIX Learn

LoRA vs QLoRA

People think QLoRA is a different way to tune a model, and it is not. Both lock the model and train two thin tables beside each big one. QLoRA also squeezes the locked model into a 4-bit form first, about a quarter of the usual size. That lets a big model be tuned on one chip, at some cost in speed.

AI💬 LLMs

Low-Rank Adaptation (LoRA)

A cheap way to fine-tune. The big model is locked and small parts are trained.

Low-rank adaptation is a cheap way to fine-tune a large model. The model's own weights are locked. Small extra pieces are added next to them, and only those are trained. These pieces are often less than 1 percent of the model's size. The result is a small file that can be swapped in and out on top of the same base model.

Example
Jack fine-tunes an 8B model with LoRA on one graphics card in an afternoon. The result is a 40 MB file he loads on top of the base model.
Why it matters
It lets one person with one graphics card fine-tune a model. Before, that took a lab.
Full entry
AI💬 LLMs

Quantized Low-Rank Adaptation (QLoRA)

LoRA on a squeezed 4-bit copy of the model. Now it fits on one card.

QLoRA is a way to fine-tune a large model on a single graphics card. It first squeezes the frozen model down to 4-bit numbers, so it takes far less memory. Then it trains small LoRA add-ons on top, in the usual way. The paper that brought it in tuned a model with 65 billion weights on one card with 48 GB of memory. The results matched full 16-bit tuning on its tests.

Example
Emma wants to tune a large open model, but she has one 48 GB card. With QLoRA the squeezed model fits, and only the small add-ons are trained.
Why it matters
Now people can tune big models at home, with no data centre.
Full entry

In one line

Low-Rank Adaptation (LoRA)
A cheap way to fine-tune. The big model is locked and small parts are trained.
Quantized Low-Rank Adaptation (QLoRA)
LoRA on a squeezed 4-bit copy of the model. Now it fits on one card.