# Quantized Low-Rank Adaptation (QLoRA)

> LoRA on a squeezed 4-bit copy of the model. Now it fits on one card.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** QLoRA, Quantized LoRA

## Think of it like

It is like packing a big coat in a small bag. Then you sew a patch on it.

## Definition

QLoRA is a way to fine-tune a large model on a single graphics card. It first squeezes the frozen model down to 4-bit numbers, so it takes far less memory. Then it trains small LoRA add-ons on top, in the usual way. The paper that brought it in tuned a model with 65 billion weights on one card with 48 GB of memory. The results matched full 16-bit tuning on its tests.

## Example

Emma wants to tune a large open model, but she has one 48 GB card. With QLoRA the squeezed model fits, and only the small add-ons are trained.

## Why it matters

Now people can tune big models at home, with no data centre.

## Common mistake

Thinking the squeezed model is what you ship. The 4-bit copy is for training, and you pick the format you serve later.

## Three principles

- **Squeeze the base.** Frozen weights in 4-bit.
- **Train LoRA on top.** The same small add-ons.
- **One card.** Big models fit in less memory.

## Related terms

- [Low-Rank Adaptation (LoRA)](https://learn.benibix.com/term/ai/low-rank-adaptation)
- [Parameter-Efficient Fine-Tuning (PEFT)](https://learn.benibix.com/term/ai/parameter-efficient-fine-tuning)
- [Quantization (AI)](https://learn.benibix.com/term/ai/quantization)
- [Fine-Tuning](https://learn.benibix.com/term/ai/fine-tuning)
- [Parameter Count](https://learn.benibix.com/term/ai/parameter-count)

## Told apart from

- [LoRA vs QLoRA](https://learn.benibix.com/compare/lora-vs-qlora): People think QLoRA is a different way to tune a model, and it is not. Both lock the model and train two thin tables beside each big one. QLoRA also squeezes the locked model into a 4-bit form first, about a quarter of the usual size. That lets a big model be tuned on one chip, at some cost in speed.

## Taught in

- [Fine-Tuning and LoRA](https://learn.benibix.com/learn/ai-llms/fine-tuning-and-lora): LLMs, 9/15

![Three panels comparing how much of a model changes. Full fine-tuning: every weight changes, which takes the most memory and many big GPUs. PEFT, such as LoRA: the weights are frozen and only a small highlighted add-on learns, with far less memory. QLoRA: the frozen weights are squeezed to 4-bit and the same add-on trains on top, so it fits on one GPU.](https://learn.benibix.com/diagrams/ai2-peft.svg)

---

Source: https://learn.benibix.com/term/ai/qlora
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
