# Low-Rank Adaptation (LoRA)

> A cheap way to fine-tune. The big model is locked and small parts are trained.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** LoRA, LoRA Fine-Tuning

## Think of it like

Think of it like you add a small sticker to a big map. The map stays the same.

## Definition

Low-rank adaptation is a cheap way to fine-tune a large model. The model's own weights are locked. Small extra pieces are added next to them, and only those are trained. These pieces are often less than 1 percent of the model's size. The result is a small file that can be swapped in and out on top of the same base model.

## Example

Jack fine-tunes an 8B model with LoRA on one graphics card in an afternoon. The result is a 40 MB file he loads on top of the base model.

## Why it matters

It lets one person with one graphics card fine-tune a model. Before, that took a lab.

## Common mistake

Thinking the file works on any model. It only fits the exact base model it was trained on.

## Three principles

- **Base stays locked.** Its weights do not change.
- **Small add-on pieces.** Often under 1 percent.
- **Swap in and out.** A small file per skill.

## Related terms

- [Fine-Tuning](https://learn.benibix.com/term/ai/fine-tuning)
- [Parameter Count](https://learn.benibix.com/term/ai/parameter-count)
- [Matrix](https://learn.benibix.com/term/general/matrix)
- [Parameter-Efficient Fine-Tuning (PEFT)](https://learn.benibix.com/term/ai/parameter-efficient-fine-tuning)
- [Quantized Low-Rank Adaptation (QLoRA)](https://learn.benibix.com/term/ai/qlora)

## Told apart from

- [LoRA vs QLoRA](https://learn.benibix.com/compare/lora-vs-qlora): People think QLoRA is a different way to tune a model, and it is not. Both lock the model and train two thin tables beside each big one. QLoRA also squeezes the locked model into a 4-bit form first, about a quarter of the usual size. That lets a big model be tuned on one chip, at some cost in speed.

## Taught in

- [Fine-Tuning and LoRA](https://learn.benibix.com/learn/ai-llms/fine-tuning-and-lora): LLMs, 9/15

![A large grid of weights is locked and does not change. Beside it, two thin strips of new weights are trained, and their product is added to the big grid.](https://learn.benibix.com/diagrams/llm-lora.svg)

---

Source: https://learn.benibix.com/term/ai/low-rank-adaptation
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
