💬Low-Rank Adaptation (LoRA)
Also known as LoRA, LoRA Fine-Tuning
A cheap way to fine-tune. The big model is locked and small parts are trained.
Think of it like
Think of it like you add a small sticker to a big map. The map stays the same.
Example
Jack fine-tunes an 8B model with LoRA on one graphics card in an afternoon. The result is a 40 MB file he loads on top of the base model.
Why it matters
It lets one person with one graphics card fine-tune a model. Before, that took a lab.
Definition
Common mistake
Key takeaways
- 1Base stays locked. Its weights do not change.
- 2Small add-on pieces. Often under 1 percent.
- 3Swap in and out. A small file per skill.