# Parameter-Efficient Fine-Tuning (PEFT)

> A way to tune a model. Only a small add-on learns, and the rest stays the same.

**Field:** Artificial Intelligence  
**Category:** LLMs  
**Also known as:** PEFT, Efficient Fine-Tuning

## Think of it like

It is like a sticky note on a book. You add the note, not a new book.

## Definition

Parameter-efficient fine-tuning is a set of ways to tune a model by training only a small part of it. The model's own weights stay frozen. A small add-on, often under one percent of the size, learns the new task. That needs far less memory than changing every weight. The add-on is a small file, so you can keep one base model and swap add-ons for each job. LoRA is the best-known one.

## Example

Jack wants a model that writes in his shop's style. He trains a small add-on on one graphics card, and keeps a different add-on for his support replies.

## Why it matters

It lets people tune large models on one graphics card, not a room full of them.

## Common mistake

Expecting it to teach many new facts. It is best at style, format and narrow tasks.

## Three principles

- **Freeze the model.** Its own weights do not change.
- **Train a small add-on.** A tiny part learns the task.
- **Swap add-ons.** One base, many small files.

## Related terms

- [Fine-Tuning](https://learn.benibix.com/term/ai/fine-tuning)
- [Low-Rank Adaptation (LoRA)](https://learn.benibix.com/term/ai/low-rank-adaptation)
- [Quantized Low-Rank Adaptation (QLoRA)](https://learn.benibix.com/term/ai/qlora)
- [Instruction Tuning](https://learn.benibix.com/term/ai/instruction-tuning)
- [Parameter Count](https://learn.benibix.com/term/ai/parameter-count)

## Taught in

- [Fine-Tuning and LoRA](https://learn.benibix.com/learn/ai-llms/fine-tuning-and-lora): LLMs, 9/15

![Three panels comparing how much of a model changes. Full fine-tuning: every weight changes, which takes the most memory and many big GPUs. PEFT, such as LoRA: the weights are frozen and only a small highlighted add-on learns, with far less memory. QLoRA: the frozen weights are squeezed to 4-bit and the same add-on trains on top, so it fits on one GPU.](https://learn.benibix.com/diagrams/ai2-peft.svg)

---

Source: https://learn.benibix.com/term/ai/parameter-efficient-fine-tuning
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-18
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
