Skip to content
BENIBIX Learn

Model Pruning

AI💻 Running LocallyModel Compression

Also known as Pruning (AI), Neural Network Pruning, Weight Pruning

A way to shrink a model. The parts it needs least are cut out.

Think of it like

Think of it like trimming a bush. You cut the weak twigs, and it still looks like the same bush.

Example

A team prunes a quarter of the layers from a speech model and trains it for one more day. It runs a third faster on phones and scores almost the same.

Why it matters

It shrinks a model in a new way, not like quantization. So the two can be used at once.

Definition

Model pruning makes a model smaller by taking out the parts that matter least. It can take out single weights that are close to zero. Or it can take out whole parts, such as neurons, attention heads or layers. After pruning, the model is often trained a bit more, so it can make up for what was lost. Taking out whole parts is what makes it run faster on normal hardware. Pruning is often used with quantization and distillation.
Before and after pruning. The weakest links are cut, and a smaller network is left.

Common mistake

Cutting single weights and hoping it runs faster. Zeros spread all over save space but often no time, unless whole parts are cut.

Key takeaways

  1. 1
    Cut what matters least. Weights or whole parts.
  2. 2
    Train a little more. To recover.
  3. 3
    Whole parts for speed. Scattered cuts save space.

Compare