💻Quantization (AI)
Also known as Model Quantization, LLM Quantization, 4-bit Quantization
A way to shrink a model. Each weight is kept with fewer bits.
Think of it like
Think of it like rounding prices to the nearest dollar. The list is shorter, and you lose just a bit.
Example
Lucy's laptop has 16 GB of memory. The model she wants is 16 GB at full size, so she downloads a 4-bit version of about 5 GB, and it runs well.
Why it matters
It is the main reason good models fit on normal laptops and phones at all.
Definition
Common mistake
Key takeaways
- 1Fewer bits per weight. 16 down to 4.
- 2Smaller and faster. Less to read.
- 3Small loss at first. Big loss if pushed.