💻Model Precision
Also known as FP16, F16, BF16, Bfloat16, FP32, Half Precision, Floating-Point Precision
How many bits each weight in a model gets. Fewer bits make a smaller model.
Think of it like
It is like writing a price as 3.14159 or just 3.14. The short one takes less room.
Example
A model page lists the weights as BF16. Jack knows that means 2 bytes a weight. So a 7 billion weight model is about 14 GB.
Why it matters
It sets the file size. It also sets the memory a model needs before any shrinking.
Definition
Common mistake
Key takeaways
- 1Bits per weight. Size grows with the bits.
- 216 is normal. Half the size of FP32.
- 3Below 16. That step is called quantization.