🔧Memory Bandwidth
Also known as Memory Throughput
How fast data moves from memory to the chip. It often sets AI speed.
Think of it like
Think of it like the width of a road to a shop. A wide road brings more goods each minute.
Example
Lucy's GPU moves about 1,000 GB per second. Her model is 10 GB, so the GPU can read it about 100 times a second. That caps it at about 100 words a second.
Why it matters
It explains why two chips with the same raw speed can run AI at very different rates.
Definition
Common mistake
Key takeaways
- 1Data per second. From memory to chip.
- 2Can be the limit. Fast chips still wait.
- 3Sets text speed. Weights are read for every word.