# Memory Bandwidth

> How fast data moves from memory to the chip. It often sets AI speed.

**Field:** Infrastructure  
**Category:** Hardware  
**Also known as:** Memory Throughput

## Think of it like

Think of it like the width of a road to a shop. A wide road brings more goods each minute.

## Definition

Memory bandwidth is how much data can move between memory and a chip each second. It is measured in gigabytes per second. A chip that does sums very fast still has to wait if the data comes in slowly. When an AI model writes text, it reads all of its weights for every word, so this speed often sets how fast the words come out. GPU memory is many times faster than normal RAM.

## Example

Lucy's GPU moves about 1,000 GB per second. Her model is 10 GB, so the GPU can read it about 100 times a second. That caps it at about 100 words a second.

## Why it matters

It explains why two chips with the same raw speed can run AI at very different rates.

## Common mistake

Judging an AI chip only by how many sums it can do. If memory cannot feed it, those sums sit idle.

## Three principles

- **Data per second.** From memory to chip.
- **Can be the limit.** Fast chips still wait.
- **Sets text speed.** Weights are read for every word.

## Related terms

- [Video Memory (VRAM)](https://learn.benibix.com/term/infrastructure/video-memory)
- [Random Access Memory (RAM)](https://learn.benibix.com/term/infrastructure/random-access-memory)
- [Bandwidth](https://learn.benibix.com/term/infrastructure/bandwidth)
- [Throughput](https://learn.benibix.com/term/infrastructure/throughput)
- [Unified Memory](https://learn.benibix.com/term/infrastructure/unified-memory)

## Told apart from

- [Memory Bandwidth vs Network Bandwidth](https://learn.benibix.com/compare/memory-bandwidth-vs-bandwidth): People often hear bandwidth and think of internet speed. Network bandwidth is how much data moves over a link between machines. Memory bandwidth is how much moves inside one machine, between memory and a chip. The second is often thousands of times bigger. It is the one that sets how fast AI writes.

## Taught in

- [RAM and Memory Bandwidth](https://learn.benibix.com/learn/infrastructure-hardware/ram-and-memory-bandwidth): Computer Hardware, 3/8

![A CPU joined to RAM by a thin pipe, and a GPU joined to its video memory by a thick pipe. The pipe width is the memory bandwidth.](https://learn.benibix.com/diagrams/hw-memory.svg)

---

Source: https://learn.benibix.com/term/infrastructure/memory-bandwidth
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
