Skip to content
BENIBIX Learn

Video Memory (VRAM)

Infrastructure🔧 HardwareMemory

Also known as VRAM, GPU Memory, Graphics Memory

The GPU's own fast memory. A model must fit in it to run on the GPU.

Think of it like

Think of it like the GPU has its own desk. It is right next to it, so it does not walk far.

Example

Emma's graphics card has 12 GB of video memory. A model with 7 billion weights at 2 bytes each needs about 14 GB, so it does not fit.

Why it matters

To run AI at home, video memory is most often the limit. It decides the biggest model you can load.

Definition

Video memory is the fast memory on a graphics card that only the GPU uses. It sits right next to the GPU chip. For the GPU, it is much faster than normal RAM. To run an AI model on a GPU, the model's weights usually have to fit in its video memory. Home graphics cards often have 8 to 32 gigabytes, and data centre GPUs have 80 or more.
Video memory sits on the graphics card. Its pipe to the GPU is much wider than the one to RAM.

Common mistake

Counting only the weights. Running a model also needs room for the text it is working on, which can take several more gigabytes.

Key takeaways

  1. 1
    On the graphics card. Right beside the GPU.
  2. 2
    Very fast. Quicker for the GPU than RAM.
  3. 3
    The model must fit. Or it will not run there.

Compare

Learn it properly