Skip to content
BENIBIX Learn

VRAM and Unified Memory

Level 2 · Graphics cards and AI chips

What video memory is and why an AI model has to fit in it. How unified memory lets the CPU and GPU share one pool instead of copying.

Course contentsLesson 5 of 8

Level 2 · Graphics cards and AI chips

  1. CPU vs GPU vs TPU
  2. VRAM and Unified Memory

Level 3 · Drives and how fast they are

  1. HDD vs SSD vs NVMe
  2. IOPS and Drive Speed

Level 4 · Power and start-up

  1. Power Supply and UEFI

After this lesson you can

  • Work out if a model fits in VRAM
  • Explain what unified memory removes
  • Say what each kind costs you

11 mincovers 2 glossary entries

Worth reading first: RAM and Memory Bandwidth, CPU vs GPU vs TPU

Why the model has to fit in the GPU's memory

VRAM, or video memory, is the fast memory on a graphics card that only the GPU uses. It sits right beside the GPU chip. To run an AI model on a GPU, the model's numbers usually have to fit in it. Home cards carry 8 to 32 gigabytes. Data centre cards carry 80 or more.

The sum is simple. A model stores each of its numbers in 2 bytes at the usual size. So a model with 7 billion numbers needs about 14 gigabytes, plus some room to work. A card with 12 gigabytes cannot hold it, and a card with 16 or 24 can.

In most computers, the CPU has its RAM and the GPU has its VRAM. Data the GPU needs is first copied across from RAM. That copy takes time. And the VRAM on a card is fixed: you cannot add a stick to it later, the way you can with RAM.

Unified memory takes the two pools away and leaves one. The CPU and GPU sit on one chip and read the same bytes, so nothing is copied. The memory sits right beside the chip and is fast. Some desktop machines built this way carry 128 gigabytes or more.

That makes unified memory a cheap way to run large models at home. A model too big for any home graphics card can load into a big pool of shared memory. It runs slower than on a top data centre card, but it runs at all.

Each kind has a cost. A graphics card with its own VRAM is fast and can be swapped for a better one. Unified memory cannot be changed after you buy the machine. So you choose its size on the day, and you pay for that choice for years.

People often look at RAM when they should look at VRAM. A desktop with 64 gigabytes of RAM and a card with 8 gigabytes of VRAM still cannot run a 14 gigabyte model on the GPU. It can only run it on the CPU, far more slowly.

Two pools with a copy between them, or one pool both chips share.

Worked example

Jack runs a model at home

Jack wants to run a model with 13 billion numbers. At 2 bytes each that is 26 gigabytes. His graphics card has 12 gigabytes of VRAM, so it does not fit.

His friend's desktop has 64 gigabytes of unified memory. The model loads there with room to spare and writes about 10 words a second.

The words for this lesson2

Check yourself

How much VRAM does a model with 7 billion numbers need, at 2 bytes each?

Finished this lesson?

Marking it complete also marks its 2 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.