# Model Memory Requirements

> How much memory a model needs to run. Weights first, then room for the chat.

**Field:** Artificial Intelligence  
**Category:** Running Locally  
**Also known as:** VRAM Requirements, LLM Memory Requirements, GPU Memory Requirements

## Think of it like

It is like packing a suitcase. It all has to fit, or the lid will not shut.

## Definition

Model memory requirements are how much memory a model needs to run. The biggest part is the weights: the count of weights times the bytes each one takes. An 8 billion weight model at 16 bits needs about 16 GB just for that. Shrunk to about 5 bits a weight, it needs about 5 GB. On top of that comes room for the chat so far, which grows as the chat gets longer. If it does not all fit in the graphics card's memory, part spills into slower memory and it slows down a lot.

## Example

Emma's graphics card has 8 GB of memory. An 8 billion weight model at 16 bits needs about 16 GB, so it will not fit. The same model shrunk to Q4_K_M is about 5 GB, and it runs fast.

## Why it matters

It tells you before you download if a model will run on your machine, and how fast.

## Common mistake

Counting only the file size. A long chat needs more room on top, so leave some memory spare.

## Three principles

- **Weights first.** Count times the bytes for each.
- **The chat needs room.** Long chats take more memory.
- **Fit or crawl.** Spilling over slows it down a lot.

## Related terms

- [Model Weights File](https://learn.benibix.com/term/ai/model-weights)
- [Parameter Count](https://learn.benibix.com/term/ai/parameter-count)
- [Quantization (AI)](https://learn.benibix.com/term/ai/quantization)
- [KV Cache](https://learn.benibix.com/term/ai/kv-cache)
- [CPU Offloading](https://learn.benibix.com/term/ai/cpu-offloading)
- [Context Window](https://learn.benibix.com/term/ai/context-window)
- [Video Memory (VRAM)](https://learn.benibix.com/term/infrastructure/video-memory)

![A tall bar for the memory one model needs, made of three stacked parts: the weights at the bottom, the largest; room for the chat, which grows with a long chat; and a small part for the program itself. A dashed line across it marks the memory on the graphics card. Below the line the model runs fast; the part above it spills into slower memory.](https://learn.benibix.com/diagrams/ai4-memory-budget.svg)

---

Source: https://learn.benibix.com/term/ai/model-memory-requirements
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
