# CPU vs GPU vs TPU

> Why a GPU has thousands of small cores and a CPU has a few big ones. What a TPU or other AI chip adds, and where CUDA fits in.

**Course:** [Computer Hardware](https://learn.benibix.com/learn/infrastructure-hardware)  
**Time:** 12 min

**After this lesson you can:**

- Say why a GPU suits AI work
- Tell a GPU from an AI accelerator
- Explain why CUDA ties AI code to NVIDIA

## Why AI runs on GPUs and not CPUs

A CPU has a few big cores that can each do any job. A GPU, or graphics processing unit, has thousands of small cores that all do the same simple sum at once. One home graphics card has over 16,000 of them. AI work is millions of the same sum, so it runs far faster on a GPU.

GPUs were built to draw games. A screen has millions of dots, and each one needs the same kind of sum every frame. That is the same shape of work as AI, where huge grids of numbers are multiplied together. So the chips made for games became the chips for AI.

The CPU is still in charge. It runs the system, reads files and hands the heavy maths to the GPU. The GPU does the sums and hands the answers back. A big AI model is trained on thousands of GPUs, linked together, for weeks.

An AI accelerator goes one step further. It drops the parts a graphics chip needs for games and packs in more units for multiplying grids. That makes it faster for AI and uses less power. Some sit in large data centres. Smaller ones, often called NPUs, sit in phones and laptops.

The TPU, or tensor processing unit, is Google's own AI accelerator. Google uses it to train and run its own models, and rents it out through its cloud. Many TPUs are linked into large groups, called pods, to train the biggest models.

CUDA is NVIDIA's way to run any program on its GPUs. NVIDIA first released it in 2007. Most AI tools use it under the hood, so most AI code is written for NVIDIA chips. Other chip makers have their own versions, but far less code runs on them. That is a big part of why NVIDIA leads this market.

People often think a GPU is better than a CPU at everything. It is not. A GPU is slow at work with many steps that each depend on the last. A chain of choices, like serving web pages or running a database, still suits the CPU best.

## CPU, GPU and AI accelerator side by side

| Chip | Cores | Best at | Example |
| --- | --- | --- | --- |
| CPU | A few to over 100 big ones | Any task, step after step | Running the system and apps |
| GPU | Thousands of small ones | The same sum on lots of data | Games and training AI |
| AI accelerator | Large blocks of grid units | Only AI maths, at low power | A TPU, or an NPU in a phone |

## Which chip fits the job

1. **Pick the CPU.** When the work is a chain of steps, such as serving web pages, a database or office apps.
2. **Pick a GPU.** When you train or run AI models yourself and need the tools that work everywhere.
3. **Pick an AI accelerator.** When you rent AI time in a cloud that has one. Or when a phone must run a small model on battery.

![Three chips drawn as grids. The CPU has a few large cores. The GPU has thousands of small cores. The AI accelerator is mostly one big block of units that multiply grids of numbers.](https://learn.benibix.com/diagrams/hw-processors.svg)

### Example: A start-up trains an image model

A team of four trains a model that spots faulty parts in photos. On a server CPU with 32 cores, one pass over their 200,000 photos takes 9 hours.

On one rented GPU, the same pass takes 25 minutes. Their code uses CUDA, so they rent NVIDIA cards rather than switch tools.

**Words in this lesson:** [Graphics Processing Unit (GPU)](https://learn.benibix.com/term/infrastructure/graphics-processing-unit), [AI Accelerator](https://learn.benibix.com/term/infrastructure/ai-accelerator), [Tensor Processing Unit (TPU)](https://learn.benibix.com/term/infrastructure/tensor-processing-unit), [NVIDIA CUDA](https://learn.benibix.com/term/infrastructure/cuda)

> **The one difference.** A CPU does any job, one step after another. A GPU does one small sum on thousands of numbers at once. An AI chip does only that.

**Question:** Why do AI models train on GPUs instead of CPUs?

**Answer:** GPUs can do the same sum on thousands of numbers at once. AI work is millions of the same small sum. A GPU has thousands of small cores that do those sums side by side, while a CPU has a few cores made for steps in a row.

---

Source: https://learn.benibix.com/learn/infrastructure-hardware/cpu-vs-gpu-vs-tpu
Author: Behnia Zarei (a developer in Vienna, Austria, who builds and teaches data, AI and business intelligence at BENIBIX)
Publisher: BENIBIX Learn (https://learn.benibix.com/)
Last updated: 2026-11-19
Licence: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Quote freely with attribution.
