CPU vs GPU vs TPU
Level 2 · Graphics cards and AI chips
Why a GPU has thousands of small cores and a CPU has a few big ones. What a TPU or other AI chip adds, and where CUDA fits in.
Course contentsLesson 4 of 8
Level 1 · The board and the processor
Level 2 · Graphics cards and AI chips
Level 3 · Drives and how fast they are
Level 4 · Power and start-up
After this lesson you can
- Say why a GPU suits AI work
- Tell a GPU from an AI accelerator
- Explain why CUDA ties AI code to NVIDIA
12 mincovers 4 glossary entries
Worth reading first: CPU Cores and Clock Speed
Why AI runs on GPUs and not CPUs
A CPU has a few big cores that can each do any job. A GPU, or graphics processing unit, has thousands of small cores that all do the same simple sum at once. One home graphics card has over 16,000 of them. AI work is millions of the same sum, so it runs far faster on a GPU.
GPUs were built to draw games. A screen has millions of dots, and each one needs the same kind of sum every frame. That is the same shape of work as AI, where huge grids of numbers are multiplied together. So the chips made for games became the chips for AI.
The CPU is still in charge. It runs the system, reads files and hands the heavy maths to the GPU. The GPU does the sums and hands the answers back. A big AI model is trained on thousands of GPUs, linked together, for weeks.
An AI accelerator goes one step further. It drops the parts a graphics chip needs for games and packs in more units for multiplying grids. That makes it faster for AI and uses less power. Some sit in large data centres. Smaller ones, often called NPUs, sit in phones and laptops.
The TPU, or tensor processing unit, is Google's own AI accelerator. Google uses it to train and run its own models, and rents it out through its cloud. Many TPUs are linked into large groups, called pods, to train the biggest models.
CUDA is NVIDIA's way to run any program on its GPUs. NVIDIA first released it in 2007. Most AI tools use it under the hood, so most AI code is written for NVIDIA chips. Other chip makers have their own versions, but far less code runs on them. That is a big part of why NVIDIA leads this market.
People often think a GPU is better than a CPU at everything. It is not. A GPU is slow at work with many steps that each depend on the last. A chain of choices, like serving web pages or running a database, still suits the CPU best.
CPU, GPU and AI accelerator side by side
| Chip | Cores | Best at | Example |
|---|---|---|---|
| CPU | A few to over 100 big ones | Any task, step after step | Running the system and apps |
| GPU | Thousands of small ones | The same sum on lots of data | Games and training AI |
| AI accelerator | Large blocks of grid units | Only AI maths, at low power | A TPU, or an NPU in a phone |
Which chip fits the job
- 1
Pick the CPU
When the work is a chain of steps, such as serving web pages, a database or office apps.
- 2
Pick a GPU
When you train or run AI models yourself and need the tools that work everywhere.
- 3
Pick an AI accelerator
When you rent AI time in a cloud that has one. Or when a phone must run a small model on battery.
Worked example
A start-up trains an image model
The words for this lesson4
Check yourself
Why do AI models train on GPUs instead of CPUs?
Finished this lesson?
Marking it complete also marks its 4 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.