Skip to content
BENIBIX Learn

CPU vs GPU vs TPU

Level 2 · Graphics cards and AI chips

Why a GPU has thousands of small cores and a CPU has a few big ones. What a TPU or other AI chip adds, and where CUDA fits in.

Course contentsLesson 4 of 8

Level 2 · Graphics cards and AI chips

  1. CPU vs GPU vs TPU
  2. VRAM and Unified Memory

Level 3 · Drives and how fast they are

  1. HDD vs SSD vs NVMe
  2. IOPS and Drive Speed

Level 4 · Power and start-up

  1. Power Supply and UEFI

After this lesson you can

  • Say why a GPU suits AI work
  • Tell a GPU from an AI accelerator
  • Explain why CUDA ties AI code to NVIDIA

12 mincovers 4 glossary entries

Worth reading first: CPU Cores and Clock Speed

Why AI runs on GPUs and not CPUs

A CPU has a few big cores that can each do any job. A GPU, or graphics processing unit, has thousands of small cores that all do the same simple sum at once. One home graphics card has over 16,000 of them. AI work is millions of the same sum, so it runs far faster on a GPU.

GPUs were built to draw games. A screen has millions of dots, and each one needs the same kind of sum every frame. That is the same shape of work as AI, where huge grids of numbers are multiplied together. So the chips made for games became the chips for AI.

The CPU is still in charge. It runs the system, reads files and hands the heavy maths to the GPU. The GPU does the sums and hands the answers back. A big AI model is trained on thousands of GPUs, linked together, for weeks.

An AI accelerator goes one step further. It drops the parts a graphics chip needs for games and packs in more units for multiplying grids. That makes it faster for AI and uses less power. Some sit in large data centres. Smaller ones, often called NPUs, sit in phones and laptops.

The TPU, or tensor processing unit, is Google's own AI accelerator. Google uses it to train and run its own models, and rents it out through its cloud. Many TPUs are linked into large groups, called pods, to train the biggest models.

CUDA is NVIDIA's way to run any program on its GPUs. NVIDIA first released it in 2007. Most AI tools use it under the hood, so most AI code is written for NVIDIA chips. Other chip makers have their own versions, but far less code runs on them. That is a big part of why NVIDIA leads this market.

People often think a GPU is better than a CPU at everything. It is not. A GPU is slow at work with many steps that each depend on the last. A chain of choices, like serving web pages or running a database, still suits the CPU best.

CPU, GPU and AI accelerator side by side

ChipCoresBest atExample
CPUA few to over 100 big onesAny task, step after stepRunning the system and apps
GPUThousands of small onesThe same sum on lots of dataGames and training AI
AI acceleratorLarge blocks of grid unitsOnly AI maths, at low powerA TPU, or an NPU in a phone

Which chip fits the job

  1. 1

    Pick the CPU

    When the work is a chain of steps, such as serving web pages, a database or office apps.

  2. 2

    Pick a GPU

    When you train or run AI models yourself and need the tools that work everywhere.

  3. 3

    Pick an AI accelerator

    When you rent AI time in a cloud that has one. Or when a phone must run a small model on battery.

Few big cores, thousands of small ones, or one block made for a single kind of sum.

Worked example

A start-up trains an image model

A team of four trains a model that spots faulty parts in photos. On a server CPU with 32 cores, one pass over their 200,000 photos takes 9 hours.

On one rented GPU, the same pass takes 25 minutes. Their code uses CUDA, so they rent NVIDIA cards rather than switch tools.

The words for this lesson4

Check yourself

Why do AI models train on GPUs instead of CPUs?

Finished this lesson?

Marking it complete also marks its 4 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.