Skip to content
BENIBIX Learn

NVIDIA CUDA

Infrastructure🔧 HardwareProcessors

Also known as CUDA, CUDA Toolkit, Compute Unified Device Architecture

NVIDIA's way to run any program on its GPUs. Most AI training uses it.

Think of it like

It is like one voice that tells a thousand cooks what to do.

Example

Jack installs PyTorch on a computer with an NVIDIA card. One line checks that PyTorch can see CUDA, so his model will train on the GPU.

Why it matters

So much AI code is written for CUDA that NVIDIA's chips are hard to replace. This is a big reason NVIDIA leads in AI chips.

Definition

CUDA is NVIDIA's platform for running general programs on its GPUs. NVIDIA first released it in 2007. It lets code split one big job into thousands of small pieces, and each runs on its own GPU core. Most AI tools, such as PyTorch and TensorFlow, use CUDA under the hood. It only works on NVIDIA chips. Other chip makers have their own versions, but far less AI code is written for them.

Example code

console
$ python -c "import torch; print(torch.cuda.is_available())"
True
PyTorch asks if CUDA and a GPU are ready. True means the work can run on the GPU.
Your code asks PyTorch. PyTorch asks CUDA, and CUDA hands the work to thousands of GPU cores.

Common mistake

Expecting CUDA code to run on any GPU. It needs an NVIDIA card, and a driver new enough for its CUDA version.

Key takeaways

  1. 1
    Programs on the GPU. Not just graphics any more.
  2. 2
    NVIDIA only. Other makers' chips need other tools.
  3. 3
    AI's base layer. PyTorch and TensorFlow build on it.

Learn it properly