Spark LampWhy Does AI Run on GPUs?
Why AI needs so much simple arithmetic, how GPUs and tensor cores do it in parallel, why moving data is the real bottleneck, and what it takes to run and train a model.
Modules
Each module explains one idea in plain words, lets you try it, then checks what stuck.
- 01
Why AI Is So Computationally Expensive
See why AI needs an enormous amount of very simple maths, and how to estimate exactly how much.
- 02
CPUs vs GPUs: A Few Experts vs an Army of Workers
Understand the different design ideas behind a CPU and a GPU, and when each one wins.
- 03
The Math Behind AI: Matrix Multiplication
See that a whole layer of neurons is one matrix multiplication, the operation GPUs are built around.
- 04
Parallel Computing: Thousands of Calculations at Once
Learn what makes work parallel, how even a sum can be split up, and the limit no number of workers can beat.
- 05
Why Moving Data Can Be Slower Than Computing
Discover that a GPU is often waiting for data, not doing maths, and what decides which one limits it.
- 06
Inside GPU Memory: Registers, Cache and VRAM
Learn the layers of memory in a GPU, and why “close and small” beats “far and big”.
- 07
How Thousands of GPU Threads Work Together
Understand how work is organised on a GPU: threads, warps, blocks and SMs, and the two ways to waste them.
- 08
Tiling: The Trick That Makes Matrix Multiplication Faster
Learn the most important trick in GPU programming: load a piece of data once, then reuse it many times.
- 09
Why AI Uses Different Number Precisions
See how storing numbers with fewer bits saves memory and gains speed, and what each bit is for.
- 10
Tensor Cores: Hardware Built for AI
See how tensor cores multiply whole blocks of numbers in one step, and why smaller numbers make them faster.
- 11
How Much GPU Memory Does an AI Model Need?
Put it all together: estimate the memory, the number of GPUs and the speed needed to run a real model.
- 12
Why Training Needs Far More Hardware Than Running a Model
See why training a model takes vastly more memory and compute than running it, and how clusters make it possible.