How Does ChatGPT Actually Work?
How a chatbot works inside: tokens, meaning, attention and the transformer, how models are trained and scaled, and why they hallucinate.
Modules
Each module explains one idea in plain words, lets you try it, then checks what stuck.
- 01
The Secret Behind LLMs: Predicting the Next Token
Understand the one simple job every LLM does, over and over: guess the next token.
- 02
How AI Breaks Language Into Tokens
See how text becomes the numbers a model can work with, and why words get cut into pieces.
- 03
From Words to Numbers: Understanding Embeddings
See how each token becomes a list of numbers, and how those numbers come to capture meaning.
- 04
How AI Represents Meaning as Vectors
Understand how similarity and meaning become geometry: angles and directions between vectors.
- 05
Why Word Order Matters
See why a pile of word vectors loses who-did-what-to-whom, and how models stamp tokens with their position.
- 06
Why Transformers Replaced Older Language Models
Understand what held older language models back, and the two problems the transformer solved.
- 07
Attention: How AI Connects Words Across a Sentence
Understand attention: how each token gathers information from the tokens that matter for it.
- 08
Multi-Head Attention: Looking at Language From Different Angles
See why transformers run many attentions side by side, each free to look for a different kind of link.
- 09
Inside the Transformer: The Engine Behind Modern LLMs
See the whole machine: how tokens flow up a tall stack of identical blocks and come out as predictions.
- 10
How AI Generates an Answer, One Token at a Time
Follow one question all the way from your keyboard to a finished answer, pass by pass.
- 11
Why the Same Prompt Can Produce Different Answers
Learn how the model turns probabilities into one actual choice, and what temperature and top-p really do.
- 12
How LLMs Learn From the Internet and Other Data
See where an LLM’s knowledge comes from: what pre-training data is, how it’s cleaned, and what it produces.
- 13
Scaling Laws: Why Bigger Models Get Smarter, Until They Don't
Understand why bigger models got predictably better, how to split a budget between size and data, and where it stops.
- 14
Fine-Tuning and RLHF: Teaching AI to Follow Instructions
Understand the training stages that turn a text predictor into a helpful assistant.
- 15
LoRA: How to Fine-Tune AI Without Retraining Everything
See how LoRA fine-tunes a huge model by training a tiny add-on instead of every weight.
- 16
Reasoning Models: Why Some AI Systems Think Before Answering
Understand why writing out steps makes models smarter, and what reasoning models are trained to do.
- 17
Context Windows: How Much Can an LLM Read at Once?
Understand what a context window is, what fills it, and what happens when a conversation outgrows it.
- 18
Why Bigger Context Windows Don't Always Mean Better Memory
See why a model that can read a huge input doesn’t necessarily use all of it well.
- 19
Why AI Hallucinates
Understand why LLMs state false things confidently, and what actually helps.
- 20
Mixture of Experts: How AI Activates Only Part of a Model
See how mixture-of-experts models store many parameters but use only a few of them for each token.
- 21
Can We Make an AI Model Smaller Without Ruining It?
See how models are shrunk to fewer bits per weight, and the trick that keeps them accurate.