Learn Large Language Models (LLMs)
What is inside the model behind a chat app, and why it writes the way it does. Fifteen lessons start with the numbers inside a model. Then come tokens, the context window, and how a model is trained and tuned. The last ones cover how to pick a model, why some think before they answer, and the three ways they go wrong. It assumes nothing. It stops at the model. Writing prompts and building apps on one are separate subjects.
Not started
0 of 15 lessons done
Level 1 · What an LLM is
score0250/22 wordsThe numbers inside a model, and the tokens it reads. How it picks each one, and how much it can see at once.
- Large Language Model BasicsWhat a large language model is, and what its billions of numbers do. Why a bigger model needs far more memory to run.10 min4 terms
- Tokens and TokenizersWhat a token is, and how a tokenizer cuts text into numbered pieces. Why a model writes its reply one token at a time.10 min3 terms
- Temperature and SamplingHow a model picks each token from a list of chances. What temperature, top-k, top-p and beam search change about that pick.12 min8 terms
- The LLM Context WindowHow much text a model can see at once, and how a chat is sent as messages with roles. What happens when the space runs out.12 min7 terms
Level 2 · Inside the model
score25400/11 wordsThe design that almost every LLM uses. How each token finds the other words it needs.
- Transformer ArchitectureThe design inside almost every LLM. How it turns tokens into lists of numbers, keeps word order, and passes them up through its layers.13 min8 terms
- Self-Attention and Multi-HeadHow each token pulls meaning from the others. Why long texts cost so much, and what many heads and faster maths change.13 min3 terms
Level 3 · How a model is trained
score40650/23 wordsReading trillions of tokens, then learning to follow requests. Tuning a model for one job, and copying a big one into a small one.
- Pretraining an LLMHow a model learns from trillions of tokens of text. What scaling laws predict, and why the result does not follow requests yet.12 min5 terms
- Post-Training and RLHFHow a base model becomes a helper that does what you ask. Example answers, people's picks, a model that scores, and answers a program can check.14 min8 terms
- Fine-Tuning and LoRAHow to train a finished model for one job. Full fine-tuning against LoRA and QLoRA, and why tuning is a poor way to add facts.13 min5 terms
- Knowledge DistillationHow a big model teaches a small one, and what small models are good at. The risk of training a model on text that models wrote.12 min5 terms
Level 4 · Choosing and running a model
score65850/13 wordsOpen or closed, thinking or quick, and what makes a reply fast. The choices behind every app built on a model.
- Open-Weight vs Closed ModelsThe difference between a model you download and one you reach online. What a frontier model and a dated snapshot mean when you pick one.12 min5 terms
- Reasoning Models and ThinkingWhy some models write out their working before they answer, and what that costs. Which jobs it helps, and which it only slows down.11 min4 terms
- LLM Speed and StreamingThe two numbers that decide how fast a model feels. How streaming, the KV cache and speculative decoding each cut the wait.12 min4 terms
Level 5 · Where LLMs go wrong
score851000/5 wordsMade-up facts, out-of-date facts, eager agreement, and losing track in a long input. How to spot each one, and what to do.
- LLM HallucinationsWhy a model states false facts in a calm, sure tone, and where it happens most. How old facts and eager agreement add to it.12 min3 terms
- Long Context ProblemsWhy a model misses facts in the middle of a long input, and why answers get worse as it grows. How to send less, in a better order.10 min2 terms
Words in this course
The 74 glossary entries this course teaches, in the order it teaches them. Open any of them on its own.
- Large Language Model (LLM)
- Model Parameter
- Parameter Count
- Foundation Model
- Token (AI)
- Tokenizer
- Next-Token Prediction
- LLM Sampling
- Greedy Decoding
- LLM Temperature
- Top-k Sampling
- Top-p Sampling
- Beam Search
- LLM Non-Determinism
- Log Probabilities (Logprobs)
- Context Window
- Prompt (AI)
- Message Roles
- System Prompt
- Max Tokens
- Stop Sequence
- Context Compaction
- Transformer
- Positional Encoding
- Attention Mechanism
- Encoder (Transformer)
- Decoder (Transformer)
- Mixture of Experts (MoE)
- Gating Network
- Mixtral
- Self-Attention
- Multi-Head Attention
- FlashAttention
- Pretraining
- Training Data
- Scaling Laws
- Chinchilla Scaling Laws
- Emergent Abilities
- Post-Training
- Base Model
- Instruct Model
- Instruction Tuning
- Reinforcement Learning from Human Feedback (RLHF)
- Reward Model
- Direct Preference Optimization (DPO)
- RL with Verifiable Rewards (RLVR)
- Fine-Tuning
- Parameter-Efficient Fine-Tuning (PEFT)
- Low-Rank Adaptation (LoRA)
- Quantized Low-Rank Adaptation (QLoRA)
- Catastrophic Forgetting
- Knowledge Distillation
- Small Language Model (SLM)
- Synthetic Data
- Model Collapse
- DistilBERT
- Open-Weight Model
- Closed Model
- Frontier Model
- Multimodal Model
- Model Snapshot
- Reasoning Model
- Extended Thinking
- Test-Time Compute
- OpenAI o-series
- Time to First Token (TTFT)
- LLM Streaming
- KV Cache
- Speculative Decoding
- Hallucination (AI)
- Knowledge Cutoff
- Sycophancy (AI)
- Lost in the Middle
- Context Rot