A study roadmap for LLMs and agents, ordered by what depends on what
Most AI reading lists are ordered by source: this book’s chapters, then that course’s lectures, then a pile of papers by date. That is a fine way to collect material and a poor way to learn it, because the order a source was written in is not the order the ideas depend on each other.
So I reordered mine by dependency. The rule is simple: nothing goes on the list before the things it needs. Here is the map, and the reasoning behind the places where the order is not obvious.
The map
BUILD IT BY HAND WHY SCALE SYSTEMS
autodiff scaling laws roofline + KV cache
→ tokenization → emergence → FlashAttention
→ attention (and its critics) → parallelism
→ transformer block → inference serving
→ train a small GPT → hardware economics
│
▼
POST-TRAINING ARCHITECTURE'S OTHER ANSWERS
instruction tuning mixture of experts
→ reward models, RLHF state-space models
→ DPO anatomy of a frontier model
→ reasoning models
│
▼
AGENTS DOES IT WORK? DEPLOYMENT
workflow vs agent evaluation security
→ context + memory → multi-agent governance
→ retrieval (RAG)
→ tools and harnesses
1. Build it by hand
Autodiff → tokenization → attention → the transformer block → training a small GPT.
Everything later is a statement about these pieces, so they come first and they
get built, not just read about. Training failures you will meet later (vanishing
gradients, dead units, exploding norms) are statements about the backward pass,
and you cannot diagnose a backward pass you have only ever called as
loss.backward().
Attention is easier to learn as an answer to a problem than as a formula: early translation models squeezed a whole sentence into one fixed-size vector, and attention was the fix. The transformer then mostly removed things around it.
Two placement notes:
- Decoding belongs here. Greedy, temperature, top-k, top-p and the autoregressive loop itself are easy to skip, but speculative decoding and best-of-N sampling both assume them later. The natural time to learn them is right after training something you can sample from.
- Tokenization is a judgment call. Karpathy teaches it after building a character-level model, so you feel the vocabulary problem first. I put it early because so much odd LLM behavior (spelling, arithmetic, the cost of non-English text) turns out to be a tokenizer artifact. If it feels unmotivated, moving it later costs nothing.
Sources: Karpathy’s Neural Networks: Zero to Hero series (micrograd, makemore, nanoGPT, build-nanogpt, minbpe); Simon Prince, Understanding Deep Learning; Jurafsky and Martin, Speech and Language Processing (3rd ed. draft).
2. Why anyone spent the money
Scaling laws → emergence, and the case against it.
Scaling laws read naturally as a constrained optimization: minimize loss subject to a compute budget, where compute is roughly 6 × parameters × tokens. The Chinchilla result corrected an earlier estimate of how to split that budget, and that correction is why models since 2022 train on far more tokens.
Emergence is worth studying together with the critique that the jumps come from discontinuous metrics rather than discontinuous capabilities. The critique is the better lesson, because it is really a lesson about measurement.
Sources: Kaplan et al. (2020); Hoffmann et al., “Chinchilla” (2022); Wei et al., “Emergent Abilities” (2022) and Schaeffer et al.’s rebuttal (2023); Sutton, “The Bitter Lesson.”
3. The systems layer
Roofline and the KV cache first, then everything else.
This is the stop I most want to get right, because one idea carries it: compare an operation’s arithmetic intensity to the hardware’s compute-to-bandwidth ratio, and you know whether it is compute-bound or memory-bound. Prefill is compute-bound; decode is bandwidth-bound. FlashAttention, the parallelism strategies, serving tricks like paged KV caches and continuous batching, and even why high-bandwidth memory is the expensive part of the chip, all read as corollaries once that is in place.
Sources: the JAX team’s How to Scale Your Model; Hugging Face’s Ultra-Scale Playbook; the FlashAttention and vLLM/PagedAttention papers.
4. Post-training
Instruction tuning → reward models and RLHF → DPO → reasoning models.
A base model is not a broken chatbot; it is a different object, and supervised fine-tuning does a surprising amount of the visible work. RLHF’s reward model is a pairwise-preference model (Bradley-Terry) fit to human comparisons.
The hard dependency here is DPO after RLHF, close together. DPO’s derivation starts from the RLHF objective, solves for the optimal policy in closed form, and inverts it so the reward model disappears into a classification loss. It is only elegant if the objective it inverts is fresh.
Reasoning models close the section: reinforcement learning on verifiable rewards, set against minimalist results showing how far a small set of good examples plus a decoding trick can go.
Sources: Nathan Lambert, Reinforcement Learning from Human Feedback; InstructGPT (2022); DPO (Rafailov et al., 2023); DeepSeek-R1 (2025); s1 (2025).
5. Architecture’s other answers
Mixture of experts, state-space models, and one frontier model up close.
I read these three as one argument about a single constraint. Mixture of experts decouples parameter count from compute per token. State-space models decouple memory from sequence length: a fixed-size state instead of a KV cache that grows with every token. A frontier model like DeepSeek-V3 shows a third option, compressing the KV cache rather than eliminating it.
That is why state-space models come after both attention and the KV cache. Their whole point is “attention was adopted for parallel training, and it costs you a cache that grows with context; here is the road not taken.” Without those two stops, the idea has no motivation.
Sources: Switch Transformer and Mixtral papers; HiPPO, S4 and Mamba; The Annotated S4; the DeepSeek-V3 technical report.
6. Agents
Workflow vs agent → context and memory → retrieval → tools and harnesses.
The most useful definition I have found is Anthropic’s: a workflow follows predefined code paths, an agent directs its own. Many systems called agents are workflows, and that is often the right engineering call. I want to build the basic reason-act loop by hand before touching a framework.
Retrieval sits after context management on purpose. RAG is one strategy for filling a limited context window, not a separate technology, and it makes more sense framed that way.
Sources: Anthropic, “Building Effective Agents”; the ReAct paper; Chip Huyen, AI Engineering; the Model Context Protocol spec.
7. Does it work?
Evaluation → multi-agent systems.
Evaluation comes first, and it may be the highest-leverage stop on the whole map. Public benchmarks rarely predict the failures of a specific application, so you end up building your own evals, in layers: automated checks, model-based screening, human judgment. Reliability metrics like pass^k from τ-bench are closer to what matters in production than a single pass rate.
Multi-agent systems come after because the literature on them is an empirical argument, with strong reported wins on one side and catalogued failure modes on the other. You cannot weigh that argument without eval literacy.
There is a thread worth noticing here: the emergence critique back in section 2 and evaluation here are the same lesson about measurement.
Sources: Hamel Husain’s writing on evals; τ-bench; SWE-bench; the MAST failure taxonomy paper.
8. Deployment reality
Security → governance.
Prompt injection has no reliable fix yet, for an architectural reason: instructions and data arrive in the same token stream with no privilege boundary. Simon Willison’s “lethal trifecta” (private data, untrusted content, and a way to send data out) is the clearest mental model I have seen. Retrieval from section 6 is one of the legs: retrieved text is untrusted text.
Sources: Simon Willison’s writing on prompt injection; the OWASP Top 10 for LLM and agentic applications; the NIST AI Risk Management Framework.
What I’m skipping, on purpose
- CNNs, GANs, diffusion. Off the critical path to language models. Diffusion deserves its own map.
- BERT and encoder models, beyond a mention. Useful history, less useful for a decoder-only world.
- Framework tours. They age quickly and teach configuration rather than mechanism.
- Every linear-attention variant. State-space models make the same argument once, with the best-documented example.
Where I am
At the very first stop: autodiff, by way of Karpathy’s micrograd. The next post will be about that, and about the one-line detail that makes or breaks a hand-written backward pass.