Part 1: Course Overview

Work in progress

This introductory lecture talks about the three dimensions of scaling: compute, data and parameters. It then moves to fine-tuning, inference scaling and, finally, agents.

Reading material (listed on the course site for the next lecture, Test-time Compute Scaling):

Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (Brown et al. 2024)
Archon: An Architecture Search Framework for Inference-Time Techniques (Saad-Falcon et al. 2024)
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters (Snell et al. 2024)
How Do Large Language Monkeys Get Their Power (Laws)? (Schaeffer et al. 2025)

After following and digesting the lecture, you should be able to answer the following:

Quiz

1. Which three things do the LLM scaling laws say you can increase to get better models? Context length, temperature and prompt length Parameters, training data and compute Vocabulary size, languages and batch size

2. What's the difference between few-shot and zero-shot prompting? Few-shot includes a few examples in the prompt; zero-shot gives only the task Few-shot fine-tunes on a few examples; zero-shot uses no training data Few-shot works with small models; zero-shot needs large ones

3. You plot a benchmark score against the number of parameters. What does an emergent ability look like? A straight line going steadily up High for small models, then dropping as models grow Flat for smaller models, then a sharp jump past a certain size

4. Besides scale, what made ChatGPT so much more useful than earlier LLMs? A new architecture that replaced the transformer Fine-tuning with instruction tuning and RLHF Training on internet text for the first time

5. In RLHF (Reinforcement Learning from Human Feedback), what guides the model towards better answers? A reward model trained on human preferences A larger set of handwritten question/answer pairs A human correcting every answer during training

6. Why is it called inference scaling? The model grows bigger while answering We spend more compute at answer time, not on changing the model The model infers its own scaling laws

7. Why is inference scaling hard to use in production? It only works for maths problems It permanently changes the model's weights Many attempts per question cost too much for a fast, cheap service

8. What's the difference between chain-of-thought and a reasoning model? CoT is a prompting technique; a reasoning model is trained to think first They're the same thing with different names A reasoning model is a bigger model prompted with CoT

9. What turns an LLM into an agent? A longer context window Running it on your own computer Tools, feedback from its actions, and deciding when to stop

10. When is a verifier most useful? When there's no single right answer, like in creative writing When outputs can be checked automatically, like running tests on code Only when a human reviews every output