Pace sheet
Minute by minute, day by day
How to read this sheet
On Day 1, each module is 95 minutes: 45 minutes of lecture, then 50 of lab. On Days 2 to 5, each module is 120 minutes: 55 minutes of lecture, 55 of lab, then a 10-minute debrief. On Days 2 to 4 the second module breaks for lunch between its lecture and its lab. On Day 5, the capstone build runs on across lunch. The capstone (Module 15) has no lecture: it takes 11:30–12:25, 13:25–14:30 and 14:45–16:45 on Day 5.
Each module has two tables. The first is the lecture’s live plan, generated from the lecture page: minutes count from the start of the lecture, and “In the room” lists the checks, predictions and demos whose minutes the row includes. The second is the lab: minutes count from the start of the lab. “CP” is a checkpoint; Checkpoint N belongs to Exercise N. The last column says what participants should have passed by the end of that row. If more than a third of the room has not, apply the module’s “behind” rule. A behind rule’s minute is a lab minute unless it says lecture minute or build minute. Clock times are on the schedule.
Day 1: the lab is the 50-minute core path and ends the module. Days 2 to 5: the lab is 55 minutes: the same core path, with 5 minutes of slack for setup and downloads in its first row, so every later row starts 5 minutes later than the notebook’s own minutes suggest. If setup goes quickly, the room runs ahead and keeps the minutes. The lab table ends with the 10-minute debrief: the Explain step, taken with the whole room. Each day also opens with retrieval practice and closes with a synthesis, both on the day pages. What to say in each debrief, and what to cut when a day’s clock slips, are in the facilitator guide.
These minutes are planning estimates. No lab has been timed on Colab or on a T4. Labs 1 to 5 were timed on a shared CPU only, and Labs 2 to 5 took far longer there than these minutes allow (see the facilitator guide). The lecture plans were set on paper; no lecture has yet been given aloud against its clock (see the readiness page). Treat every row as a target until you have run the notebook on the room’s runtime and given the lecture once.
Collapsed callouts marked Optional, sections marked Reference, and checks or demos that a live plan does not name sit outside the lecture minutes. The stretch section of every lab sits outside the lab minutes.
Before Day 1: Module 0 · Coding agents in the terminal
Optional pre-work, planned at 60 minutes, with no lecture; the same rows pace the optional drop-in clinic on Day 1, 08:00–09:00. Minutes count from the start. Provisional: these rows follow the step plan on the Module 0 page. No one has yet timed the module on a fresh laptop, so they are a target, not a measurement. Install failures and what to do about them are in the facilitator guide.
| Minutes | Segment | By the end |
|---|---|---|
| 0–10 | Install one coding agent, sign in, start it in ~/agent-apps |
the agent answers the sanity prompt and leaves the folder empty |
| 10–20 | Prompts: install git and gh if missing, git identity, gh sign-in in the browser, a public repository |
gh auth status (run by the agent) shows the participant signed in; the repository’s GitHub page shows the README |
| 20–38 | App 1 by prompts: protein structure explorer (1UBQ), its separate check, the fix, its three.js page | the check matches the page’s numbers; protein/ is committed |
| 38–55 | App 2 by prompts: RFM customer segmentation, its separate check and hand-computed recency, the fix, its three.js page | the check and the recency match; rfm/ is committed |
| 55–60 | Publish by prompts: landing page, push, GitHub Pages on | both Pages URLs load (allow up to 10 minutes after enabling Pages) |
Behind at minute 10: if the agent is still not installed or signed in, pair the participant with a neighbor whose agent works. They take turns writing the prompts in the neighbor’s repository, and the participant finishes their own install at home. At 08:55: everyone stops; the rest is homework.
Day 1
Opening
| Minutes | Segment | By the end |
|---|---|---|
| 0–5 | Welcome: the intro slides with the opening lines | |
| 5–10 | Setup check: everyone runs 00-setup.ipynb |
each participant has read the provider line it prints |
Module 1 · Text as data
Lecture (45 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–5 | 1. What makes language hard | — |
| 5–14 | 2. Tokens, vocabularies and Zipf’s law | Check yourself (2 min) |
| 14–30 | 3. N-gram language models | Demo (4 min) |
| 30–35 | 4. Bag-of-words and TF-IDF | — |
| 35–40 | 5. Naive Bayes and logistic regression | — |
| 40–45 | 7. Where count-based methods stop working | Check yourself (2 min) |
Lab (50 minutes)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup, data, Exercise 1 (tokens and vocabulary) | CP1 |
| 8–20 | Exercise 2 (n-gram counts, add-k) | CP2 |
| 20–28 | Exercise 3 (perplexity) | CP3 |
| 28–33 | Exercise 4 (sampling) | CP4 |
| 33–43 | Exercise 5 (TF-IDF, two classifiers) | CP5 |
| 43–50 | Exercise 6 (precision, recall, F1; read the errors); the baseline card | CP6 |
Behind at minute 20: run Exercise 4 as a demonstration. Exercises 3, 5 and 6 may not be cut.
Module 2 · Word vectors and neural networks
Lecture (45 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. Where counts stop | — |
| 3–10 | 2. The distributional hypothesis and dense vectors | Check yourself (2 min) |
| 10–16 | 3. Skip-gram with a full softmax | — |
| 16–29 | 4. Negative sampling | Demo (4 min) |
| 29–36 | 6. PyTorch refresher | Check yourself (2 min) |
| 36–40 | 7. Feed-forward networks and backpropagation | — |
| 40–45 | 8. Bengio’s neural language model | — |
Lab (50 minutes)
| Minutes | Segment | By the end |
|---|---|---|
| 0–4 | Setup, data, three training pairs | |
| 4–16 | Exercise 1 (the SGNS loss) | CP1 |
| 16–20 | Provided: train the embeddings; predict the loss curve | first loss = (K+1)\log 2 |
| 20–27 | Exercise 2 (nearest neighbors) | CP2 |
| 27–32 | Provided: analogies and the PCA picture | |
| 32–38 | Exercise 3 (averaging embeddings) | CP3 |
| 38–46 | Exercise 4 (feed-forward classifier) | CP4 |
| 46–50 | The comparison with TF-IDF; results card |
Behind at minute 20: the analogy step (27–32) becomes a demonstration. Skip-gram training took 695 s on the build CPU, far over its 4 minutes; start it before the Exercise 1 discussion ends.
Module 3 · Sequence models
Lecture (45 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–2 | 1. Where Module 2 left us | — |
| 2–5 | 2. The recurrent neural network | — |
| 5–13 | 3. The RNN language model | — |
| 13–27 | 4. Backpropagation through time | Demo (4 min) |
| 27–39 | 5. The LSTM | Check yourself (2 min) |
| 39–45 | 6. Measuring the improvement fairly | Check yourself (2 min) |
Lab (50 minutes)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup, data, Exercise 1 (an RNN cell by hand) | CP1 |
| 8–14 | Exercise 2 (loss, perplexity, bits per character) | CP2 |
| 14–16 | Run: the n-gram baseline, measured the same way | |
| 16–20 | Run: train the RNN language model | |
| 20–29 | Exercise 3 (vanishing gradients; clipping) | CP3 |
| 29–35 | Exercise 4 (the LSTM cell) | CP4 |
| 35–42 | Run: train the LSTM; the results table | LSTM below the trigram |
| 42–50 | Exercise 5 (sampling with temperature) | CP5 |
Behind at minute 16: set QUICK = True before the training runs (a quarter of the steps; not the recorded baselines).
Module 4 · Seq2seq and attention
Lecture (45 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–5 | 1. From next-token prediction to transduction | — |
| 5–13 | 2. The encoder-decoder model | — |
| 13–19 | 3. The fixed-vector bottleneck | Check yourself (2 min) |
| 19–35 | 4. Attention | Demo (4 min) |
| 35–40 | 5. Three score functions | — |
| 40–45 | 6. Attention as soft alignment | Check yourself (2 min) |
Lab (50 minutes)
| Minutes | Segment | By the end |
|---|---|---|
| 0–3 | Setup; read the generated dates | |
| 3–13 | Exercise 1 (teacher-forced forward pass and loss) | CP1 |
| 13–21 | Exercise 2 (exact match by input length); predict which bucket fails | CP2 |
| 21–33 | Exercise 3 (dot-product attention); accuracy with and without attention | CP3 |
| 33–43 | Exercise 4 (the additive score) | CP4 |
| 43–50 | Exercise 5 (heat-maps, alignment hit rate) | CP5 |
Behind at minute 21: keep Exercises 3 and 4 (the objectives need both scores); shorten Exercise 5 to viewing the heat-maps. Training runs start while participants write their next prediction.
Day 2
Retrieval practice
| Minutes | Segment | By the end |
|---|---|---|
| 0–6 | Alone, on paper, notes closed: the five questions on the Day 2 page | an answer to each question |
| 6–10 | Compare with a neighbor | |
| 10–15 | The two most-missed questions with the room; then the opening lines |
Module 5 · The transformer
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–5 | 1. From attention over an encoder to self-attention | — |
| 5–20 | 2. Scaled dot-product attention | Demo (4 min); Check yourself (2 min) |
| 20–27 | 3. The causal mask | Check yourself (2 min) |
| 27–33 | 4. Multi-head attention | — |
| 33–38 | 5. Positional encodings | — |
| 38–47 | 6. The transformer block | Check yourself (2 min) |
| 47–51 | 7. A decoder-only language model, and the other two variants | — |
| 51–55 | 8. Cost and parallelism | — |
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup; Exercise 0 (read the model); the 5 minutes of slack | |
| 8–20 | Exercise 1 (scaled dot-product attention); predict the score spread | CP1 |
| 20–28 | Exercise 2 (the causal mask; leak test) | CP2 |
| 28–34 | Exercise 3 (positions); whole-model leak test | CP3, CP3b |
| 34–48 | Exercise 4 (train the mini-GPT and the LSTM; compare) | CP4a, CP4b |
| 48–55 | Exercise 5 (looking at the heads) | CP5 |
| 55–65 | Debrief: mini-GPT and LSTM nats per character, full budget or QUICK; the misconception “the transformer wins”; the bridge to Module 6 |
the room’s numbers on the board |
Behind at minute 28: set QUICK = True before Exercise 4 (on by default on a CPU runtime); shorten Exercise 5.
Module 6 · Pretraining and the Hugging Face stack
Lecture (55 minutes, before lunch)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. What the from-scratch transformer leaves open | — |
| 3–16 | 2. Subword tokenization: byte-pair encoding | Check yourself (2 min) |
| 16–19 | 3. Pretrain once, reuse many times | — |
| 19–27 | 4. Causal language modeling: GPT | Check yourself (2 min) |
| 27–38 | 5. Masked language modeling: BERT | Demo (4 min) |
| 38–43 | 6. The Hugging Face stack | — |
| 43–55 | 7. Fine-tuning an encoder for classification | Check yourself (2 min) |
The lecture stops at lunch, wherever it has reached. On slow Wi-Fi, participants can open Lab 6, switch on a T4 and run the setup cell as they leave, so the checkpoints download over lunch.
Lab (55 minutes, after lunch) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Reconnect; setup (start the downloads, or rerun the cell if Colab reset the runtime over lunch); the 5 minutes of slack | |
| 8–18 | Exercise 1 (byte-pair encoding by hand) | CP1 |
| 18–25 | Exercise 2 (train a tokenizer; tokens per word) | CP2 |
| 25–33 | Exercise 3 (causal LM perplexity) | CP3 |
| 33–43 | Exercise 4 (masked LM: select and corrupt) | CP4 |
| 43–55 | Exercise 5 (fine-tune an encoder); the results table | CP5a, CP5b, CP5c |
| 55–65 | Debrief: tokens per word; the encoder’s test accuracy and macro-F1 beside Labs 1 and 2, by runtime; the misconception “a pretrained model always wins by a wide margin”; the bridge to Module 7 | the room’s numbers on the board |
Behind at minute 25: run Exercise 2 as a demonstration; if still behind at minute 33, Exercise 3 too. Exercises 1, 4 and 5 may not be cut.
Module 7 · Fine-tuning and LoRA
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–4 | 1. A language model continues; it does not answer | — |
| 4–9 | 2. Chat templates | — |
| 9–17 | 3. The instruction-tuning loss | Check yourself (2 min) |
| 17–21 | 4. What full fine-tuning costs | Predict (1 min) |
| 21–39 | 5. LoRA: a low-rank update | Demo (4 min); Check yourself (2 min) |
| 39–45 | 6. LoRA in practice: where, how large, and the peft library |
— |
| 45–48 | 7. Decoding settings | — |
| 48–55 | 8. Evaluating generated text | Check yourself (2 min) |
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup; Step 0 (the base model continues); the 5 minutes of slack | |
| 8–18 | Exercise 1 (the LoRA layer) | CP1 |
| 18–23 | Exercise 2 (merging the update) | CP2 |
| 23–29 | Exercise 3 (the chat template) | CP3 |
| 29–37 | Exercise 4 (the response mask and labels) | CP4 |
| 37–47 | Exercise 5 (configure LoRA, count parameters; training starts) | CP5 |
| 47–55 | Exercise 6 (ROUGE-n; decoding settings) | CP6 |
| 55–65 | Debrief: trainable parameters, response perplexity before and after, ROUGE-1, whether replies stop; the misconception “lower perplexity means better answers”; the bridge to Module 8 | the room’s numbers on the board |
Behind at minute 37: run Exercise 6’s SAMPLING cell as a demonstration and keep rouge_n. The stretch is the first thing dropped on any day.
Closing synthesis
| Minutes | Segment | By the end |
|---|---|---|
| 0–5 | In pairs: the three prompts on the Day 2 page | |
| 5–15 | With the room: fixed and left open for Modules 5 to 7; the running table, from the debriefs’ numbers; one claim | the table on the board, kept for Day 5 |
Day 3
Retrieval practice
| Minutes | Segment | By the end |
|---|---|---|
| 0–6 | Alone, on paper, notes closed: the five questions on the Day 3 page | an answer to each question |
| 6–10 | Compare with a neighbor | |
| 10–15 | The two most-missed questions with the room; then the opening lines |
Module 8 · LLMs through APIs
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. From your own model to someone else’s | — |
| 3–12 | 2. The message list | Check yourself (2 min) |
| 12–16 | 3. Prompting patterns | — |
| 16–24 | 4. Structured output: the schema as a contract | — |
| 24–34 | 5. Tool calling: a loop you write | Check yourself (3 min) |
| 34–43 | 6. Cost and latency | Demo (4 min) |
| 43–49 | 7. Evaluating outputs | Check yourself (2 min) |
| 49–53 | 8. Failure modes and security | — |
| 53–55 | 9. A fluent answer carries no confidence signal | — |
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup; read the provider cell; the 5 minutes of slack | |
| 8–16 | Exercise 1 (parse a raw response) | CP1 |
| 16–28 | Exercise 2 (validate and retry) | CP2 |
| 28–36 | Exercise 3 (score the outputs); the evaluation run opens this exercise | CP3 |
| 36–48 | Exercise 4 (the tool loop) | CP4 |
| 48–53 | Exercise 5 (cost and latency) | CP5 |
| 53–55 | What the response did not tell you | |
| 55–65 | Debrief: per provider and path, schema-validity rate, exact match with its standard error, cost, latency; the misconception “structured output means correct output”; the bridge to Module 9 | the room’s numbers on the board |
Behind at minute 28: let the evaluation run finish while participants write Exercise 3, which is where the notebook places it; the checkpoints use the FakeProvider and do not wait for it.
Module 9 · Reinforcement and preference learning
Lecture (55 minutes, before lunch)
| Minutes | Segment | In the room |
|---|---|---|
| 0–4 | 1. No label for “better” | — |
| 4–9 | 2. The minimum reinforcement learning | — |
| 9–20 | 3. The policy gradient and REINFORCE | Check yourself (2 min) |
| 20–32 | 4. Baselines and variance | Demo (4 min) |
| 32–35 | 6. Why comparisons instead of scores | Predict (1 min) |
| 35–44 | 7. The Bradley–Terry model | Check yourself (3 min) |
| 44–55 | 8. Training a reward model | — |
The lecture stops at lunch, wherever it has reached.
Lab (55 minutes, after lunch) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Reconnect; setup; Exercise 0 (the toy environment, three pairs); the 5 minutes of slack | |
| 8–20 | Exercise 1 (returns and the REINFORCE loss) | CP1a, CP1b, CP1c |
| 20–29 | Exercise 2 (a baseline and its variance) | CP2a–CP2e |
| 29–37 | Exercise 3 (the Bradley–Terry rater) | CP3a, CP3b |
| 37–49 | Exercise 4 (reward-model loss and training) | CP4a, CP4b |
| 49–55 | Exercise 5 (normalize and save) | CP5 |
| 55–65 | Debrief: gradient variance without and with the baseline; the reward model’s held-out accuracy beside its ceiling; the misconception “a reward model should reach 100%”; the bridge to Module 10 | the room’s numbers on the board |
Behind at minute 29: run Exercise 2’s five-seed training as a demonstration. Part B (29–55) needs the Lab 9 data files; see the facilitator guide if they are missing.
Module 10 · RLHF
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. What Module 9 left open | — |
| 3–9 | 2. The InstructGPT pipeline | — |
| 9–19 | 3. The KL-regularized objective | Check yourself (2 min) |
| 19–26 | 4. Optimizing it: a per-token penalty and the policy gradient | — |
| 26–40 | 6. DPO: from the optimal policy to a classification loss | Check yourself (2 min) |
| 40–55 | 7. Failure modes | Demo (4 min); Discuss (2 min) |
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup; Step 0 (reward model and gold rule on the reference’s samples); the 5 minutes of slack | |
| 8–16 | Exercise 1 (log-probabilities of a response) | CP1 |
| 16–24 | Exercise 2 (per-token reward with a KL penalty) | CP2 |
| 24–35 | Exercise 3 (exact KL; RLHF with the penalty) | CP3 |
| 35–43 | Exercise 4 (remove the penalty: reward hacking) | CP4 |
| 43–55 | Exercise 5 (the DPO loss; DPO training); results card | CP5 |
| 55–65 | Debrief: reward gain, KL drift, gold reward and distinct bigrams with and without the penalty, and for DPO; the misconception “a higher reward-model score means a better policy”; the bridge to Module 11 | the room’s numbers on the board |
Behind at minute 24: do not cut a training run; ask the Predict questions while training runs (estimated 1.5 minutes per run on a T4, unmeasured).
Closing synthesis
| Minutes | Segment | By the end |
|---|---|---|
| 0–5 | In pairs: the three prompts on the Day 3 page | |
| 5–15 | With the room: fixed and left open for Modules 8 to 10; today’s rows of the running table; what the numbers do not say | today’s rows on the board, kept for Day 5 |
Day 4
Retrieval practice
| Minutes | Segment | By the end |
|---|---|---|
| 0–6 | Alone, on paper, notes closed: the five questions on the Day 4 page | an answer to each question |
| 6–10 | Compare with a neighbor | |
| 10–15 | The two most-missed questions with the room; then the opening lines, including the honesty rule |
Module 11 · Calibration
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. What Day 3 has left open | — |
| 3–9 | 2. What a probability should mean | Check yourself (2 min) |
| 9–16 | 3. Measuring calibration: reliability diagrams and ECE | — |
| 16–28 | 4. Proper scoring rules | Check yourself (3 min) |
| 28–39 | 6. Temperature scaling | Demo (4 min) |
| 39–45 | 7. Confidence from a language model behind an API | — |
| 45–55 | 8. Selective prediction and the cost of a wrong action | Check yourself (2 min) |
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup and Exercise 0 (which classifier, which provider); the 5 minutes of slack | |
| 8–18 | Exercise 1 (reliability bins and ECE) | CP1 |
| 18–25 | Exercise 2 (Brier score and log loss) | CP2 |
| 25–35 | Exercise 3 (temperature scaling) | CP3 |
| 35–47 | Exercise 4 (parse a stated confidence); start ask_all first |
CP4 |
| 47–55 | Exercise 5 (risk–coverage and the threshold), ending with the closing question: what would you let act alone? | CP5 |
| 55–65 | Debrief: ECE, Brier score and log loss before and after temperature scaling, with accuracy unchanged; the stated confidence’s ECE; the misconception “temperature scaling makes the model more accurate”; the closing question as the bridge to Module 12 | the room’s numbers on the board |
Timing: the closing question is part of Exercise 5’s 8 minutes. If the room is late, take it into the debrief as the bridge to Module 12, whose lecture follows the break. Behind at minute 35: start ask_all before discussing Exercise 3’s results.
Module 12 · Calibrated decisions: RLCD and Jev
Lecture (55 minutes, before lunch)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. What Module 11 left open | — |
| 3–16 | 2. Two training signals: preference and outcome | Predict (2 min) |
| 16–31 | 3. Act, ask, escalate: thresholds from costs | Demo (4 min); Check yourself (2 min) |
| 31–35 | 4. Where a decision model fits: route, guard, verify | — |
| 35–45 | 5. Case study: what is public, what is reported, what is ours | Check yourself (2 min) |
| 45–50 | 7. Case study: Jev’s interface | — |
| 50–55 | 8. Case study: probabilities, not confidence, on a reliability diagram |
— |
The lecture stops at lunch, wherever it has reached.
Lab (55 minutes, after lunch) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Reconnect; the lookalike-package warning before the setup cell installs anything; setup and Exercise 0 (look at the decisions); the 5 minutes of slack | |
| 8–22 | Exercise 1 (accuracy reward and Brier reward; our illustration) | CP1 |
| 22–32 | Exercise 2 (typed questions, typed answers); decide_all starts first |
CP2 |
| 32–40 | Exercise 3 (calibration arrays: \hat{p} against confidence) |
CP3 |
| 40–50 | Exercise 4 (act, ask, escalate) | CP4 |
| 50–55 | What this lab showed and what it did not | |
| 55–65 | Debrief: accuracy, ECE and Brier score under the two rewards (our illustration); the thresholds and the cost per case; the misconception “Jev’s confidence is the probability that its answer is right”; the bridge to Module 13 |
the room’s numbers on the board, each labeled with its path |
Behind at minute 32: shorten the Exercise 3 discussion. Never cut the closing cell (50–55): it carries the honesty rule.
Module 13 · Retrieval-augmented generation
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–3 | 1. What Module 12 left open: why retrieve | — |
| 3–8 | 2. The pipeline | Check yourself (2 min) |
| 8–12 | 3. Chunking | — |
| 12–18 | 4. Dense retrieval | — |
| 18–22 | 5. Sparse and hybrid retrieval | — |
| 22–30 | 6. Reranking: cross-encoders and a decision model | Check yourself (2 min) |
| 30–33 | 7. Generation: sources, citations, abstention | — |
| 33–42 | 8. Evaluating retrieval | Demo (4 min) |
| 42–49 | 9. Evaluating answers | Check yourself (2 min) |
| 49–51 | 10. Common failures | — |
| 51–55 | 11. LlamaIndex and LangChain | — |
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–9 | Setup and Exercise 0 (the path banner, corpus, three questions); the 5 minutes of slack | |
| 9–17 | Exercise 1 (recall@k and reciprocal rank) | CP1 |
| 17–26 | Exercise 2 (build the index; choose L and k); the sweep starts first | CP2 |
| 26–34 | Exercise 3 (the same retriever in LangChain) | CP3 |
| 34–43 | Exercise 4 (rerank the candidates) | CP4 |
| 43–53 | Exercise 5 (faithfulness); answers generated | CP5 |
| 53–55 | What this lab showed and what it did not | |
| 55–65 | Debrief: recall@k and MRR at the chosen L and k, with and without reranking; faithfulness and the two-by-two table (plumbing probes until the question set exists); the misconception “a faithful answer is a correct answer”; the bridge to Module 14 | the room’s numbers on the board |
Behind at minute 34: on the keyed Jev path, rerank test only.
Closing synthesis
| Minutes | Segment | By the end |
|---|---|---|
| 0–5 | In pairs: the three prompts on the Day 4 page | |
| 5–13 | With the room: fixed and left open for Modules 11 to 13; today’s rows of the running table; ranking or values | today’s rows on the board, kept for Day 5 |
| 13–15 | Announce the capstone pairs; each pair decides tonight which keys it will use | every participant knows their pair |
Day 5
Retrieval practice
| Minutes | Segment | By the end |
|---|---|---|
| 0–6 | Alone, on paper, notes closed: the five questions on the Day 5 page | an answer to each question |
| 6–10 | Compare with a neighbor | |
| 10–15 | The two most-missed questions with the room; then the opening lines | pairs and their path classes confirmed |
Module 14 · Agents
Lecture (55 minutes)
| Minutes | Segment | In the room |
|---|---|---|
| 0–5 | 1. What the Module 8 loop cannot do | — |
| 5–9 | 2. ReAct: reasoning and acting in one loop | — |
| 9–13 | 3. Tools: LangChain tools, runnables and tool design | — |
| 13–23 | 4. LangGraph: state, nodes, edges, conditional routing | Check yourself (2 min) |
| 23–30 | 5. Checkpoints, memory, interrupts and replay | Check yourself (2 min) |
| 30–38 | 6. Where agents fail | Check yourself (2 min) |
| 38–47 | 7. A decision model in the control loop: route, guard, verify | Demo (4 min) |
| 47–50 | 8. Measuring an agent | — |
| 50–55 | 9. Designing the harness: patterns, subagents and context | — |
Lecture 14 was the densest lecture in the desk timing of 2026-10-06, and its plan is checked on paper only. Behind at lecture minute 38 (section 7 not yet started): give section 9 as reading, as the lecture’s live-plan note says; that recovers its 5 minutes, and the core lab does not depend on it.
Lab (55 minutes) and debrief (10)
| Minutes | Segment | By the end |
|---|---|---|
| 0–8 | Setup and Exercise 0 (read the graph); the lookalike-package warning again; the 5 minutes of slack | |
| 8–14 | Exercise 1 (a safe calculator) | CP1 |
| 14–26 | Exercise 2 (thresholds as edges) | CP2 |
| 26–34 | Exercise 3 (the human-in-the-loop driver) | CP3 |
| 34–41 | Exercise 4 (fork and replay); the evaluation run starts first | CP4 |
| 41–51 | Exercise 5 (measure the agent) | CP5 |
| 51–55 | What this lab showed and what it did not | |
| 55–65 | Debrief: route accuracy; unsafe-action rate and injection success, each with its denominator, by path; the misconception “an unsafe-action rate of zero means the guard is safe”; the bridge to the capstone | the room’s numbers on the board |
Behind at minute 34: on a CPU runtime, cut the probability-shift test to 30 items (slice the list in the evaluation-run cell).
Module 15 · Capstone
240 minutes in the two module slots after Module 14, then the day’s closing slot, where the wrap-up ends: 255 minutes in all. There is no lecture table and no lab table: the plan below follows lecture 15’s timing and the starter notebook, 15-capstone.ipynb. The evaluation set does not exist yet, so the baseline and the comparison in this plan cannot be run as designed: see the readiness page.
Part I and Part II (the middle slot: the brief, then 110 minutes of build across lunch). Build minutes count from the end of the brief, as in lecture 15 and the notebook; lunch falls at build minute 45.
| Minutes | Segment | By the end |
|---|---|---|
| brief, 0–10 | Part I. The brief (lecture sections 1–4): a walk through the tables | pairs formed; path class chosen |
| build, 0–20 | Setup, self-test, write after_verify, baseline on dev then test, read ten traces |
self-test green; both baselines run |
| build, 20–30 | Choose one component; fill in the hypothesis card | hypothesis card filled |
| build, 30–45 | Make the change; rerun the self-test after every edit; iterate on dev |
self-test still green; the notebook saved to Drive |
| Lunch | ||
| build, 45–105 | Reconnect (if Colab reset the runtime, rerun the setup and the dev baseline first); continue the change |
self-test still green |
| build, 105–110 | Freeze the configuration; do not run test |
configuration frozen |
At build minute 20: check that every pair has a dev baseline and has read traces. Behind at build minute 30: any pair still without a dev baseline gets help before anyone else.
Part III (the afternoon slot, its first 105 minutes)
| Minutes | Segment | By the end |
|---|---|---|
| 0–20 | Run test on the frozen system; run the last cell; hand in the file |
submission handed in |
| 20–80 | Two-minute shares, in menu order, with questions after each group | |
| 80–105 | The combined table, read with the room; the three cautions read aloud |
More than 15 pairs: group the shares by component, four minutes per group.
Part IV. Wrap-up (30 minutes: the last 15 of the afternoon slot, then the day’s closing slot)
| Minutes | Segment | By the end |
|---|---|---|
| 0–10 | Section 7, five days, one line of ideas, with the boards from Days 2 to 4 | the table complete, with the room’s numbers |
| 10–18 | Section 8, an evaluation checklist for agentic systems | |
| 18–25 | Section 9, open problems | each pair has named one open problem and the measurement it would start with |
| 25–30 | Section 10, further study; close |
Closing synthesis
On Day 5 the closing synthesis is not a separate slot: it is the wrap-up’s last 15 minutes, and the prompts on the Day 5 page run through the whole wrap-up. Item 1 (the whole line) is section 7’s table. Item 2 (today’s table) takes Lab 14’s numbers from its debrief and the capstone’s from Part III’s combined table. Item 3 (what is still open) is section 9.