This lab works on phones, but writing code is much easier on a laptop.
LLM · Pretraining and Text Generation · LM5-L2 · Beginner
A language model gives a score to every token. What happens next is a choice, and the choice rule shapes the text more than most people expect: the same model sounds repetitive, creative or unhinged depending on three numbers.
You will explore the knobs, then write temperature, top-k and top-p yourself.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 explores temperature, top-k and top-p on one next-token distribution. Stage 2 implements temperature scaling and top-k truncation. Stage 3 implements nucleus sampling with inverse-CDF sampling. The distribution is a list of ten invented numbers; no language model runs here. Your code is graded on hidden logits and probabilities.
Educational simplification: a single ten-token distribution in Stage 1 and small random distributions in the code stages; the random number for sampling is given rather than drawn. Real decoders apply the same operations to vocabulary-sized logits at every step. All data in this lab are invented.
Get notified when new robotics labs are released. One email per new lab, nothing else.