This lab works on phones, but writing code is much easier on a laptop.
LLM · Text Data and Tokenization · LM2-L3 · Intermediate
The loss is falling, the generated text is fluent, and the validation loss is suspiciously good. The bug is not in the model: it is in how the training examples were cut.
You will build input-target pairs from a token stream and write the checks that catch the data bugs that raise no errors.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 explores context length, stride and coverage. Stage 2 builds input-target pairs. Stage 3 diagnoses four classes of data-pipeline bugs. The token streams are short lists of integers; real corpora are far larger but the indexing is identical. Your code is graded on hidden sequences and pipelines.
Educational simplification: sequences of a few dozen integer tokens, one window length per case and four injected bug types. Real pipelines add shuffling, multi-process loading and padding for variable-length data, which introduce further failure modes. All data in this lab are invented.
Get notified when new robotics labs are released. One email per new lab, nothing else.