This lab works on phones, but writing code is much easier on a laptop.
LLM · Modern LLM Architecture and Inference · LM10-L2 · Intermediate
Modern decoder-only models keep the transformer block but change its parts: keys and values are shared between query heads, the feed-forward layer is gated, and sometimes only a few of many expert feed-forward layers run for each token.
You will pick a sharing level for a cache budget, count what sharing saves, and count the difference between the parameters a model stores and the parameters it uses for one token.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 explores the trade-off in the number of key-value heads. Stage 2 counts attention parameters and cache bytes for multi-head, grouped-query and multi-query layouts. Stage 3 counts the parameters of a gated feed-forward layer and of a mixture of experts. Only parameter and byte counts are computed; no model is run and no quality is measured. Your code is graded on hidden configurations.
Educational simplification: no biases, a generic 4,096-wide layer with 32 query heads in Stage 1, a single router matrix for the mixture of experts, and counts only. Real models differ in details such as the rounding of the hidden width.
Get notified when new robotics labs are released. One email per new lab, nothing else.