This lab works on phones, but writing code is much easier on a laptop.
LLM · Efficient Fine-Tuning and Model Adaptation · LM8-L1 · Intermediate
Whether you can fine-tune a model depends less on its size than on how many of its parameters you train and how many bits you store the rest in. A 7-billion-parameter model needs over 80 GB to fine-tune fully and about 5 GB with a quantised base and LoRA adapters.
You will find that configuration, then write the two formulas that make the estimate: the number of LoRA parameters and the training memory.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 explores the memory of full fine-tuning, LoRA and quantised storage. Stage 2 counts LoRA parameters from a configuration. Stage 3 writes a memory calculator. Only arithmetic is done; the memory figures are estimates from a formula, not measurements, and nothing is trained. Your code is graded on hidden configurations.
Educational simplification: a fixed allowance of 1.5 GB for activations, float32 LoRA parameters and Adam state, no framework overhead or fragmentation, four layer types per block. Real memory use depends on batch size, sequence length, the framework and the hardware. All configurations are generic examples.
Get notified when new robotics labs are released. One email per new lab, nothing else.