FixTheRobot Explore Robotics / Robot Learning / M2 / M2-L3

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Supervised Learning for Robots · M2-L3 · Beginner

Catch the Overfit Model

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital. The steering model for Scout reports 99% on the logged runs. On the ward corridor it drifts into the wall, and the validation report looks exactly as good as it did last week.

You will find out why. First the log itself: every logged step is tagged with the run it came from, and the validation split cut the runs in two. Then the model: a polynomial that always picks its highest degree. Then the training run: weights kept from the last epoch instead of the best one. The numbers in every drawing come from real demonstrations, real fits and real gradient-descent runs.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 splits the logged steps by episode, so that no run is on both sides, and shows with a real lookup model how a step-level split reports an R squared near 98% where the honest split reports a fraction of that. Stage 2 asks you to fit polynomial ridge regressions with a small Gaussian-elimination solver and to choose the degree with the lowest validation error, on datasets whose noise differs so that the best degree changes. Stage 3 reads real stochastic-gradient-descent loss curves and returns the epoch to restore and the epoch at which a patience-based early stopper would stop. The last question asks what else could be leaking: statistics computed on all the data, duplicated episodes, labels from the future.

    Grading is result-based: your functions are run on the practice example and on freshly generated hidden data sets, and compared with the exact answers. All randomness, the data and the training runs live outside your code.

    Theory for this lab

    Supervised Learning for Robots labs

    1. M2-L1: Fit Steering with Gradient Descent
    2. M2-L2: Train a Tiny Neural Network
    3. M2-L3: Catch the Overfit Model: this lab.

    Educational simplification: the run-specific effects (a steering offset and a payload that identifies the run) are injected into real expert demonstrations to make the leak visible, the steering curves are one-dimensional, and the overfit models are a lookup table and a polynomial. Real leakage is usually subtler and spread over several features.

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.