FixTheRobot Explore Robotics / Robot Learning / M1 / M1-L4

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Foundations: Policies, Data and Evaluation · M1-L4 · Beginner

Read a Failed Training Run

Experiment log · Scout at Meridian General Hospital

Four tickets reached the Meridian General robotics desk this week, and each one says only "the new policy is worse". Scout crashed into the pillar after a flawless demo, the dashboard return climbed while deliveries stopped, a steering model fitted its data perfectly and wobbled in the hall, and a training loss went to infinity.

You will read the log and the curves of each run, name the failure mode, pick the first check that confirms it, and choose the fix before anyone retrains anything. Every number and every drawing comes from the same real simulation.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Learned policies fail in a few recurring ways that look alike from outside: the robot does the wrong thing. Here each incident has a different cause: distribution shift (the starts met in service were never in the data), reward hacking (a shaping bonus that pays more than finishing the task), overfitting (a flexible model fitted to the noise in a few points) and unstable training (a learning rate above the stability limit of the loss).

    Each stage asks for the failure mode, the first check that confirms it and the smallest fix that addresses the cause. There is no code to write; the skill is reading the evidence, and the evidence in the log and in the drawing is generated by the same run, so it cannot disagree.

    Theory for this lab

    Foundations: Policies, Data and Evaluation labs

    1. M1-L1: Record a Rollout
    2. M1-L2: Learn to Reach the Goal
    3. M1-L3: Score a Policy Fairly
    4. M1-L4: Read a Failed Training Run: this lab.

    Educational simplification: each incident is simulated so that one cause dominates (a one-parameter policy search for the reward hack, a ten-point polynomial fit for overfitting, a three-weight linear model for instability). Real failures often combine several causes.

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.