FixTheRobot Explore Robotics / Robot Learning / M3 / M3-L3

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Imitation Learning · M3-L3 · Intermediate

Add DAgger Corrections

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital, 09:40. The clone trained on the operator's demonstrations from the bottom wall works there and drives straight into the pillar from anywhere near the middle of the hall. Nobody demonstrated the middle, and nobody wants to drive a hundred more demonstrations from a joystick.

The expert controller can still be asked a question: "you are here, what would you do?". You will run the clone, ask that question at every state it reaches, add the answers to the pool and retrain, round after round, and then check how much of the improvement comes from the corrections and how much would have come from the same number of ordinary demonstrations.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 is the bookkeeping that the DAgger algorithm depends on: the pool of training data is the union of the expert's own demonstrations and every later round of expert-labelled learner states, with exact duplicates counted once. The classic mistake is to retrain on the newest round alone, which makes the policy forget the states it used to handle. Stage 2 uses that pool to train a small k-nearest-neighbour policy for every round. The simulator deploys each of your tables in the hall, from starts you never see, and reports the success of every round, so you can see the climb that interactive correction buys and the bounce that forgetting causes. Stage 3 hands you the whole loop with three controls (rounds, mixing beta, neighbours k) and a baseline: plain behaviour cloning that is given the same number of expert labels but collects them in the narrow band the expert already demonstrated.

    Grading is result-based. Your aggregate is compared row by row with the exact one; your tables are compared with the exact tables and, more importantly, are run closed-loop. Nothing in the data handed to your code reveals the answer, and the held-out starts stay in JavaScript.

    Theory for this lab

    Imitation Learning labs

    1. M3-L1: Clone the Expert
    2. M3-L2: Watch the Clone Drift
    3. M3-L3: Add DAgger Corrections: this lab.
    4. M3-L4: Clean the Demonstrations

    Educational simplification: the "expert" is the scripted controller of the shared world, answers every query exactly and instantly, and never makes a labelling mistake; the learner is a k-nearest-neighbour policy table on a 0.5 m grid rather than a neural network; and each round adds only a few short rollouts so that Python in the browser stays fast. Real DAgger needs a human or an optimising controller to label the states, and that label stream is the expensive part (lesson M3-T4).

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.