FixTheRobot Explore Robotics / Robot Learning / M3 / M3-L4

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Imitation Learning · M3-L4 · Intermediate

Clean the Demonstrations

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital, 15:10. Two operators taught Scout the delivery run, and both are good. Near the pillar one of them goes round it on the left and the other on the right. The clone trained on both logs freezes in the middle and then drives at the pillar, although every single demonstration in the dataset is a success.

You will find the conflict in the data rather than guess it, rebuild the dataset by keeping one operator's way round, and check the result by running a clone trained on your data in the hall. Then you will look at the vertical commands as classes instead of numbers, and see what the average was hiding.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 turns "the operators disagree" into a measurement: the pairs of demonstrations whose observations are close, whose vertical actions have opposite signs, and whose actions are both decisive. The magnitude rule separates a real disagreement from two near-zero actions that both mean "go straight". Stage 2 is the repair. The unit of curation is the episode, not the step: an episode is kept or dropped as a whole according to the side on which it passes the pillar, and a run that never passes the pillar is kept because it conflicts with nothing. The simulator fits a clone on your cleaned data and deploys it closed-loop from held-out starts near the pillar line. Stage 3 bins the vertical action and takes a majority vote, which is the simplest way to make a policy choose a mode instead of averaging two.

    Grading is result-based: your pairs and your cleaned data are compared with the exact ones, and the clone fitted on your data has to reach the goal. The two operators and the held-out starts stay in JavaScript.

    Theory for this lab

    Imitation Learning labs

    1. M3-L1: Clone the Expert
    2. M3-L2: Watch the Clone Drift
    3. M3-L3: Add DAgger Corrections
    4. M3-L4: Clean the Demonstrations: this lab.

    Educational simplification: the two operators are scripted variants of the shared expert that always swing to a chosen side of the pillar (the shared expert swings to the side the robot is on, so it never disagrees with itself); the clone is a ridge regression on the goal direction and the closeness of the pillar, features that do not contain which side Scout is on. A richer policy that sees its side can resolve part of the ambiguity from its own position, so real mixed datasets hurt less in this world than this experiment shows, and more in tasks where nothing in the observation separates the modes (lessons M3-T5 and M3-T6).

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.