FixTheRobot Explore Robotics / Robot Learning / M3 / M3-L1

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Imitation Learning · M3-L1 · Intermediate

Clone the Expert

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital, 09:40. The night-shift operators have driven Scout across the hall to the pharmacy goal dozens of times, steering round the pillar, and every run was logged. Two of those runs ended against the pillar and one was stopped halfway. The team wants to know whether those recordings alone are enough to teach a policy to do the job without an operator.

You will turn the logs into a training set, build the simplest policy that imitates them (look at the closest situations in the data and copy what the expert did), and then run the whole thing as an experiment. The question that matters is not whether the clone repeats the demonstrations, but whether it copes with a start nobody drove from.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 is the data step of behaviour cloning (lessons M3-T1 and M3-T2): a demonstration is a sequence of observation and action pairs, the label at step t is the action taken at step t, and the failed episodes in the log are not examples to copy. Stage 2 builds the policy itself, a k-nearest-neighbour regressor, as a table that the simulator deploys closed-loop. Stage 3 is an experiment: change the number of demonstrations and the number of neighbours, and compare the success rate on the starts the clone was trained on with the rate on new starts.

    Grading is result-based. Your dataset is compared with the exact one on hidden logs that include a crash, a cut-short run and one-step episodes. Your tables are compared with the exact nearest-neighbour answer and one of them is deployed from held-out starts. In the experiment, the setting you choose is tested on other recorded demonstrations and other held-out starts.

    Theory for this lab

    Imitation Learning labs

    1. M3-L1: Clone the Expert: this lab.
    2. M3-L2: Watch the Clone Drift
    3. M3-L3: Add DAgger Corrections
    4. M3-L4: Clean the Demonstrations

    Educational simplification: one task, a point robot with direct velocity commands, a noise-free expert, and a nearest-neighbour learner whose policy is deployed as a lookup table on a 0.5 m grid so that Python can return it. A real clone would be a neural network trained on noisy, far larger data; the questions about coverage, held-out starts and what a training score hides apply to it as well (lessons M3-T1 and M3-T2).

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.