FixTheRobot Explore Robotics / Robot Learning / M1 / M1-L2

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Foundations: Policies, Data and Evaluation · M1-L2 · Beginner

Learn to Reach the Goal

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital, 16:20. The expert controller reaches the goal every time, but it needs the full map and a hand-written avoidance rule. The team wants a small learned policy that Scout can run on its own: observation in, velocity command out.

You will fit that policy from demonstrations, then deploy it. The simulator rolls it out from starts it was never trained on and reports what happens, so each stage is an experiment you can see: first the fit, then features that fail, then features that work.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 is the learning step itself: ridge regression solved in closed form, with the bias left unpenalised. Stage 2 chooses the features. A linear map from raw offsets cannot reproduce a command that depends on direction and saturates, so the policy fails when deployed, even though the fit to the training data looks reasonable. Stage 3 adds the pillar: a policy can only represent what its features allow, and held-out starts close to the pillar expose what a training score hides.

    Grading is result-based. Your fit is compared with the exact solution on hidden datasets. For the feature stages, the simulator fits a policy to your features and deploys it from held-out starts you never see; the success rate decides.

    Theory for this lab

    Foundations: Policies, Data and Evaluation labs

    1. M1-L1: Record a Rollout
    2. M1-L2: Learn to Reach the Goal: this lab.
    3. M1-L3: Score a Policy Fairly
    4. M1-L4: Read a Failed Training Run

    Educational simplification: one task, a point robot with direct velocity commands, a noise-free expert, and a linear policy with hand-designed features. Real policies are neural networks trained on far more data, but the same questions about features, data coverage and held-out evaluation apply (lessons M1-T1 and M1-T4).

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.