FixTheRobot Explore Robotics / Robot Learning / M1 / M1-L3

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Foundations: Policies, Data and Evaluation · M1-L3 · Beginner

Score a Policy Fairly

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital. Two versions of Scout's delivery policy both "work". The dashboard shows 90% for one and 92% for the other, the second came from five training seeds, and someone wants to ship it on Friday.

You will put an honest interval around a success rate, compare two policies on the same starts without throwing away what that pairing gives you, and summarise a handful of training seeds without making them look more certain than they are. Every outcome comes from real rollouts in the simulated hall.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 replaces the textbook normal interval, which collapses to a point at 20 of 20 and leaves the range 0 to 1, with the Wilson score interval. Stage 2 compares two policies that ran on the same starts with a paired bootstrap, and asks what an interval that contains zero allows you to say. Stage 3 summarises one run per training seed with the sample standard deviation and a t interval, and asks how many seeds a given precision needs.

    Grading is result-based: your functions are run on the practice example and on freshly generated hidden test sets, and compared with the exact answer. The random resamples for the bootstrap are generated outside your code and passed in.

    Theory for this lab

    Foundations: Policies, Data and Evaluation labs

    1. M1-L1: Record a Rollout
    2. M1-L2: Learn to Reach the Goal
    3. M1-L3: Score a Policy Fairly: this lab.
    4. M1-L4: Read a Failed Training Run

    Educational simplification: policies of different quality are made by adding noise to the expert's actions or by fitting a small linear model, and the world is the single-pillar hall of the track. Real evaluations also face drifting conditions and many more factors than the starting position.

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.