FixTheRobot Explore Robotics / Robot Learning / M10 / M10-L1

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Sim-to-Real Transfer and Robustness · M10-L1 · Intermediate

Randomize the Simulator

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital, robotics bench. Scout's reaching controller was tuned in the simulator, where it swings round the pillar on the tightest path and arrives in about 31 steps without a single crash. On the east-wing floors it crashed in a third of the runs: the real motors are weaker, the floor drags harder, the commands arrive a quarter of a second late and the position readings are noisy. None of that is in the simulator.

The bench lead proposes domain randomization: train the controller on a different simulated world in every episode, drawn from ranges around what we measured. You write the sampler, the coverage check, and finally the function that chooses the ranges from measurements of six floors. The lab trains in simulation and tests on four floors you have not seen. This is a simplified toy study on the simulated Scout; no real robot data is used.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 writes sample_params: draw one world from ranges and uniform random numbers, with the actuator lag as a whole number of steps and both ends of its range possible. Stage 2 writes coverage_check: name the parameters whose real value the randomised simulator can not produce, remembering that a parameter left out of the ranges stays at its nominal value. Stage 3 writes choose_ranges: from six measured floors choose the ranges, after which JavaScript trains a controller in your randomised simulator and tests it on four hidden floors.

    Grading is result-based on floors and measurements you do not see: the trained controller must reach the goal on at least 90 % of the episodes on every hidden floor, and the randomised simulator must stay solvable, so over-wide ranges are penalised as well as missing ones. The last question is about the price of randomization (lesson M10-T2).

    Theory for this lab

    Sim-to-Real Transfer and Robustness labs

    1. M10-L1: Randomize the Simulator: this lab.
    2. M10-L2: Stress-Test a Policy
    3. M10-L3: Detect When the Policy Is Out of Distribution
    4. M10-L4: Plan a Staged Rollout

    Educational simplification: a toy transfer study on Scout with momentum, no real robot data. "Reality" is a floor world and "simulation" the nominal world; training is a grid search over 15 controllers rather than a neural network; the floors come from fixed distributions; ranges are uniform and independent. In this toy even rather narrow, well-informed ranges work, which a real simulator would not forgive as easily.

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.