FixTheRobot Explore Robotics / Robot Learning / M9 / M9-L2

This lab works on phones, but writing code is much easier on a laptop.

Robot Learning · Learning for Manipulation and Locomotion · M9-L2 · Intermediate

Shape a Gait Reward

Experiment log · Scout at Meridian General Hospital

Experiment log, Meridian General Hospital, Scout's legged prototype. The team trains a two-legged walker for the hospital's stairwell corridor. The reward is a weighted sum: speed forward, a penalty for energy, a penalty for falling. The first trained gait was fast, graceful and a disaster: it fell on every run. After the fall penalty was raised, the walker stopped falling and started to shuffle, sliding its feet along the floor, which a camera crew called "the corridor mop".

You will not write learning code. You set the weights; the page searches about a thousand gaits for the one with the highest reward (an exhaustive search standing in for training) and draws what comes out: the height of each foot over a stride, and the speed, fall probability, energy and scuffing of the gait. In the last stage the same weights are tried on robots with shorter strides, stronger legs or worn ankles.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and heapq. Your code runs in a background worker: an endless loop is stopped after a few seconds and the page stays usable. It runs in your browser and is never sent anywhere.

    Hints

    What this lab teaches

    Stage 1 shows that a reward is a set of exchange rates: with a large speed weight and a small fall penalty the best gait is a certain fall, and with the penalty raised the best gait is a shuffle, because nothing in the reward asks for lifted feet. Stage 2 adds the missing scuff term and shows that the gait flips from shuffle to walk only when the penalty exceeds the energy that lifting the feet costs. Stage 3 applies the same weights to the practice robot and to four hidden robot variants and asks for weights that do not depend on one robot.

    Grading uses the gait that the search finds: its kind must be a walk, the speed at least 0.8 m/s, the fall probability at most 0.05, scuffing exactly 0 and the energy at most 9 units (this lab's own acceptance limits). The foot-height plot is the thing to read: a shuffle is two flat lines, a hop is two traces that rise together, a walk is two feet taking turns above the 4 cm line.

    Theory for this lab

    Learning for Manipulation and Locomotion labs

    1. M9-L1: Predict Grasp Success
    2. M9-L2: Shape a Gait Reward: this lab.
    3. M9-L3: Wrap a Policy in a Safety Controller

    Educational simplification: a hand-built model of four gait parameters (step rate, step length, phase between the feet and foot lift) with formulas for speed, fall probability, energy, scuffing and asymmetry. It is not a physics simulator and not a trained policy: "training" is an exhaustive search over a grid of about 1000 gaits, so the result is exactly what the reward allows. The robot variants only scale stride, energy cost and achieved foot lift. The acceptance limits are this lab's own.

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.