8.5 — Recovery and Robustness

Learning objectives

After this lesson, you should be able to:

• Run a push test on three execution styles.
• Explain why clean demonstrations do not teach recovery.
• Sweep disturbance size to find the edge of competence.
• Compute success with retries and state the independence caveat.

The one-sentence idea

Recovery needs looking again, and data from off-course states.

Test with disturbances, not only with clean runs.

Success when nothing goes wrong is not enough

Demonstrations show a task done well. The real world pushes back: an object slides, a person bumps the table, a grasp slips. Recovery is noticing that something went wrong and correcting it; robustness is staying successful across a range of disturbances. Both are properties of closed-loop execution (lesson 1.5): a policy that re-observes can correct, one that plays back a plan cannot. Inner Monologue, for instance, adds closed-loop language feedback to planning, from sources such as success detectors, object detectors and humans, and the authors report that this significantly improves instruction completion and lets the planner replan around failures.^[1]

Recovery is also exactly where behavior cloning is weakest (lesson 3.3): the demonstrations rarely contain recoveries, so the states a disturbed robot is in were never labelled. Ross, Gordon and Bagnell's DAgger addresses this by labelling the states the policy itself visits.^[2]

Test it: push the robot mid-task

The simplest robustness test is a disturbance at a fixed time. In the experiment, a hand is moving to a goal and at step 5 it is pushed sideways by some distance. Three controllers: one that replays a plan, one that looks at every step, and one that looks every 5 steps (lesson 4.4):

import numpy as np

def run(mode, push, steps=20, goal=1.0):
    pos, plan, mid = 0.0, goal / steps, None
    for t in range(steps):
        if t == 9: mid = abs(pos - goal)                              # distance from the goal four steps after the push
        if mode == "replay":
            move = plan                                               # never looks
        elif mode == "every step":
            move = 0.4 * (goal - pos)                                 # looks every step
        else:
            if t % 5 == 0: plan_now = (goal - pos) / (steps - t)      # looks every 5 steps and plans the rest evenly
            move = plan_now
        if t == 5: pos += push                                        # the disturbance
        pos += move
    return mid, abs(pos - goal)

print("distance from the goal after a push of 0.3 (four steps later, and at the end):")
for mode in ("replay", "chunks of 5", "every step"):
    mid, end = run(mode, 0.3)
    print(f"  {mode:<12} {mid:.3f}  {end:.3f}")

distance from the goal after a push of 0.3 (four steps later, and at the end):
  replay       0.250  0.300
  chunks of 5  0.250  0.000
  every step   0.055  0.000

The replaying controller ends exactly as far off as it was pushed. The controllers that look again both end on the goal, but at different speeds: four steps after the push the per-step controller is already close (0.055 away) while the chunked one, which has to wait for its next look, is still 0.25 away. How much that delay matters depends on the task, and on how long the robot goes without looking.

Teaching recovery: demonstrations from off-course states

Even a closed-loop learned policy only recovers in states it has seen labelled. A policy cloned from perfect demonstrations has no idea what to do after a push. The remedy in the spirit of lesson 3.3 is to add data from perturbed states, labelled with what the expert would do from there. Using the unstable-car toy of lesson 3.3:

import numpy as np
rng = np.random.default_rng(0)

def expert(e): return -1.8 * e

def drive(policy, noise, steps=40):
    e, seen = 0.0, []
    for _ in range(steps):
        seen.append(e)
        e = 1.5 * e + policy(e) + rng.normal(0, noise)
        if abs(e) > 1.0: return seen, True
    return seen, False

def nearest_policy(states, actions):
    return lambda e: actions[np.argmin(np.abs(states - e))]

def failure_rate(policy, trials=200):
    return float(np.mean([drive(policy, 0.12)[1] for _ in range(trials)]))

clean = np.concatenate([drive(expert, 0.03)[0] for _ in range(30)])                    # the expert drives gently
noisy = np.concatenate([drive(expert, 0.12)[0] for _ in range(30)])                    # the expert is disturbed, but its action is recorded as normal

print("trained on clean demonstrations:     fails in", failure_rate(nearest_policy(clean, expert(clean))), "of runs")
print("trained with disturbed demonstrations: fails in", failure_rate(nearest_policy(noisy, expert(noisy))), "of runs")

trained on clean demonstrations:     fails in 0.315 of runs
trained with disturbed demonstrations: fails in 0.0 of runs

Toy model. The same one-number “car” as lesson 3.3. Injecting disturbances while recording expert labels is one way to cover off-course states; it is not the only one and it needs a safe way to disturb the robot.

Covering states the robot will actually be pushed into turns a brittle policy into a recovering one.

Find the breaking point

Sweep the disturbance size and plot (here, print) success against it. The curve shows how much margin the policy has, and where it breaks:

import numpy as np
rng = np.random.default_rng(0)

def succeeds(push, steps=20, goal=1.0, tol=0.05):
    pos = 0.0
    for t in range(steps):
        pos += 0.4 * (goal - pos) + rng.normal(0, 0.01)
        if t == 5: pos += push
        if abs(pos) > 3.0: return False                    # fell outside the workspace
    return abs(pos - goal) < tol

for push in (0.0, 0.5, 1.0, 2.0, 3.5):
    rate = np.mean([succeeds(push) for _ in range(300)])
    print(f"push of {push:>3}: success {rate:.2f}")

push of 0.0: success 1.00
push of 0.5: success 1.00
push of 1.0: success 1.00
push of 2.0: success 1.00
push of 3.5: success 0.00

Success is high until the push leaves the workspace and then collapses, a sharp edge rather than a gentle slope. Knowing the edge tells you what disturbances the system can be trusted with.

Retries: a simple recovery layer

A system-level recovery: detect failure (a success detector, lesson 7.6 labels) and try again. If a single attempt succeeds with probability p and attempts are independent, k attempts succeed with probability 1 − (1 − p)^k:

p = 0.6
for k in (1, 2, 3, 5):
    print(f"up to {k} attempt(s): success {1 - (1 - p) ** k:.3f}")

up to 1 attempt(s): success 0.600
up to 2 attempt(s): success 0.840
up to 3 attempt(s): success 0.936
up to 5 attempt(s): success 0.990

Independence is the catch: if the failure is systematic (the object is out of reach), retrying repeats it. And a retry needs a reliable failure detector, and a safe way to reset the scene. Inner Monologue's feedback sources play this role at the planning level.^[1]

Common misconceptions

Engineering takeaways

1. Replay cannot recover; looking can.
2. Cover off-course states.
3. Sweep to find the edge.
4. Retries need independence.

Test what happens after the push.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Last in this phase: how model speed limits how often a robot can be commanded.

Back to the VLA roadmap