11.5 — Hierarchical Planning and Task Decomposition

Learning objectives

After this lesson, you should be able to:

• Compute success over a chain of steps.
• Show how per-step retries change the result.
• Write a planner that tracks facts.
• Describe language as a middle layer.

The one-sentence idea

Plan in steps, check each step.

Small per-step failure rates compound; checks and retries stop the compounding.

Long tasks are many short tasks

“Clean the kitchen” is not one skill; it is dozens of steps over minutes. A single policy that maps images and an instruction to motor commands is poorly matched to such horizons. The common answer is a hierarchy: a high level decides what to do next in an abstract form, a low level carries out each step. You have met pieces of this already. SayCan combines a language model's knowledge of what is useful with value functions that say what is feasible, choosing a skill that is both and repeating until the output is a terminate step.^[1] Code as Policies has language models write policy code that calls perception and control APIs.^[2] Inner Monologue adds closed-loop language feedback so the planner can replan around failures.^[3] π0.5's training examples include semantic subtask prediction alongside low-level actions,^[4] and Gemini Robotics builds its VLA on top of an embodied-reasoning model.^[5]

RT-H makes the middle layer explicit and human-readable: it uses language as a layer between tasks and actions. The robot predicts fine-grained language motions such as “move arm forward” from the task, then predicts the action from the task, the vision and the predicted motion; because the middle layer is language, a person can intervene during execution with a new language motion.^[6]

Why hierarchy is not just tidiness: long chains multiply risk

If each of n steps succeeds with probability p independently, the whole task succeeds with pⁿ. A good-looking per-step rate collapses over a long task:

for p in (0.99, 0.95, 0.90):
    row = "  ".join(f"{n:>2} steps: {p ** n:5.2f}" for n in (1, 5, 10, 20, 40))
    print(f"per-step success {p:.2f} -> {row}")

per-step success 0.99 ->  1 steps:  0.99   5 steps:  0.95  10 steps:  0.90  20 steps:  0.82  40 steps:  0.67
per-step success 0.95 ->  1 steps:  0.95   5 steps:  0.77  10 steps:  0.60  20 steps:  0.36  40 steps:  0.13
per-step success 0.90 ->  1 steps:  0.90   5 steps:  0.59  10 steps:  0.35  20 steps:  0.12  40 steps:  0.01

At 95% per step, a 20-step task succeeds a bit over a third of the time. Two things help: raise the per-step rate, and add a check-and-retry at each step, which a hierarchy makes possible because it knows what each step was meant to achieve (lesson 8.5):

p, n = 0.90, 20
for attempts in (1, 2, 3):
    per_step = 1 - (1 - p) ** attempts
    print(f"up to {attempts} attempt(s) per step -> per-step success {per_step:.3f} -> whole {n}-step task {per_step ** n:.3f}")

up to 1 attempt(s) per step -> per-step success 0.900 -> whole 20-step task 0.122
up to 2 attempt(s) per step -> per-step success 0.990 -> whole 20-step task 0.818
up to 3 attempt(s) per step -> per-step success 0.999 -> whole 20-step task 0.980

Toy model. Assumes independent attempts and a reliable success check, both of which can fail in practice (lesson 8.5).

With three attempts per step, a task that would succeed 12% of the time succeeds about 98% of the time, provided failures are independent and detectable.

Decomposing, and tracking what is done

A high-level planner needs a representation of the task: steps, what each needs before it can start and what it changes. A simple symbolic version tracks a set of true facts and picks the next step whose requirements are met:

steps = {
    "open the drawer":        {"needs": set(),                          "adds": {"drawer open"}},
    "pick up the cup":        {"needs": set(),                          "adds": {"holding cup"}},
    "put the cup in the drawer": {"needs": {"drawer open", "holding cup"}, "adds": {"cup in drawer"}, "removes": {"holding cup"}},
    "close the drawer":       {"needs": {"cup in drawer"},               "adds": {"drawer closed"}, "removes": {"drawer open"}},
}
facts, done, order = set(), set(), []
while len(done) < len(steps):
    ready = [s for s in steps if s not in done and steps[s]["needs"] <= facts]
    if not ready: break
    step = ready[0]
    facts |= steps[step]["adds"]; facts -= steps[step].get("removes", set()); done.add(step); order.append(step)
print("order found:", order)
print("facts at the end:", sorted(facts))

order found: ['open the drawer', 'pick up the cup', 'put the cup in the drawer', 'close the drawer']
facts at the end: ['cup in drawer', 'drawer closed']

The planner never mentions joint angles; the low level (a VLA or controller) does. The fact set is what lets it notice that a step was not achieved and replan.

Language as the middle layer, and why people can correct it

With a language middle layer, a failed step can be fixed by changing a few words. Our toy has a table from language motions to small movements. The “policy” looks up each motion; a person corrects one motion in the middle and the result changes:

import numpy as np

moves = {"move arm forward": np.array([0.1, 0.0]), "move arm left": np.array([0.0, 0.1]), "move arm right": np.array([0.0, -0.1]), "move arm back": np.array([-0.1, 0.0])}

def run(plan):
    pos = np.zeros(2)
    for word in plan: pos = pos + moves[word]
    return np.round(pos, 2)

plan = ["move arm forward", "move arm forward", "move arm left", "move arm forward"]
goal = np.array([0.3, -0.1])
print("plan:", plan)
print("ends at", run(plan).tolist(), "-> goal", goal.tolist())
plan[2] = "move arm right"                                       # a person replaces one language motion
print("after correcting step 3 to 'move arm right': ends at", run(plan).tolist())

plan: ['move arm forward', 'move arm forward', 'move arm left', 'move arm forward']
ends at [0.3, 0.1] -> goal [0.3, -0.1]
after correcting step 3 to 'move arm right': ends at [0.3, -0.1]

Toy model. A lookup table standing in for a learned model. In RT-H the language motions are predicted by a model and condition the action prediction.

Correcting in language is much easier than teleoperating a correction, and the correction can become training data. The RT-H abstract presents this as a way to learn from human intervention in language.^[6]

Where hierarchies break

Interfaces between levels are failure points: the planner may ask for something the low level has no skill for (SayCan's feasibility scores exist to prevent that^[1]); the low level may fail without the planner knowing (hence feedback); and a chain of levels adds latency (lesson 8.6). A hierarchy also needs its middle layer to be expressive enough: if “move arm forward” cannot describe what is needed, no amount of correction helps.

Common misconceptions

Engineering takeaways

1. p^n collapses.
2. Retries need independence and detection.
3. Track facts to replan.
4. Language layers are correctable.

Make every step checkable.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: what it takes to trust a learned policy to run on its own.

Back to the VLA roadmap