3.2 — Demonstrations, Trajectories and Robot Datasets

Learning objectives

After this lesson, you should be able to:

• Define demonstration, episode, step and dataset.
• Count the training windows an episode provides.
• Explain why neighbouring steps leak information between train and test.
• Split a dataset by episode.

The one-sentence idea

A dataset is many recorded episodes of observation-action steps.

Neighbouring steps are near-copies, so split by episode.

A demonstration is a short film with controls

A demonstration is a recording of a task being done, usually by a person steering the robot (teleoperation). At every moment it stores what the robot observed and what the person commanded. One whole attempt, from start to finish, is an episode or trajectory; each moment inside it is a step. A dataset is a pile of episodes, usually with a language instruction attached to each.

The numbers are large because robots are slow to teach. RT-1 was trained on about 130,000 episodes covering more than 700 tasks, collected by 13 robots over 17 months.^[1] OpenVLA used 970,000 real-world robot demonstrations.^[2] The Open X-Embodiment collaboration pooled more than a million trajectories from 22 robot embodiments.^[3]

How an episode is stored

Tools differ in details, but the idea is the same. RLDS, for instance, is a toolkit for recording, replaying, editing and sharing data for sequential decision making, built around a standardised, lossless format.^[4] LeRobot’s dataset format stores synchronised camera video and tabular state and action data, and indexing it gives one step.^[5] Here is the shape in plain Python:

episode = {
    "instruction": "move the hand to the goal",
    "steps": [
        {"observation": {"hand": [0.00, 0.00], "goal": [0.6, 0.4]}, "action": [0.12, 0.08]},
        {"observation": {"hand": [0.12, 0.08], "goal": [0.6, 0.4]}, "action": [0.12, 0.08]},
        {"observation": {"hand": [0.24, 0.16], "goal": [0.6, 0.4]}, "action": [0.12, 0.08]},
    ],
}
print("instruction:", episode["instruction"])
print("steps in this episode:", len(episode["steps"]))
first = episode["steps"][0]
print("first observation:", first["observation"])
print("first action:", first["action"])

instruction: move the hand to the goal
steps in this episode: 3
first observation: {'hand': [0.0, 0.0], 'goal': [0.6, 0.4]}
first action: [0.12, 0.08]

Toy model. A tiny episode written by hand, not any real dataset’s schema.

Everything a policy ever learns comes from rows like these: “given this observation (and instruction), the demonstrator did this action.”

From episodes to training samples

Training takes a step (or a short history of steps) as the input and the action, or the next few actions, as the target. Counting what you get from an episode is simple arithmetic:

episodes = 200
steps_per_episode = 150
history = 2                    # the policy sees the last 2 observations
chunk = 10                     # and predicts the next 10 actions

per_episode = steps_per_episode - history + 1 - (chunk - 1)     # windows that fit fully inside the episode
print("training windows per episode:", per_episode)
print("training windows in the dataset:", episodes * per_episode)
print("plain single steps in the dataset:", episodes * steps_per_episode)

training windows per episode: 140
training windows in the dataset: 28000
plain single steps in the dataset: 30000

Notice that the windows overlap: two neighbouring ones differ by one step. That has a nasty consequence for evaluation, shown next.

Split by episode, not by step

To know whether a policy really learned the task, you test it on data it did not train on. But neighbouring steps in an episode are almost identical, so if you split at random by step, the test steps have near-twins in the training set and the score looks great. The honest split keeps whole episodes apart. The experiment below uses the simplest possible learner, “find the most similar training step and copy its action”:

import numpy as np
rng = np.random.default_rng(0)

def make_episode():
    goal = rng.uniform(-1, 1, 2)
    hand = np.zeros(2)
    obs, act = [], []
    for _ in range(20):
        a = 0.15 * (goal - hand) / max(np.linalg.norm(goal - hand), 1e-6) + rng.normal(0, 0.01, 2)   # the demonstrator
        obs.append(np.concatenate([hand, goal])); act.append(a)
        hand = hand + a
    return np.array(obs), np.array(act)

episodes = [make_episode() for _ in range(40)]
ep_id = np.repeat(np.arange(40), 20)
X = np.concatenate([e[0] for e in episodes]); Y = np.concatenate([e[1] for e in episodes])

def nn_error(train, test):
    errs = []
    for i in test:
        j = train[np.argmin(np.linalg.norm(X[train] - X[i], axis=1))]    # the most similar training step
        errs.append(np.linalg.norm(Y[j] - Y[i]))
    return float(np.mean(errs))

all_steps = rng.permutation(len(X))
step_train, step_test = all_steps[:640], all_steps[640:]             # random split by step
ep_train = np.where(ep_id < 32)[0]; ep_test = np.where(ep_id >= 32)[0]   # whole episodes held out

print("error with a step-level split:   ", round(nn_error(step_train, step_test), 3))
print("error with an episode-level split:", round(nn_error(ep_train, ep_test), 3))

error with a step-level split:    0.036
error with an episode-level split: 0.132

Toy model. Made-up reaching task and a nearest-neighbour “policy”. The size of the gap depends on the task; the direction of the effect, leakage between neighbouring steps, is general.

The step-level split reports an error of 0.036 against 0.132 for held-out episodes, more than three times smaller, but it is measuring memory, not skill. For robot data, always hold out whole episodes, and ideally whole scenes or setups.

Common misconceptions

Engineering takeaways

1. Dataset ⊃ episodes ⊃ steps.
2. Each step holds an observation and an action.
3. Windows overlap.
4. Hold out whole episodes.

Test on episodes the model has never seen.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: behavior cloning, the simplest way to learn a policy from these demonstrations.

Back to the VLA roadmap