7.3 — Running Pretrained Policies

Learning objectives

After this lesson, you should be able to:

• Check a robot against a checkpoint’s expectations.
• Wrap a chunked model in a policy with an action queue.
• Explain why reset matters.
• Detect missed deadlines in a control loop.

The one-sentence idea

A checkpoint is useful only inside a correct wrapper.

Check expectations, reset per episode, watch the rate.

What “running a pretrained policy” involves

Released models come as weights plus a description of what they expect: image names and sizes, state layout, how actions are defined and normalised. LeRobot, for instance, is a library whose datasets store synchronised camera video and tabular state and action data, and in which indexing a dataset returns one time step with observations and an action as tensors,^[1] so a policy and a dataset share a common vocabulary. This course runs in a browser sandbox with no model downloads, so this lesson teaches the structure around a pretrained policy with small stand-ins you can run. The structure is the same for a real checkpoint: you check its expectations, wrap it in a stateful policy object, feed it observations at a fixed rate, and handle chunks.

Step 1: read what the checkpoint expects

Every checkpoint should ship a small description of its inputs and outputs. Before the first call, compare it with what your robot provides. A mismatch caught here costs a line of error text; caught later, it costs a confusing failure on hardware:

checkpoint = {
    "name": "demo-policy",
    "image_keys": {"front": (224, 224, 3), "wrist": (224, 224, 3)},
    "state_dim": 7,
    "action_dim": 7,
    "chunk_length": 10,
}

robot = {"image_keys": {"front": (480, 640, 3)}, "state_dim": 7, "action_dim": 6}

def compatibility(ckpt, robot):
    issues = []
    for key, shape in ckpt["image_keys"].items():
        if key not in robot["image_keys"]:
            issues.append(f"missing camera '{key}'")
        elif robot["image_keys"][key] != shape:
            issues.append(f"camera '{key}' is {robot['image_keys'][key]}, checkpoint expects {shape} (resize needed)")
    for field in ("state_dim", "action_dim"):
        if ckpt[field] != robot[field]:
            issues.append(f"{field}: robot has {robot[field]}, checkpoint has {ckpt[field]}")
    return issues

for issue in compatibility(checkpoint, robot):
    print("-", issue)

- camera 'front' is (480, 640, 3), checkpoint expects (224, 224, 3) (resize needed)
- missing camera 'wrist'
- action_dim: robot has 6, checkpoint has 7

Toy model. A made-up checkpoint description. Real checkpoints carry equivalent information in their configuration files.

Notice that two of the three issues are fixable in software (resize; map the action) while the missing wrist camera is not: you need a different model or camera.

Step 2: a stateful policy with an action queue

Chunk-predicting models (lesson 4.4) give several actions per call, but the robot wants one action per control step. A small wrapper keeps a queue: pop one action per step, and call the model again only when the queue is empty. Resetting between episodes must clear the queue:

import numpy as np

class ChunkedPolicy:
    def __init__(self, model, execute_steps):
        self.model, self.execute_steps, self.queue, self.calls = model, execute_steps, [], 0

    def reset(self):                                   # call at the start of every episode
        self.queue = []

    def select_action(self, observation):
        if not self.queue:                             # out of planned actions: ask the model again
            chunk = self.model(observation)            # shape (chunk_length, action_dim)
            self.calls += 1
            self.queue = list(chunk[: self.execute_steps])
        return self.queue.pop(0)

def toy_model(observation):                            # a stand-in for a real checkpoint
    return np.tile(0.1 * (1.0 - observation), (10, 1))

policy = ChunkedPolicy(toy_model, execute_steps=5)
policy.reset()
obs = 0.0
for step in range(12):
    obs += float(policy.select_action(np.array([obs]))[0])
print("model calls in 12 control steps:", policy.calls)
print("final position:", round(obs, 3))

model calls in 12 control steps: 3
final position: 0.8

12 control steps, 3 model calls. Forgetting reset() is a classic bug: the first step of a new episode would execute leftover actions from the last one.

Step 3: a fixed-rate control loop

Robots need actions at a steady rate. A loop must start each step at the right time and notice when the model made it late. This simulation uses a virtual clock, so it is exact and fast:

rate_hz = 10
period = 1.0 / rate_hz
compute_time = [0.02] * 7 + [0.18] + [0.02] * 7 + [0.25] + [0.02] * 4      # seconds per step; two slow model calls

clock = 0.0
late = []
for i, work in enumerate(compute_time):
    deadline = (i + 1) * period
    clock += work
    if clock > deadline:
        late.append((i, round(clock - deadline, 2)))                         # missed the deadline by this many seconds
        # a real loop must decide here: skip, repeat the last action, or stop
    clock = max(clock, deadline)                                            # wait until the next period if early
print("control steps:", len(compute_time))
print("steps that missed their deadline (index, seconds late):", late)

control steps: 20
steps that missed their deadline (index, seconds late): [(7, 0.08), (8, 0.0), (15, 0.15), (16, 0.07)]

The two slow model calls (steps 7 and 15) each made the next step late too, so four steps missed their deadline (the 0.0 is a rounded tiny delay). What the loop does then is a safety decision, not a detail: repeating the last action, holding still, or stopping are different choices with different consequences (lesson 9.7). Whatever the policy, the loop must be written to handle it explicitly.

Putting it together

The recipe for a real checkpoint: read its configuration, check compatibility with your robot, build preprocessing and decoding that match its training (lessons 4.3 and 7.1), wrap it in a stateful policy with a queue, and run it in a monitored fixed-rate loop. Start in simulation (lesson 7.7) or with the robot held still, and watch the numbers before trusting the motion.

Common misconceptions

Engineering takeaways

1. Compatibility check first.
2. Queue for chunks; reset per episode.
3. Fixed-rate loop.
4. Decide what happens when late.

Wrap it carefully before it moves anything.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: fine-tuning a pretrained model on your own demonstrations.

Back to the VLA roadmap