4.4 — Temporal Action Prediction and Action Chunking

Learning objectives

After this lesson, you should be able to:

• Define an action chunk and receding-horizon execution.
• Explain how chunking shortens the effective horizon.
• Explain temporal ensembling and what it costs.
• Name the trade-off in choosing the chunk length.

The one-sentence idea

Predict a short plan, execute part of it, look again.

Fewer decisions mean less accumulated error, but slower reactions.

Plan a few steps, not one

A simple policy answers “what do I do now?” at every time step. Action chunking changes the question to “what do I do over the next k steps?”. The policy outputs a short sequence, the robot executes some or all of it, and then the policy looks again. ACT introduced this for robot imitation: it shortens the effective horizon by predicting actions in chunks, and with it a low-cost bimanual system learned tasks such as opening a translucent condiment cup with 80–90% success from 10 minutes of demonstrations.^[1] Diffusion Policy uses the same pattern under another name, receding-horizon control: part of a predicted sequence is executed, then the policy is queried again.^[2]

Why a shorter effective horizon helps

Every time the policy decides, it can make a small mistake, and (lesson 1.1) mistakes accumulate. If a task takes 400 control steps and the policy decides at each one, there are 400 chances to drift. If it decides once per chunk of 20, there are 20. A rough picture, assuming each decision adds an independent error of the same size:

import numpy as np

task_steps = 400
error_per_decision = 0.01                                # the typical size of one decision's mistake
for chunk in (1, 5, 20, 100):
    decisions = task_steps // chunk
    total = error_per_decision * np.sqrt(decisions)      # independent errors add in variance
    print(f"chunk {chunk:>3}: {decisions:>3} decisions -> typical accumulated error {total:.3f}")

chunk   1: 400 decisions -> typical accumulated error 0.200
chunk   5:  80 decisions -> typical accumulated error 0.089
chunk  20:  20 decisions -> typical accumulated error 0.045
chunk 100:   4 decisions -> typical accumulated error 0.020

Toy model. A rule of thumb assuming independent errors of equal size. Real errors correlate, and a longer chunk is itself harder to predict accurately, so this is half the story. The other half is next.

The other half: a longer chunk means the robot goes longer without looking, so it reacts more slowly to surprises (lesson 1.5). The chunk length trades fewer decisions against slower reactions.

Overlapping chunks and smoothing

One more trick is to ask for a new chunk at every step. Then at any moment several chunks, made at different times, each contain a prediction for the current step. Averaging them is called temporal ensembling, which ACT uses together with chunking to reduce compounding errors.^[1] Each prediction has some noise, and averaging reduces it. Compare executing only the newest prediction with averaging all that cover the current step (the weights here are our own simple choice):

import numpy as np
rng = np.random.default_rng(0)

T, k = 100, 10
truth = np.sin(np.linspace(0, 3 * np.pi, T + k))                       # the motion we want
# at each time t the policy predicts the next k steps, with some prediction noise
chunks = [truth[t:t + k] + rng.normal(0, 0.05, k) for t in range(T)]

newest_only = np.array([chunks[t][0] for t in range(T)])               # use only the first step of the newest chunk
ensembled = np.zeros(T)
for t in range(T):
    preds, weights = [], []
    for age in range(min(k, t + 1)):                                   # chunks started age steps ago
        preds.append(chunks[t - age][age]); weights.append(np.exp(-0.3 * age))
    ensembled[t] = np.average(preds, weights=weights)

def jitter(x):  return float(np.abs(np.diff(x, 2)).mean())
print("jitter, newest chunk only:", round(jitter(newest_only), 4))
print("jitter, ensembled:        ", round(jitter(ensembled), 4))
print("distance from the true motion:", round(float(np.abs(newest_only - truth[:T]).mean()), 4), "->", round(float(np.abs(ensembled - truth[:T]).mean()), 4))

jitter, newest chunk only: 0.1038
jitter, ensembled:         0.0372
distance from the true motion: 0.0383 -> 0.0149

Toy model. Noisy predictions of a sine wave and exponential weights chosen by us. ACT’s exact weighting is described in its paper.

The ensembled motion is smoother and closer to the truth. The price is a slight lag, since older predictions carry weight, and a model call at every step. Without ensembling, a robot that executes one chunk and then switches to the next can jump at the boundary, because two separate predictions rarely agree exactly.

Common misconceptions

Engineering takeaways

1. Chunk = k future actions.
2. Receding horizon: execute part, then re-plan.
3. Ensembling averages overlapping predictions.
4. Choose k for the task and the hardware.

Think a few steps ahead, then look again.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: ACT, the policy that made action chunking popular.

Back to the VLA roadmap