4.2 — Discrete Action Tokens vs. Continuous Actions

Learning objectives

After this lesson, you should be able to:

• Explain how actions become tokens.
• Estimate the quantisation error of N bins.
• Count the tokens needed for a chunk of actions.
• Explain why a token distribution can represent two valid options.

The one-sentence idea

Tokens reuse language machinery; continuous outputs keep precision and speed.

Each choice trades resolution, length and ambiguity differently.

Words or numbers?

A language model’s native output is a choice from a list: the next token. To make a VLA from one, the cheapest trick is to turn actions into tokens. RT-1 discretises each action dimension into 256 bins,^[1] and OpenVLA does the same, writing the bins over 256 rarely used tokens of the language model’s vocabulary and training with ordinary next-token prediction.^[2] The alternative is to predict real-valued vectors directly. π0, for example, pairs a pretrained vision-language model with a separate action expert that generates action chunks with flow matching.^[3]

Neither is “right”. This lesson compares them on three practical points: precision, length and ambiguity.

Precision: bins have a resolution

With N bins over a range, an action can only be one of N values, so a smooth motion becomes a staircase. Here is a smooth 1-second joint motion, quantised with different numbers of bins:

import numpy as np

t = np.linspace(0, 1, 200)
trajectory = 0.4 * np.sin(2 * np.pi * t)                 # a smooth motion between -0.4 and 0.4 radians
low, high = -0.5, 0.5

def quantise(x, bins):
    idx = np.minimum(((x - low) / (high - low) * bins).astype(int), bins - 1)
    return low + (idx + 0.5) / bins * (high - low)

for bins in (8, 32, 256):
    err = np.abs(quantise(trajectory, bins) - trajectory)
    print(f"{bins:>4} bins: largest error {err.max():.4f} rad, steps visible: {len(np.unique(np.round(quantise(trajectory, bins), 6)))} different values")

   8 bins: largest error 0.0625 rad, steps visible: 8 different values
  32 bins: largest error 0.0156 rad, steps visible: 26 different values
 256 bins: largest error 0.0020 rad, steps visible: 160 different values

256 bins already make the error small for this motion. But note the cost hidden here: this is one dimension at one moment in time.

Length: every number is a token

If every number is a token, the number of tokens is dimensions × time steps. A language-model-style policy produces them one after another, so long action chunks get expensive. The FAST paper describes the problem: simple per-dimension, per-timestep binning schemes typically perform poorly when learning dexterous skills from high-frequency robot data.^[4] A back-of-envelope calculation shows the size of it:

dims = 14                      # two 7-number arms, as in ACT
for chunk_steps in (1, 10, 50, 100):
    tokens = dims * chunk_steps
    print(f"{chunk_steps:>3} steps x {dims} dims = {tokens:>4} tokens, so {tokens} sequential model steps if tokens are generated one by one")

  1 steps x 14 dims =   14 tokens, so 14 sequential model steps if tokens are generated one by one
 10 steps x 14 dims =  140 tokens, so 140 sequential model steps if tokens are generated one by one
 50 steps x 14 dims =  700 tokens, so 700 sequential model steps if tokens are generated one by one
100 steps x 14 dims = 1400 tokens, so 1400 sequential model steps if tokens are generated one by one

Toy model. The count is plain arithmetic for one-token-per-number binning. Real systems also use different tokenizers (lesson 4.8) or generate the numbers in parallel.

Continuous heads avoid this: one forward pass can output a whole chunk. The price is a more complicated output layer, and training method (lessons 4.6 and 4.7).

Ambiguity: a list can say “either”

Recall lesson 3.4: if half the demonstrations go left (−1) and half right (+1), a regression trained with squared error predicts 0, straight into the obstacle. A discrete output assigns probability to each bin, so it can hold both options and you sample one of them:

import numpy as np
rng = np.random.default_rng(0)

demos = np.array([-1.0] * 50 + [1.0] * 50)
regression_answer = demos.mean()                         # the best squared-error answer

bins = np.linspace(-1, 1, 5)                             # five possible actions: -1, -0.5, 0, 0.5, 1
counts = np.array([np.sum(np.isclose(demos, b)) for b in bins])
probs = counts / counts.sum()                            # what a classifier would learn
samples = rng.choice(bins, size=1000, p=probs)

print("regression predicts:", regression_answer)
print("token probabilities:", probs.tolist())
print("share of sampled actions that avoid the middle:", float(np.mean(np.abs(samples) == 1.0)))

regression predicts: 0.0
token probabilities: [0.5, 0.0, 0.0, 0.0, 0.5]
share of sampled actions that avoid the middle: 1.0

The generative continuous methods (diffusion and flow matching, lessons 4.6 and 4.7) solve the same problem differently: they sample a possible action instead of computing an average. Diffusion Policy’s authors credit its formulation with handling multimodal action distributions.^[5]

A short comparison

Tokens: reuse the language-model machinery unchanged, handle ambiguity naturally; resolution is limited by the number of bins and a chunk costs many tokens. Continuous outputs: precise and fast to produce for a whole chunk; need a generative or regression head and a matching training method. Real VLAs sit on both sides, and OpenVLA and π0 are a good pair to compare (lessons 6.1 and 6.3).

Common misconceptions

Engineering takeaways

1. 256 bins: fine resolution for coarse motions.
2. Tokens = dims × steps.
3. Distributions can hold several options.
4. Continuous heads output a chunk in one pass.

Words are easy for language models; numbers are easy for robots.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: normalising actions for training and decoding model outputs back into robot commands.

Back to the VLA roadmap