6.2 — Octo

Learning objectives

After this lesson, you should be able to:

• State Octo’s training data and instruction types.
• Explain how one policy can serve robots with different sensors.
• Explain why training only a small part can be enough to adapt.
• Place Octo relative to OpenVLA.

The one-sentence idea

A pretrained generalist that is meant to be adapted.

Flexible inputs, flexible goals, cheap adaptation.

A policy designed to be adapted

Octo’s goal was not the largest model but a reusable starting point. The Octo team describes a large transformer-based policy trained on 800,000 trajectories from the Open X-Embodiment dataset, which can be instructed with language commands or goal images.^[1] The pitch: a useful generalist policy must handle diverse sensors and action spaces, accommodate commonly used robotic platforms, and finetune readily to new domains. The authors report that Octo can be effectively finetuned to robot setups with new sensory inputs and action spaces within a few hours on standard consumer GPUs, tested across 9 robotic platforms.^[1] For a guided tour of the paper itself, with its experiments, a re-check of its tables and a critical review, see the Octo paper guide.

That is a different emphasis from OpenVLA (lesson 6.1): Octo takes language or goal images, and puts the weight on adaptation to new robots. Three ideas from that design are worth seeing in code.

Idea 1: one policy, two ways to say what you want

A goal can be a sentence (“put the spoon in the bowl”) or a picture of the finished scene. If both are turned into the same kind of goal vector, the same policy can use either. In the toy, a “goal image” is a picture of where the hand should end up and a “language goal” is a place name; both are mapped to a target position vector before the policy sees them:

import numpy as np

places = {"left shelf": np.array([-0.8, 0.5]), "right shelf": np.array([0.8, 0.5]), "table": np.array([0.0, -0.4])}

def goal_from_language(text):
    return places[text]                                    # a (very small) language encoder

def goal_from_image(image):
    y, x = np.unravel_index(image.argmax(), image.shape)   # a (very small) image encoder: find the bright spot
    return np.array([x / 9 * 2 - 1, 1 - y / 9 * 2])

def policy(hand, goal):                                    # the same policy for both
    return 0.3 * (goal - hand)

hand = np.array([0.0, 0.0])
image = np.zeros((10, 10)); image[2, 8] = 1.0              # a picture with a bright spot at the top right

print("language goal 'left shelf' -> action", np.round(policy(hand, goal_from_language("left shelf")), 2).tolist())
print("image goal (bright spot top right) -> action", np.round(policy(hand, goal_from_image(image)), 2).tolist())

language goal 'left shelf' -> action [-0.24, 0.15]
image goal (bright spot top right) -> action [0.23, 0.17]

Toy model. Two tiny hand-written encoders that map into one goal vector. Real encoders are learned and the goal enters the transformer as tokens.

The point is the shared interface: whatever encodes the goal, the policy only needs the goal vector.

Idea 2: different robots have different sensors

One robot has two cameras, another has one camera and a wrist camera, a third adds touch. A transformer reads a sequence of tokens, so it can accept a different number of tokens per robot, as long as the missing ones are handled. A mask tells attention which tokens are real:

import numpy as np
rng = np.random.default_rng(0)

d = 8
def tokens_for(robot):                                      # each robot provides its own observation tokens
    return {"robot A (2 cameras)": 2, "robot B (1 camera + wrist)": 2, "robot C (1 camera)": 1}[robot]

max_tokens = 3
for robot in ("robot A (2 cameras)", "robot C (1 camera)"):
    n = tokens_for(robot)
    X = np.zeros((max_tokens, d)); X[:n] = rng.normal(size=(n, d))        # real tokens, then padding
    mask = np.array([True] * n + [False] * (max_tokens - n))
    scores = X @ X.T / np.sqrt(d)
    scores[:, ~mask] = -1e9                                                # nobody attends to padding
    w = np.exp(scores - scores.max(axis=1, keepdims=True)); w /= w.sum(axis=1, keepdims=True)
    print(robot, "-> attention paid to padding:", float(np.round(w[:, ~mask].sum(), 6)))

robot A (2 cameras) -> attention paid to padding: 0.0
robot C (1 camera) -> attention paid to padding: 0.0

With the mask, padding gets zero attention, so the same network can serve robots that observe different things. This is one way to build the flexibility the Octo authors ask of a generalist policy.

Idea 3: adapt cheaply, keep what was learned

A new robot has a different action space (say 3 numbers instead of 2). Adapting means keeping most of the pretrained network and training a small piece, such as a new output layer, on a few demonstrations of the new robot. In the toy, pretrained “features” compress the observation, and only a new linear output layer is trained:

import numpy as np
rng = np.random.default_rng(0)

d_in, d_feat = 40, 4
true_features = rng.normal(size=(d_in, d_feat))                       # the structure that "pretraining" discovered
new_robot_map = rng.normal(size=(d_feat, 3))                           # the new robot: 3 action numbers

def demos(n):
    X = rng.normal(size=(n, d_in))
    return X, (X @ true_features) @ new_robot_map + rng.normal(0, 0.05, (n, 3))

Xt, Yt = demos(3000)
for n in (8, 20, 100):
    X, Y = demos(n)
    # from scratch: learn 40 -> 3 directly
    W_scratch = np.linalg.lstsq(X.T @ X + 1e-3 * np.eye(d_in), X.T @ Y, rcond=None)[0]
    # adapt: reuse the pretrained 40 -> 4 features, learn only the new 4 -> 3 output layer
    F = X @ true_features
    W_head = np.linalg.lstsq(F.T @ F + 1e-3 * np.eye(d_feat), F.T @ Y, rcond=None)[0]
    e_s = np.mean((Xt @ W_scratch - Yt) ** 2); e_h = np.mean((Xt @ true_features @ W_head - Yt) ** 2)
    print(f"{n:>4} demos: from scratch {e_s:8.4f} | new head on pretrained features {e_h:.4f}")

   8 demos: from scratch 104.3997 | new head on pretrained features 0.0041
  20 demos: from scratch  57.4402 | new head on pretrained features 0.0036
 100 demos: from scratch   0.0043 | new head on pretrained features 0.0027

Toy model. The pretrained features are handed over exactly, which is optimistic. The shape of the result, a few demonstrations suffice when only a small part is learned, is the argument for adaptable policies.

In practice more than the head may be fine-tuned, and the Octo paper reports its own protocol and results; the toy only illustrates the logic.

Where Octo sits

Octo is a generalist policy, trained on robot data; unlike RT-2 and OpenVLA it is not described here as starting from a pretrained vision-language model, so its language understanding is not that of a web-scale model. That is a trade-off worth keeping in mind when you compare models in lesson 6.6.

Common misconceptions

Engineering takeaways

1. 800k Open X-Embodiment trajectories.
2. Language or goal images.
3. Fine-tunes in hours on consumer GPUs (per the paper).
4. Masks handle missing inputs.

Octo: pretrain once, adapt anywhere.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: π0 and π0.5, which generate continuous actions with flow matching.

Back to the VLA roadmap