3.1 — Robot Observations, Actions and State Representations

Learning objectives

After this lesson, you should be able to:

• Distinguish state, observation and action.
• Explain why a single frame can hide information and how history helps.
• Explain why the same end-effector action needs different joint motions.
• Normalise and de-normalise actions with dataset statistics.

The one-sentence idea

The policy never sees the state, only observations of it, and it answers with an action.

Know what each number means, and in which units and frame.

Three words that are easy to mix up

State is the full truth about the world and the robot at one moment: where every joint is, where every object is, how fast things move. A robot never gets it. Observation is what the robot actually receives: camera images, joint readings, maybe a force sensor. And action is what the policy sends back: the command for the next moment.

The gap between state and observation is a standard part of the theory: when the state can only be observed indirectly, choosing actions is modelled as a partially observable Markov decision process (POMDP).^[1] Robot sensing is also noisy, and probabilistic robotics treats both sensing and motion as uncertain.^[2] In data, the three words meet in one record: indexing a LeRobotDataset returns a single time step with observations and an action as PyTorch tensors.^[3]

One picture is not the whole state

A single camera frame shows where things are, not how fast they are moving. Here a ball rolls along a line. One observation gives only its position (with sensor noise). The policy wants to know the velocity too, to catch it. Two frames in a row make velocity visible:

import numpy as np
rng = np.random.default_rng(0)

true_velocity = 0.5                                    # metres per second (the hidden part of the state)
dt = 0.1
positions = np.arange(30) * true_velocity * dt         # where the ball really is
camera = positions + rng.normal(0, 0.005, size=30)     # what the camera reports (noisy position only)

one_frame = camera[10]
print("one frame says the ball is at", round(float(one_frame), 3), "m, but says nothing about speed")

two_frames = (camera[10] - camera[9]) / dt
ten_frames = np.polyfit(np.arange(10) * dt, camera[:10], 1)[0]
print("two frames estimate the speed as", round(float(two_frames), 2), "m/s")
print("ten frames estimate the speed as", round(float(ten_frames), 2), "m/s (true value 0.5)")

one frame says the ball is at 0.497 m, but says nothing about speed
two frames estimate the speed as 0.53 m/s
ten frames estimate the speed as 0.5 m/s (true value 0.5)

Toy model. A ball on a line and a made-up noise level. The point is that the observation is a partial, noisy view of the state, and that a short history recovers what a single frame hides.

This is why many robot policies are given more than the latest image: a few recent frames, or the robot’s own joint readings, along with the picture.

What an action can mean

The word “action” hides a design choice. Two common ones: a joint command (move joint 1 by this much, joint 2 by that much) or an end-effector command (move the gripper 1 cm to the right). They are connected by the arm’s geometry, which you met in the kinematics lesson. The same end-effector command needs different joint motions depending on the arm’s pose:

import numpy as np

L1, L2 = 1.0, 1.0                                       # link lengths of a two-joint arm in a plane

def hand(q):
    return np.array([L1*np.cos(q[0]) + L2*np.cos(q[0]+q[1]),
                     L1*np.sin(q[0]) + L2*np.sin(q[0]+q[1])])

def joint_change_for(q, delta_xy, eps=1e-6):
    J = np.column_stack([(hand(q + eps*np.eye(2)[i]) - hand(q)) / eps for i in range(2)])   # how the hand moves per joint
    return np.linalg.solve(J, delta_xy)

want = np.array([0.01, 0.0])                            # "move the hand 1 cm in x"
for name, q in (("arm bent", np.array([0.3, 1.8])), ("arm nearly straight", np.array([0.3, 0.5]))):
    dq = joint_change_for(q, want)
    print(name, "-> joint change (degrees):", np.round(np.degrees(dq), 2).tolist())

arm bent -> joint change (degrees): [-0.3, -0.27]
arm nearly straight -> joint change (degrees): [0.83, -1.97]

The command “1 cm in x” is the same in both cases, but the joints must move differently. An end-effector action is easier to share between different robots; a joint action is closer to the motors. Which one a dataset records is part of its definition, and a policy trained on one cannot be fed the other without conversion.

Putting numbers on a common scale

Joint angles may range over a few radians, gripper openings over a few centimetres. Models train much more easily when all inputs and outputs have similar sizes, so datasets are usually normalised with statistics computed from the data. OpenVLA, for example, bins each action dimension between the 1st and 99th quantile of the training actions and de-normalises the decoded actions back to the robot’s physical scale. The version below uses mean and standard deviation, a simpler common choice.^[4][4]

import numpy as np
rng = np.random.default_rng(0)

actions = np.column_stack([rng.normal(0.00, 0.02, 1000),      # x movement in metres
                           rng.normal(0.00, 0.02, 1000),      # y movement in metres
                           rng.uniform(0.0, 0.08, 1000)])     # gripper opening in metres
mean, std = actions.mean(axis=0), actions.std(axis=0)

normalised = (actions - mean) / std
print("before: std per dimension", np.round(actions.std(axis=0), 3).tolist())
print("after:  std per dimension", np.round(normalised.std(axis=0), 3).tolist())

restored = normalised * std + mean                              # what the robot needs at run time
print("restored exactly:", bool(np.allclose(restored, actions)))

before: std per dimension [0.02, 0.02, 0.023]
after:  std per dimension [1.0, 1.0, 1.0]
restored exactly: True

A common beginner bug: normalising with one dataset’s numbers and running on a robot whose numbers differ. Keep the statistics with the model.

Common misconceptions

Engineering takeaways

1. State is hidden; observations are what arrives.
2. History can recover what one frame hides.
3. Joint and end-effector actions are different spaces.
4. Keep normalisation statistics with the model.

Always ask: what exactly are these numbers, and in what units?

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: how these observation-action pairs are recorded as demonstrations and stored as datasets.

Back to the VLA roadmap