9.1 — Policy Inference as a Robotics Component

Learning objectives

After this lesson, you should be able to:

• Write a typed request and response with an action definition.
• Reject stale responses by timestamp.
• Explain why queue depth adds latency.
• Implement a heartbeat watchdog.

The one-sentence idea

Treat the policy like any untrusted but useful remote service.

Validate, timestamp, and watch.

A policy is a component, not the robot

In research code a policy is a function in a notebook. On a robot it is one part of a larger system, with other parts that must not wait for it or trust it blindly: sensors, controllers, safety monitors. Robotics software treats such parts as separate programs that talk to each other. In ROS 2, for example, the system is a graph of nodes that communicate over topics (publish/subscribe channels), services and actions, and nodes can run in one process, in separate processes or on different machines.^[1] A model that needs a big GPU often runs on another machine than the robot controller, so the boundary between them is a real interface with real delays and real failures.

This lesson is about designing that boundary: a clear contract of inputs, outputs and timing, and the checks on each side.

The contract: what goes in, what comes out, how fresh

Write the interface as data types with explicit units and timestamps. Every request carries the time its observation was taken, and every response says which request it answers, which action definition it uses and how long it took. A schema with validation catches mistakes at the boundary:

from dataclasses import dataclass
import math

@dataclass
class Request:
    stamp: float                 # seconds: when the observation was captured
    image_shape: tuple           # (height, width, 3)
    state: list                  # joint positions in radians
    instruction: str

@dataclass
class Response:
    request_stamp: float         # which observation this answers
    action: list                 # 7 numbers, metres and radians (delta end-effector pose + gripper)
    action_definition: str       # the contract version, e.g. "ee_delta_v1"
    compute_ms: float

def validate_response(r, expected_definition="ee_delta_v1", expected_dim=7):
    problems = []
    if r.action_definition != expected_definition: problems.append(f"action definition {r.action_definition} != {expected_definition}")
    if len(r.action) != expected_dim: problems.append(f"action has {len(r.action)} numbers, expected {expected_dim}")
    if any(not math.isfinite(x) for x in r.action): problems.append("action contains NaN or infinity")
    return problems

good = Response(10.00, [0.01, 0.0, -0.02, 0, 0, 0.1, 1.0], "ee_delta_v1", 85.0)
bad  = Response(10.00, [0.01, float("nan"), 0, 0, 0, 0], "joint_abs_v2", 85.0)
print("good response:", validate_response(good) or "ok")
for p in validate_response(bad): print("bad response:", p)

good response: ok
bad response: action definition joint_abs_v2 != ee_delta_v1
bad response: action has 6 numbers, expected 7
bad response: action contains NaN or infinity

Toy model. A made-up interface. Real systems define the same things in their message definitions (ROS 2 messages, protobuf, JSON schemas).

The action definition field matters most: it is how the receiver knows whether the numbers are joint angles or hand motions (lesson 4.1).

Freshness: reject answers that arrive too late

The policy answers about the world as it was when the picture was taken. If the answer arrives late, the world has moved (lesson 8.6). The consumer must compare the request's timestamp with the current time and refuse stale responses:

max_age_s = 0.25

def accept(now, response_request_stamp):
    age = now - response_request_stamp
    return age <= max_age_s, round(age, 3)

for now, stamp in ((10.10, 10.00), (10.30, 10.00), (11.50, 10.00)):
    ok, age = accept(now, stamp)
    print(f"response about the world at t={stamp}, received at t={now}: age {age}s ->", "use" if ok else "DISCARD")

response about the world at t=10.0, received at t=10.1: age 0.1s -> use
response about the world at t=10.0, received at t=10.3: age 0.3s -> DISCARD
response about the world at t=10.0, received at t=11.5: age 1.5s -> DISCARD

What to do instead depends on the safety design: repeat the last action for a short while, slow to a stop, or hand control to a fallback (lesson 9.7). The decision belongs in the component that owns the robot, not in the model.

Messages can be lost: choose the delivery policy deliberately

Networks drop and queue messages. ROS 2 exposes this through quality-of-service policies: reliability ranges from reliable (TCP-like) to best-effort (UDP-like), history keeps the last N messages or all of them, and there are further policies for durability, deadlines, lifespan and liveliness.^[1] For a stream of camera frames, the latest frame is what matters; queueing old ones only adds delay. For a rare command like “stop”, you want guaranteed delivery. Different messages, different policies. The effect of a queue on latency is easy to compute:

frame_period_ms = 33                 # camera at about 30 Hz
service_ms = 80                      # the policy takes 80 ms per frame, longer than the frame period

for depth in (1, 5, 20):
    queued_delay = depth * service_ms
    print(f"queue depth {depth:>2}: a frame can wait up to {queued_delay:>4} ms behind older frames before it is processed")
print("with depth 1 (keep only the latest), the policy always works on the newest frame it can get")

queue depth  1: a frame can wait up to   80 ms behind older frames before it is processed
queue depth  5: a frame can wait up to  400 ms behind older frames before it is processed
queue depth 20: a frame can wait up to 1600 ms behind older frames before it is processed
with depth 1 (keep only the latest), the policy always works on the newest frame it can get

The model is slower than the camera, so any backlog only grows. Dropping old frames keeps the observation fresh at the cost of skipping some.

Health: know when the component is not well

A policy server can hang, run out of memory or return garbage. The robot-side code should watch for a missing heartbeat and for responses that repeatedly fail validation. A simple watchdog, with a virtual clock for clarity:

timeout_s = 0.5
heartbeats = [0.0, 0.2, 0.4, 0.6, 0.8, 1.9, 2.1]          # times at which the policy server reported "alive"

last = heartbeats[0]
for t in heartbeats[1:]:
    if t - last > timeout_s:
        print(f"t={last:.1f}..{t:.1f}: no heartbeat for {t - last:.1f}s -> trip the watchdog (stop or fall back)")
    last = t
print("last heartbeat seen at", last)

t=0.8..1.9: no heartbeat for 1.1s -> trip the watchdog (stop or fall back)
last heartbeat seen at 2.1

Watchdogs, validation and freshness checks are small and unglamorous. They are what makes a big learned model safe to place inside a robot.

Common misconceptions

Engineering takeaways

1. Typed contract with action definition.
2. Timestamps and freshness.
3. Queue depth = delay.
4. Heartbeat watchdog.

Trust the model less than the contract around it.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: adapters that turn a model’s outputs into a particular robot’s commands.

Back to the VLA roadmap