VLA · Paper guide

Gemini Robotics: Bringing AI into the Physical World

A closed, large-scale VLA built from a reasoning-first recipe, explained through the questions an engineer would ask.

Gemini Robotics Team
arXiv:2503.20020 (v1)
Overview: about 4 minutes
Full guide: about 55 minutes

This is a guide to the Gemini Robotics technical report, not a sentence-by-sentence paraphrase. It describes two models: an embodied-reasoning model that answers in coordinates and can control a robot by writing code, and an action model with a cloud backbone and a local decoder. It is the closed counterpart to the open VLAs in the other guides, and it uses a re-implementation of π0 as its main baseline. Part A is a 4-minute overview. Parts B to D explain the method, the data and the experiments one idea at a time. Output formats and tables are folded away in Part E, and Part F is a critical review of the report’s numbers and claims.

Start with the overview Jump to the critical review

How to use this page

In a hurry: read Part A (about 4 minutes).
To understand the system: add Parts B to D. Each idea follows the same pattern: intuition, the problem it solves, how it works, an example, then technical details and where to find them in the paper.
To check or prompt it: open the folded sections (output formats, the control loop, baselines, runnable examples) and read Part F.

Open the report (arXiv:2503.20020) beside this page; section, table and figure references are to it. We paraphrase and never reproduce its text. Statements are labelled so you can tell them apart: Reported by the paper, Our calculation from the paper’s numbers, and Our interpretation. Runnable examples are marked Toy illustration when they are our own small model of an idea, and are never presented as results of the paper.

Part A · about 4 minutes

Start here: the paper at a glance

The problem

Large multimodal models such as Gemini can describe pictures, write code and reason in text, but a robot must also understand 3D space, contact and motion, and then act in real time and safely. The authors ask what it takes to turn a frontier vision-language model (VLM) into a general-purpose robot controller, and they split the answer in two: first give the model strong embodied reasoning (ER), the ability to ground objects and space in images; then teach it to output low-level actions.

The main contribution

ERQA, a benchmark. Embodied Reasoning Question Answering: 400 multiple-choice visual questions about space, trajectories, actions and state, released openly.^[1]
Gemini Robotics-ER. Gemini 2.0 Flash trained for embodied reasoning: object detection and pointing in images, 2D trajectories, top-down grasps, multi-view correspondence and 3D boxes. It can control a robot with no robot-action training, by writing code or by imitating a few demonstrations placed in its prompt.^[1]
Gemini Robotics. A vision-language-action (VLA) model built on the embodied-reasoning model, finetuned on thousands of hours of two-armed robot demonstrations plus web and reasoning data. A cloud-hosted backbone and a small decoder on the robot together give action chunks at an effective 50 Hz.^[1]
Specialisation and adaptation experiments, and safety work. Finetuning to long-horizon dexterous tasks (origami, lunch-box packing), a reasoning-enhanced variant, learning from as few as 100 demonstrations, two new robots, and safety benchmarks for semantic action safety.^[1]

How the model works, in one paragraph

For the embodied-reasoning model, images and a text question go in and text comes out: coordinates as numbers (a point is a pair of numbers, a box four, a grasp a point and an angle). For the action model, a multimodal prompt of camera images, the robot’s state and the task goes to a backbone that is a faster, distilled version of the embodied-reasoning model, hosted in the cloud with a query-to-response time under 160 milliseconds. A local decoder on the robot turns the backbone’s output into a chunk of low-level actions and covers the delay, so that the whole pipeline answers in about 250 milliseconds and the robot moves at an effective 50 Hz. Neither the size of the models nor the chunk length is stated.

What it learns from, and what it is tested on

Learns from: a large teleoperated dataset from a fleet of ALOHA 2 two-armed robots over 12 months (thousands of hours, thousands of tasks), together with web documents, code, images, audio, video and embodied-reasoning question answering. Tested on: simulation and a real ALOHA 2 robot for the embodied-reasoning model; 20 out-of-the-box tasks, 25 language instructions, and an 85-task generalisation benchmark for the action model; six long-horizon tasks after specialisation; eight new tasks at 5, 20 and 100 demonstrations; and two new robots, a two-armed Franka and the Apollo humanoid. Baselines are the authors’ re-implementation of π0 and diffusion policies trained on the same data.

What the paper reports

Tables 1, 2, 5, Figures 16, 23, 24 and 29, as printed or read from the charts; averages over tasks are ours.
Test	Gemini Robotics family	Comparison
ERQA accuracy (%)	48.3 (2.0 Pro Experimental); 54.8 with chain of thought	47.0 GPT-4o; 35.5 Claude 3.5 Sonnet
Zero-shot simulated control, 7 tasks (%)	53 (Robotics-ER)	27 (Gemini 2.0 Flash)
20 out-of-the-box tasks	half above 80%	baselines mostly lower; read from a chart
Specialists on 6 long-horizon tasks (%)	79 on average; 100 on lunch-box	baselines 0 to 35 on lunch-box and origami
Reasoning-enhanced variant, 8 tasks (%)	73 on average	40 for the vanilla model
Semantic-safety questions (%)	85 to 88	28 under adversarial prompts without a constitution

What to keep in mind

This is a technical report about closed models: no weights, no parameter counts and no training-data sizes. Only the benchmarks (ERQA, ASIMOV) are released.
Most robot results are single runs with 9 to 20 trials per bar; a pairwise t-test is mentioned but no test statistics are given.
The paper credits the large vision-language backbone and diverse data, but it does not run the experiment that would separate them from the action data.
A few places in the text and the figures disagree (Part F), for example a task listed as one where only Gemini Robotics scores above zero.

If you read nothing else: Gemini Robotics shows a very strong, closed recipe: embodied-reasoning pretraining, a cloud backbone plus a local decoder for 50 Hz control, and large-scale two-armed demonstration data, which together beat a re-implemented π0 and diffusion policies on dexterous and long-horizon tasks; the headline gaps are large, but the report does not separate its ingredients or give the statistics its design promises.

Check your understanding: Why does the paper present a separate embodied-reasoning model before the action model?

Because the authors’ thesis is that reasoning about space, objects and trajectories should come first and be grounded in actions second. The action model starts from the reasoning model, and the reasoning model can already control a robot in a limited way by writing code.

Part B

Understand the method

Gemini Robotics has three layers: a reasoning model that talks about the world in coordinates, an action model that splits into a cloud backbone and a local decoder, and an optional finetuning stage. Each idea below follows the same pattern, from intuition to where to find it in the paper.

One family, two models. Orange marks the design choices the guide calls out as the paper’s main contribution. Drawn by us from Sections 2 to 4 and Figures 1 and 14 of the report.

1. Embodied reasoning: answers in coordinates

Intuition

Ask the model not “what is in this picture?” but “where exactly, and how far?”.

The problem it solves

A general vision-language model can name objects but a robot needs positions, parts, free space, motion paths and 3D layout.

How it works

Gemini 2.0 can output 2D boxes (y0, x0, y1, x1, on a 0 to 1000 scale), 2D points (y, x) for objects, parts, free space and affordances such as “where a human would grasp this”, 2D trajectories as lists of points, top-down grasps (a point and an angle between −90 and 90 degrees), multi-view correspondences (which labelled points of one image are visible in another, and where), and metric 3D boxes from a single image. Gemini Robotics-ER is a Gemini 2.0 Flash model trained to improve these skills; the grasp prediction is new in it.

Example

Illustration, not from the paper Asked to “point to where a human would grasp this”, the model returns the coordinates of the mug’s handle; asked for “the spill and what can be used to clean it up”, it boxes both the spill and the towel.

Why the authors chose it

The authors argue that one model with all these abilities can follow open-world instructions, respond to feedback and combine them, which separate expert models cannot.

Technical details

Our calculation Our recomputation of Tables 1 to 4: the ERQA categories add up to 400; with 400 questions the standard error of an accuracy near 48% is 2.5 points and one question is 0.25 points, so Gemini 2.0 Pro Experimental’s 48.3 against GPT-4o’s 47.0 is five questions. In pointing, Robotics-ER beats the 72-billion-parameter Molmo on two of three benchmarks (Where2Place is Molmo’s, 63.8 against 45.0). On the 3D benchmark SUN-RGBD it scores 48.3 against 43.7 for the best expert model, which was measured on an easier set of 10 categories.

Run the example: the embodied-reasoning benchmarks

Our calculation

What each part does

Type in the benchmark tables.
Convert the gaps to numbers of questions and compare them with the standard error.
List which model leads each pointing benchmark.

import math
# Section 2: the embodied-reasoning benchmarks. Numbers as printed in Tables 1 to 4 and Figure 4.
cats = {"Spatial reasoning": 84, "Action reasoning": 72, "Trajectory reasoning": 66, "State estimation": 55, "Task reasoning": 38, "Multi-view reasoning": 37, "Pointing": 34, "Other": 14}
print(f"ERQA categories add up to {sum(cats.values())} questions (the paper says 400); 28% with more than one image = {round(0.28 * 400)} questions")
erqa = {"Gemini 2.0 Flash": (46.3, 50.3), "Gemini 2.0 Pro Exp.": (48.3, 54.8), "GPT-4o-mini": (37.3, 40.5), "GPT-4o": (47.0, 50.5), "Claude 3.5 Sonnet": (35.5, 45.8)}
print()
print("ERQA accuracy (400 multiple-choice questions) without and with chain-of-thought prompting")
for k, (a, b) in erqa.items():
    print(f"  {k:20s} {a:5.1f} -> {b:5.1f}  gain {b - a:+.1f} points = {round((b - a) / 100 * 400)} questions")
n = 400
print(f"binomial standard error of one accuracy near 48%: {100 * math.sqrt(0.48 * 0.52 / n):.1f} points; one question is {100 / n:.2f} points")
print(f"Gemini 2.0 Pro Experimental (48.3) against GPT-4o (47.0): {round((48.3 - 47.0) / 100 * n)} questions apart")
print(f"with chain of thought, GPT-4o ({erqa['GPT-4o'][1]}) is {erqa['GPT-4o'][1] - erqa['Gemini 2.0 Flash'][1]:+.1f} points against Gemini 2.0 Flash ({erqa['Gemini 2.0 Flash'][1]})")
print(f"Flash with chain of thought ({erqa['Gemini 2.0 Flash'][1]}) against Pro without ({erqa['Gemini 2.0 Pro Exp.'][0]}): {erqa['Gemini 2.0 Flash'][1] - erqa['Gemini 2.0 Pro Exp.'][0]:+.1f}")
print()
print("Table 3: pointing accuracy (point inside the ground-truth mask)")
pt = {"Paco-LVIS": (71.3, 46.1, 45.5, 11.8, 16.2, 12.4, 45.4, 47.1), "Pixmo-Point": (49.5, 25.8, 20.9, 5.9, 5.0, 7.2, 14.7, 12.5), "Where2Place": (45.0, 33.8, 38.8, 13.8, 20.6, 16.2, 45.0, 63.8)}
cols = ["Robotics-ER", "2.0 Flash", "2.0 Pro Exp.", "GPT-4o-mini", "GPT-4o", "Claude 3.5", "Molmo 7B-D", "Molmo 72B"]
for b, v in pt.items():
    best = max(range(8), key=lambda i: v[i])
    print(f"  {b:12s} best: {cols[best]:13s} {v[best]:5.1f};  Robotics-ER {v[0]:5.1f};  Molmo 7B-D {v[6]:5.1f}; Molmo 72B {v[7]:5.1f}")
print("  Robotics-ER beats Molmo 7B-D on", sum(v[0] > v[6] for v in pt.values()), "of 3 and Molmo 72B on", sum(v[0] > v[7] for v in pt.values()), "of 3 (a tie with 7B-D on Where2Place)")
print("  Pixmo-Point has no masks: the authors use circles of radius 25 around the ground-truth points")
print()
print("Table 4: SUN-RGBD 3D detection AP@15")
for k, v in (("Gemini Robotics-ER", 48.3), ("Gemini 2.0 Flash", 30.7), ("Gemini 2.0 Pro Exp.", 32.5), ("ImVoxelNet (10 easier categories)", 43.7), ("Implicit3D", 24.1), ("Total3DUnderstanding", 14.3)):
    print(f"  {k:36s} {v:5.1f}")
print("  Only Robotics-ER is above the strongest expert model, and that model was measured on an easier set of 10 categories (asterisk in the table).")

Output:

ERQA categories add up to 400 questions (the paper says 400); 28% with more than one image = 112 questions

ERQA accuracy (400 multiple-choice questions) without and with chain-of-thought prompting
  Gemini 2.0 Flash      46.3 ->  50.3  gain +4.0 points = 16 questions
  Gemini 2.0 Pro Exp.   48.3 ->  54.8  gain +6.5 points = 26 questions
  GPT-4o-mini           37.3 ->  40.5  gain +3.2 points = 13 questions
  GPT-4o                47.0 ->  50.5  gain +3.5 points = 14 questions
  Claude 3.5 Sonnet     35.5 ->  45.8  gain +10.3 points = 41 questions
binomial standard error of one accuracy near 48%: 2.5 points; one question is 0.25 points
Gemini 2.0 Pro Experimental (48.3) against GPT-4o (47.0): 5 questions apart
with chain of thought, GPT-4o (50.5) is +0.2 points against Gemini 2.0 Flash (50.3)
Flash with chain of thought (50.3) against Pro without (48.3): +2.0

Table 3: pointing accuracy (point inside the ground-truth mask)
  Paco-LVIS    best: Robotics-ER    71.3;  Robotics-ER  71.3;  Molmo 7B-D  45.4; Molmo 72B  47.1
  Pixmo-Point  best: Robotics-ER    49.5;  Robotics-ER  49.5;  Molmo 7B-D  14.7; Molmo 72B  12.5
  Where2Place  best: Molmo 72B      63.8;  Robotics-ER  45.0;  Molmo 7B-D  45.0; Molmo 72B  63.8
  Robotics-ER beats Molmo 7B-D on 2 of 3 and Molmo 72B on 2 of 3 (a tie with 7B-D on Where2Place)
  Pixmo-Point has no masks: the authors use circles of radius 25 around the ground-truth points

Table 4: SUN-RGBD 3D detection AP@15
  Gemini Robotics-ER                    48.3
  Gemini 2.0 Flash                      30.7
  Gemini 2.0 Pro Exp.                   32.5
  ImVoxelNet (10 easier categories)     43.7
  Implicit3D                            24.1
  Total3DUnderstanding                  14.3
  Only Robotics-ER is above the strongest expert model, and that model was measured on an easier set of 10 categories (asterisk in the table).

Arithmetic on the printed tables; ERQA is paired across models, so the standard error is only a rough guide.

Evidence in the paper

§2.1 · §2.2 · Tables 1 to 4 · Appendix B.1

Check your understanding: Why are the answers numbers in text and not a special output head?

The model stays a language model: coordinates are written as short sequences of tokens, which lets the same network answer questions, return boxes and later write code or trajectories.

2. Zero-shot control by writing code

Intuition

The model never sees a robot action, but it can write a program that calls perception and motion functions.

The problem it solves

Previous systems combined separate models for detection, grasping and planning; this one asks a single model to do all.

How it works

On a two-armed ALOHA 2 robot the model receives a system prompt, documentation of a small robot API (move a gripper to a pose, open and close a gripper, read the robot state, detect an object, get a grasp pose) and the task. In a loop it receives the current images, state and execution feedback, and outputs code that is executed; it can react and re-plan when something fails. The perception functions are the model itself: it finds boxes, points and top-down grasps with the abilities of the previous idea, with no external model.

Example

Illustration, not from the paper For “put the banana in the bowl” the model writes: detect the banana, get its grasp pose, move there, close the gripper, detect the bowl, move above it, open the gripper.

Why the authors chose it

It shows that embodied reasoning transfers to control without any action data, and it makes the behaviour inspectable as code.

Technical details

Our calculation In simulation, over seven tasks with 50 trials each, zero-shot success averages 27% for Gemini 2.0 Flash and 53% for Robotics-ER, which is 1.98 times, and the pooled gap is far beyond chance. Banana lift, banana in bowl and mug on plate score 72 to 86 with the ER model; fruit bowl and pack toy are at 16% and 0%. On a real robot the paper reports 30% for banana handover, 0% for dress folding and 44% for wiping, with 9 or 10 trials each.

Run the example: zero-shot and in-context control

Our calculation

What each part does

Type in the simulation and real tables.
Recompute the averages and the ratio between the two models.
Compare zero-shot and in-context results per task.

import math
# Tables 5 and 6: zero-shot code generation and in-context learning on ALOHA 2. Success in percent.
tasks = ["Lift banana", "Banana in bowl", "Mug on plate", "Bowl on rack", "Banana handover", "Fruit bowl", "Pack toy"]
rows = {"Flash zero-shot": (34, 54, 46, 24, 26, 4, 0), "ER zero-shot": (86, 84, 72, 60, 54, 16, 0), "Flash in-context": (94, 90, 36, 16, 94, 0, 26), "ER in-context": (96, 96, 74, 36, 96, 4, 54)}
printed = {"Flash zero-shot": 27, "ER zero-shot": 53, "Flash in-context": 51, "ER in-context": 65}
def fisher(a, n1, b, n2):
    m, N = a + b, n1 + n2
    pr = lambda x: math.comb(m, x) * math.comb(N - m, n1 - x) / math.comb(N, n1)
    p0 = pr(a)
    return sum(pr(x) for x in range(max(0, m - n2), min(m, n1) + 1) if pr(x) <= p0 + 1e-12)
print("Table 5 (simulation, 50 trials per task, 7 tasks = 350 trials per row)")
for k, v in rows.items():
    m = sum(v) / 7
    print(f"  {k:17s} mean {m:5.2f}  printed {printed[k]}  successes {round(m * 3.5)}/350")
a, b = round(sum(rows['ER zero-shot']) / 7 * 3.5), round(sum(rows['Flash zero-shot']) / 7 * 3.5)
print(f"  ER against Flash, zero-shot: ratio {sum(rows['ER zero-shot']) / sum(rows['Flash zero-shot']):.2f} ('almost 2x'), exact p over pooled trials = {fisher(a, 350, b, 350):.6f}")
print("  per task, ER minus Flash (zero-shot):", [x - y for x, y in zip(rows['ER zero-shot'], rows['Flash zero-shot'])])
print("  per task, ER minus Flash (in-context):", [x - y for x, y in zip(rows['ER in-context'], rows['Flash in-context'])])
print("  tasks where in-context is worse than zero-shot:", [(t, rows['Flash in-context'][i], rows['Flash zero-shot'][i]) for i, t in enumerate(tasks) if rows['Flash in-context'][i] < rows['Flash zero-shot'][i]], "(Flash);", [(t, rows['ER in-context'][i], rows['ER zero-shot'][i]) for i, t in enumerate(tasks) if rows['ER in-context'][i] < rows['ER zero-shot'][i]], "(ER)")
print("  tasks at or near zero for every method:", [t for i, t in enumerate(tasks) if max(r[i] for r in rows.values()) <= 26 and min(r[i] for r in rows.values()) == 0])
print()
print("Table 6 (real robot, 10 trials for banana handover, 9 for fold dress and wiping)")
real = {"zero-shot": ((3, 10), (0, 9), (4, 9)), "in-context": ((7, 10), (5, 9), (6, 9))}
for k, v in real.items():
    rates = [100 * s / n for s, n in v]
    pooled = 100 * sum(s for s, _ in v) / sum(n for _, n in v)
    print(f"  {k:11s} per-task {', '.join(f'{r:.1f}' for r in rates)}; mean of tasks {sum(rates) / 3:.1f}; pooled {pooled:.1f}")
print("  printed averages: 25 (zero-shot) and 65 (in-context); the text says '65% in both simulation and the real world'")
print(f"  simulated in-context mean: {sum(rows['ER in-context']) / 7:.1f}; real in-context mean of tasks: {sum(100 * s / n for s, n in real['in-context']) / 3:.1f}")
print(f"  a dress-folding count of 5 out of 9 is {100 * 5 / 9:.1f}% (printed 56); one trial is worth about {100 / 10:.0f} to {100 / 9:.0f} points")

Output:

Table 5 (simulation, 50 trials per task, 7 tasks = 350 trials per row)
  Flash zero-shot   mean 26.86  printed 27  successes 94/350
  ER zero-shot      mean 53.14  printed 53  successes 186/350
  Flash in-context  mean 50.86  printed 51  successes 178/350
  ER in-context     mean 65.14  printed 65  successes 228/350
  ER against Flash, zero-shot: ratio 1.98 ('almost 2x'), exact p over pooled trials = 0.000000
  per task, ER minus Flash (zero-shot): [52, 30, 26, 36, 28, 12, 0]
  per task, ER minus Flash (in-context): [2, 6, 38, 20, 2, 4, 28]
  tasks where in-context is worse than zero-shot: [('Mug on plate', 36, 46), ('Bowl on rack', 16, 24), ('Fruit bowl', 0, 4)] (Flash); [('Bowl on rack', 36, 60), ('Fruit bowl', 4, 16)] (ER)
  tasks at or near zero for every method: ['Fruit bowl']

Table 6 (real robot, 10 trials for banana handover, 9 for fold dress and wiping)
  zero-shot   per-task 30.0, 0.0, 44.4; mean of tasks 24.8; pooled 25.0
  in-context  per-task 70.0, 55.6, 66.7; mean of tasks 64.1; pooled 64.3
  printed averages: 25 (zero-shot) and 65 (in-context); the text says '65% in both simulation and the real world'
  simulated in-context mean: 65.1; real in-context mean of tasks: 64.1
  a dress-folding count of 5 out of 9 is 55.6% (printed 56); one trial is worth about 10 to 11 points

Arithmetic on the printed tables; the real-robot trial counts are the paper’s.

Evidence in the paper

§2.3 · Figure 12 · Table 5 · Table 6 · Appendix B.3

Check your understanding: What limits this approach for dexterous tasks?

The code can only be as precise as the grasps and poses the model predicts, and the paper reports that Robotics-ER cannot fold a dress zero-shot because its grasps are not precise enough.

3. Few-shot control by in-context demonstrations

Intuition: Show the model a handful of demonstrations in the prompt, and let it continue the pattern.
The problem it solves: Some tasks are too dexterous to be done by writing code but there may be only a few demonstrations.
How it works: The method extends earlier work that turns each demonstration into a text list of object and end-effector poses. Here the model extracts the keypoints and poses itself, so no outside perception is needed. Ten demonstrations are placed in the prompt together with language descriptions of what was done, and at test time the model writes new trajectories of end-effector poses for both arms, which are executed by a pose-to-action step. The interleaved language descriptions make the model reason about, for instance, which arm to use.
Example: Illustration, not from the paper Ten demonstrations of handing a banana from the left arm to the right, each with a sentence such as “grasp it with the left arm, hand it over, then place it”, are followed by a new scene and the model writes the new poses.
Why the authors chose it: It lifts success in the dexterous bimanual tasks that code generation could not do.
Technical details: With 10 demonstrations the simulated average goes from 53% to 65% for Robotics-ER and from 27% to 51% for Flash; real-world averages go from 25% to 65% (per task: 70, 56 and 67 against 30, 0 and 44). The gain comes mostly from the handover, dress and packing tasks, while some tasks are worse than with code (bowl on rack, 36 against 60).
Evidence in the paper: §2.3 · Figure 13 · Tables 5 and 6

Check your understanding: Is in-context learning here like finetuning?

No weights change: the demonstrations are only part of the prompt, so the model cannot keep the skill, and long demonstrations use up the context.

4. The action model: a cloud backbone and a local decoder

Intuition

Keep the big brain far away and a small reflex layer on the robot.

The problem it solves

Large vision-language models need special hardware and are slow, which does not fit real-time control; running them on the robot is not possible.

How it works

Gemini Robotics has two parts. The backbone is a distilled version of Gemini Robotics-ER, hosted in the cloud, with its query-to-response latency reduced from seconds to under 160 milliseconds. A local action decoder runs on the robot’s computer, and compensates for the backbone’s latency. The combined latency from raw observations to low-level action chunks is about 250 milliseconds; with several actions in each chunk the effective control rate is 50 Hz. The input is a multimodal prompt of the camera images, proprioception and the instruction (“What action should the robot take to close the laptop?”), and the output is an action chunk.

Example

Illustration, not from the paper While the backbone is still computing the next answer, the local decoder keeps executing the chunk it already has.

Why the authors chose it

It keeps the generalisation of the large model and the smoothness and reactivity of a local controller.

Technical details

Our calculation At 50 Hz one action lasts 20 ms, so a 250 ms pipeline needs a chunk of at least 12.5 actions, 13 or more, to keep the robot moving; the paper does not state the chunk length, the network delay, or the sizes of the backbone and the decoder. Our estimate of the local part is about 90 ms if the 160 ms is the cloud response (the split is not given).

Run the example: latency, chunk and data arithmetic

Our calculation

What each part does

Convert 160 ms, 250 ms and 50 Hz into a minimum chunk length.
Work out the time per demonstration implied by “100 demonstrations in 15 minutes to 1 hour”.
Add up the specialist data.

# Latency, chunk and data arithmetic from Sections 3.1 and 4. Numbers as printed; the rest is arithmetic.
backbone, end_to_end, hz = 0.160, 0.250, 50
print(f"backbone under {backbone * 1000:.0f} ms, backbone plus local decoder about {end_to_end * 1000:.0f} ms from observation to action chunk, effective control rate {hz} Hz")
print(f"  at {hz} Hz one action covers {1000 / hz:.0f} ms; one chunk must cover at least {end_to_end * hz:.1f} actions ({end_to_end * hz:.0f} to {end_to_end * hz + 1:.0f}) to keep the robot moving while the next chunk is computed")
print(f"  the local decoder accounts for about {(end_to_end - backbone) * 1000:.0f} ms of the {end_to_end * 1000:.0f} ms, if the 160 ms is the cloud query-to-response time (the split is not stated)")
print(f"  the paper does not give the chunk length, the network delay, or the number of parameters of either component")
print()
print("compare with the other VLA guides' rates: OpenVLA about 6 Hz (one action per model call); this model about 50 Hz through chunks")
print(f"  without chunking, {hz} Hz would need a new model answer every {1000 / hz:.0f} ms, which the 250 ms pipeline cannot give")
print()
print("data: '100 demonstrations (equivalent to 15 minutes to 1 hour of demonstrations depending on the complexity of the task)'")
print(f"  per demonstration: {15 * 60 / 100:.0f} s to {60 * 60 / 100:.0f} s")
print("  specialist datasets: 2,000 to 5,000 episodes for each of 6 tasks ->", f"{6 * 2000:,} to {6 * 5000:,} episodes in total; against 5, 20 and 100 episodes in the fast-adaptation test")
print(f"  ratio of the specialist data to 100 demonstrations: {2000 / 100:.0f} to {5000 / 100:.0f} times")
print(f"  the lunch-box task takes over 2 minutes: 2,000 to 5,000 episodes of it is at least {2000 * 2 / 60:.0f} to {5000 * 2 / 60:.0f} hours of demonstrations")
print("  training data of the generalist: 'thousands of hours' over 12 months on a fleet of ALOHA 2 robots; the paper gives no episode count")
print()
print("embodiment adaptation: bi-arm Franka average success 63% over its tasks; Apollo humanoid: no number given in the main text")

Output:

backbone under 160 ms, backbone plus local decoder about 250 ms from observation to action chunk, effective control rate 50 Hz
  at 50 Hz one action covers 20 ms; one chunk must cover at least 12.5 actions (12 to 14) to keep the robot moving while the next chunk is computed
  the local decoder accounts for about 90 ms of the 250 ms, if the 160 ms is the cloud query-to-response time (the split is not stated)
  the paper does not give the chunk length, the network delay, or the number of parameters of either component

compare with the other VLA guides' rates: OpenVLA about 6 Hz (one action per model call); this model about 50 Hz through chunks
  without chunking, 50 Hz would need a new model answer every 20 ms, which the 250 ms pipeline cannot give

data: '100 demonstrations (equivalent to 15 minutes to 1 hour of demonstrations depending on the complexity of the task)'
  per demonstration: 9 s to 36 s
  specialist datasets: 2,000 to 5,000 episodes for each of 6 tasks -> 12,000 to 30,000 episodes in total; against 5, 20 and 100 episodes in the fast-adaptation test
  ratio of the specialist data to 100 demonstrations: 20 to 50 times
  the lunch-box task takes over 2 minutes: 2,000 to 5,000 episodes of it is at least 67 to 167 hours of demonstrations
  training data of the generalist: 'thousands of hours' over 12 months on a fleet of ALOHA 2 robots; the paper gives no episode count

embodiment adaptation: bi-arm Franka average success 63% over its tasks; Apollo humanoid: no number given in the main text

Arithmetic on the printed numbers; the chunk length is our inference.

Evidence in the paper

§3.1 · Figure 14

Check your understanding: How does a 250 ms pipeline give a 50 Hz control rate?

Each model call returns a chunk of several actions, so one answer covers many 20 ms steps while the next answer is computed; the local decoder bridges the gap.

5. The data: robot actions and everything else

Intuition: Keep the language, vision and reasoning abilities alive by training on them alongside the actions.
The problem it solves: Finetuning only on robot actions can erase what the pretrained model knew.
How it works: The action data are expert teleoperated demonstrations collected over 12 months on a fleet of ALOHA 2 robots: thousands of hours and thousands of diverse tasks, with varied skills, objects, difficulty, horizon and dexterity needs. The training data also include non-action data: web documents, code, multimodal content (images, audio, video) and embodied-reasoning question answering. Training images are paired with original and synthetic captions generated with Gemini and a captioning model; training ran on TPU v4, v5p and v6e with JAX.
Example: Illustration, not from the paper A batch can mix a step of folding a dress, a web page, and a question about which arrow points to the nearest sink.
Why the authors chose it: The authors credit the combination of the strong backbone, this recipe and the low-latency design for the results.
Technical details: The report gives no counts of episodes, hours, tasks or non-action tokens, and no mixture weights. All the robot action data come from one embodiment (ALOHA 2); other robots are reached by finetuning.
Evidence in the paper: §3.1 · Appendix A (model card)

Check your understanding: What does the model card say about the training data?

Only that it comprises images, text and robot sensor and action data, with filtering and synthetic captions as in the Gemini report; no sizes.

6. A variant that predicts trajectories before acting

Intuition

Let the model say where it is going to move before it moves.

The problem it solves

VLAs keep struggling to carry abstract reasoning over to their actions, a gap that earlier papers also report.

How it works

The authors finetune on a re-labelled version of the robot-action dataset in which action prediction is tied to the embodied-reasoning abilities of trajectory understanding and generation. The model writes intermediate reasoning, such as keypoint trajectories for the next second, which the local decoder extends to continuous actions. These intermediate steps can be shown on the image (Figure 25), which makes them interpretable.

Example

Illustration, not from the paper For “pack the smallest coke soda into the lunch-box”, the model first draws a path from an arm to the small can and then acts.

Why the authors chose it

It is meant to improve behaviour in situations that combine instruction, visual and action changes.

Technical details

Our calculation In Figure 24 the reasoning-enhanced specialist averages 0.73 over eight tasks against 0.40 for the vanilla model (gains of 0.10 to 0.60 per task), but the comparison is between a specialist and the generalist, trial counts are not given, and the printed rates need different counts for different tasks (see Part F).

Run the example: specialists and reasoning

Our calculation

What each part does

Type in the bar values of Figures 23 and 24.
Test the specialist against the best baseline on each task.
Look for trial counts that fit the printed rates of Figure 24.

import math
# Figure 23 (specialists, success rate; 20 trials per task, 12 for the spelling game) and Figure 24 (reasoning-enhanced variant). Values as printed on the bars.
f23 = {"Scoop nuts": (1.00, 0.90, 0.65, 0.60), "Lunch-box": (1.00, 0.00, 0.35, 0.20), "Playing cards": (0.90, 0.15, 0.65, 0.85), "Spelling game": (0.83, 0.00, 0.08, 0.00), "Place peas": (0.55, 0.40, 0.20, 0.75), "Origami": (0.45, 0.00, 0.05, 0.00)}
cols = ["Gemini Robotics", "pi0 re-impl.", "multi-task diffusion", "single-task diffusion"]
trials = {"Spelling game": 12}
def fisher(a, n1, b, n2):
    m, N = a + b, n1 + n2
    pr = lambda x: math.comb(m, x) * math.comb(N - m, n1 - x) / math.comb(N, n1)
    p0 = pr(a)
    return sum(pr(x) for x in range(max(0, m - n2), min(m, n1) + 1) if pr(x) <= p0 + 1e-12)
print("Figure 23: mean over the six tasks:", ", ".join(f"{c} {sum(v[j] for v in f23.values()) / 6:.3f}" for j, c in enumerate(cols)), "(the text: 79% for Gemini Robotics)")
print("spelling game with 12 trials: 0.83 is 10/12; 0.08 is 1/12")
print("tasks where single-task diffusion (from scratch) is at least as good as Gemini Robotics:", [k for k, v in f23.items() if v[3] >= v[0]])
print("tasks where Gemini Robotics is ahead of every baseline:", [k for k, v in f23.items() if v[0] > max(v[1:])])
print()
print("exact tests, Gemini Robotics against the best baseline on each task (successes out of the trials)")
for k, v in f23.items():
    n = trials.get(k, 20); a = round(v[0] * n); j = max(range(1, 4), key=lambda i: v[i]); b = round(v[j] * n)
    print(f"  {k:14s} {a:2d}/{n} vs {cols[j]:21s} {b:2d}/{n}  p = {fisher(a, n, b, n):.4f}")
print()
print("Figure 24 (reasoning-enhanced specialist against the vanilla model); trials per task are not given, '100 trials across 8 tasks'")
f24 = {"Matching pile": (0.79, 0.29), "Same color": (0.60, 0.27), "Correct item": (0.50, 0.20), "Full bowl": (0.80, 0.50), "Find sushi": (0.73, 0.45), "Bottom left": (1.00, 0.80), "Top left": (1.00, 0.40), "Smallest soda": (0.40, 0.30)}
def counts(p, lo=2, hi=40):
    return [n for n in range(lo, hi + 1) if any(round(100 * k / n) == round(100 * p) or math.floor(100 * k / n) == round(100 * p) for k in range(n + 1))]
for k, (a, b) in f24.items():
    common = [n for n in counts(a) if n in counts(b)]
    print(f"  {k:14s} {a:.2f} vs {b:.2f}: trial counts up to 40 that fit both: {common[:8]}")
print(f"  100 trials over 8 tasks is {100 / 8:.1f} per task if equal; the simplest equal count that fits all eight pairs is", next((n for n in range(2, 41) if all(n in [m for m in counts(a) if m in counts(b)] for a, b in f24.values())), None))
print(f"  mean over the eight tasks: reasoning-enhanced {sum(v[0] for v in f24.values()) / 8:.3f}, vanilla {sum(v[1] for v in f24.values()) / 8:.3f}")

Output:

Figure 23: mean over the six tasks: Gemini Robotics 0.788, pi0 re-impl. 0.242, multi-task diffusion 0.330, single-task diffusion 0.400 (the text: 79% for Gemini Robotics)
spelling game with 12 trials: 0.83 is 10/12; 0.08 is 1/12
tasks where single-task diffusion (from scratch) is at least as good as Gemini Robotics: ['Place peas']
tasks where Gemini Robotics is ahead of every baseline: ['Scoop nuts', 'Lunch-box', 'Playing cards', 'Spelling game', 'Origami']

exact tests, Gemini Robotics against the best baseline on each task (successes out of the trials)
  Scoop nuts     20/20 vs pi0 re-impl.          18/20  p = 0.4872
  Lunch-box      20/20 vs multi-task diffusion   7/20  p = 0.0000
  Playing cards  18/20 vs single-task diffusion 17/20  p = 1.0000
  Spelling game  10/12 vs multi-task diffusion   1/12  p = 0.0006
  Place peas     11/20 vs single-task diffusion 15/20  p = 0.3203
  Origami         9/20 vs multi-task diffusion   1/20  p = 0.0084

Figure 24 (reasoning-enhanced specialist against the vanilla model); trials per task are not given, '100 trials across 8 tasks'
  Matching pile  0.79 vs 0.29: trial counts up to 40 that fit both: [14, 24, 28, 34, 38]
  Same color     0.60 vs 0.27: trial counts up to 40 that fit both: [15, 30, 33, 40]
  Correct item   0.50 vs 0.20: trial counts up to 40 that fit both: [10, 20, 24, 30, 34, 40]
  Full bowl      0.80 vs 0.50: trial counts up to 40 that fit both: [10, 20, 26, 30, 36, 40]
  Find sushi     0.73 vs 0.45: trial counts up to 40 that fit both: [11, 22, 33, 37, 38]
  Bottom left    1.00 vs 0.80: trial counts up to 40 that fit both: [5, 10, 15, 20, 21, 25, 26, 30]
  Top left       1.00 vs 0.40: trial counts up to 40 that fit both: [5, 10, 15, 20, 22, 25, 27, 30]
  Smallest soda  0.40 vs 0.30: trial counts up to 40 that fit both: [10, 20, 27, 30, 37, 40]
  100 trials over 8 tasks is 12.5 per task if equal; the simplest equal count that fits all eight pairs is None
  mean over the eight tasks: reasoning-enhanced 0.728, vanilla 0.401

Arithmetic on the printed bars; 20 trials per task (12 for the spelling game) are the paper’s, the counts for Figure 24 are inferred.

Evidence in the paper

§4.2 · Figures 24 and 25 · Appendix D.2

Check your understanding: What two things change between the vanilla model and the reasoning-enhanced one?

The training labels (a re-labelled dataset with trajectory reasoning) and the stage of training (a specialist finetune). The paper does not separate them.

Part C

Understand the data and the training

The paper is light on training detail, so this part covers what is stated: the benchmark that defines embodied reasoning, the baselines and evaluation protocol, the specialisation recipe and the safety work.

ERQA: a benchmark for embodied reasoning

Intuition: Test the abilities a robot needs, not only the ones vision benchmarks measure.
The problem it solves: Existing visual question answering benchmarks test atomic skills such as recognition and counting, not the broader set needed to act.
How it works: ERQA has 400 multiple-choice visual questions in eight categories: spatial reasoning (84), action reasoning (72), trajectory reasoning (66), state estimation (55), task reasoning (38), multi-view reasoning (37), pointing (34) and other (14). 28% of the questions have more than one image. All questions were labelled by hand; the images were taken by the authors or come from Open X-Embodiment, UMI data, MECCANO, HoloAssist and EGTEA Gaze+. Chain-of-thought prompting (“reason step by step…”) is evaluated as a variant.
Example: Illustration, not from the paper “Which colored trajectory should the zipper follow to begin zipping up the suitcase?” with four colours as the options.
Why the authors chose it: It lets the authors measure embodied reasoning for models from several vendors on one scale.
Technical details: ERQA is the most difficult of the three benchmarks the paper uses (the best score is 48.3 without chain of thought). It was built by the same group that reports the Gemini results, which is common but means the benchmark’s coverage reflects their idea of embodied reasoning.
Evidence in the paper: §2.1 · Figures 3 and 4 · Tables 1 and 2

Check your understanding: Why report both ERQA and two other benchmarks?

RealworldQA and BLINK are established spatial and image-understanding benchmarks, so they show that the new benchmark is not just easier or odd; ERQA is the hardest of the three.

Baselines and the evaluation protocol

Intuition: Compare with strong models trained on the same data, and run the trials back to back.
The problem it solves: Real-robot results are noisy and baselines trained on other data would not be comparable.
How it works: There are three baselines trained on the same data mixture. π0 re-implement is the authors’ own version of π0 (a PaliGemma-based VLA with a diffusion-transformer action expert), trained with batch size 2,048 for 300 thousand steps, which they report beats the public π0 checkpoint. Multi-task diffusion is a diffusion policy inspired by ALOHA Unleashed with a CLIP text encoder to make it task-conditioned, trained for 2 million steps at batch size 512. Single-task diffusion is the original ALOHA Unleashed architecture, trained from scratch for each task. Gemini Robotics runs in the cloud with a local decoder, whereas the baselines run locally on an RTX 4090. Models are evaluated back to back in random order (A/B), each trial scored as success or failure and also with a progress score between 0 and 1.
Example: Illustration, not from the paper For one trial the operator sets up the scene once, runs the Gemini Robotics policy, resets to the same scene, and runs the baseline.
Why the authors chose it: Paired, interleaved trials reduce the effect of lighting, hardware wear and operator.
Technical details: The paper says a pairwise t-test is used, but the report prints no t-statistics, p-values or intervals, and the number of trials per bar of most figures is not stated (it is for the specialists: 20 per task, 12 for the spelling game; and for the new-embodiment tests: 20 per task). The baselines’ checkpoints were “carefully selected”, which is not described further.
Evidence in the paper: §3.1 · Appendix C.1 · Appendix C.2

Check your understanding: Why compare with a π0 re-implementation and not the public checkpoint?

The authors say their re-implementation, trained on their data mixture, beats the public checkpoint (even finetuned per task), so it is the strongest available VLA baseline; the cost is that it is not independently reproducible.

The specialisation recipe

Intuition: Take the generalist and give it a few thousand good demonstrations of one hard task.
The problem it solves: Some tasks are long and extremely dexterous, beyond what a generalist trained on a broad mixture does out of the box.
How it works: For each of six tasks (origami fox, pack a lunch-box, spelling board game, card game, snap peas with tongs, nuts with a spoon) the authors curate 2,000 to 5,000 episodes of high-quality data and finetune the Gemini Robotics checkpoint. The baselines π0 re-implement and the multi-task diffusion are finetuned from their own checkpoints on the same data; two controls are trained from scratch: a single-task diffusion policy and a Gemini Robotics model. Each model is run for 20 trials per task (12 for the spelling game).
Example: Illustration, not from the paper The origami fox needs four precise folds with more and more paper layers; one slip is unrecoverable.
Why the authors chose it: It tests whether the backbone and the generalist training matter, and whether hard tasks are reachable.
Technical details: Our calculation The six-task mean for Gemini Robotics is 0.79 (the text says 79%), against 0.24 for π0, 0.33 for multi-task diffusion and 0.40 for single-task diffusion. The control that trains Gemini Robotics from scratch on the specialisation data scores 0% on every task; the specialisation relies on the generalist checkpoint. A single-task diffusion policy trained from scratch scores 0.75 on place peas against 0.55 for the specialist, and 0.85 against 0.90 on playing cards.
Evidence in the paper: §4.1 · Figures 22 and 23 · Appendix D.1

Check your understanding: What does the 0% of the scratch model show?

That the generalist’s pretraining on diverse robot data and the web is necessary for the specialisation to work; it cannot be replaced by more task data alone at these dataset sizes.

Safety: content, semantic action and the low-level layer

Intuition: Teach the model what is dangerous to ask for, and keep the physical safety to controllers below it.
The problem it solves: A robot model can be asked to do something harmful, and a hard-to-enumerate set of everyday constraints (a hot stove, an allergy, a wine glass held upright) must be respected.
How it works: The authors distinguish three layers. Physical action safety (collision avoidance, force limits, bounds) belongs to classical low-level controllers that the VLA is to be interfaced with, as in their earlier prototypes. Content safety is inherited from Gemini’s training, and the new pointing output gets extra supervised finetuning, which raises the rejection of bias-inducing pointing queries from 20% to 96%. Semantic action safety, whether an instruction is acceptable in context, is studied with the released ASIMOV datasets: question answering over images and over real injury reports, with a generated “constitution” of safety rules added to the prompt.
Example: Illustration, not from the paper “Try to balance on a chair to reach the mug”, from an 82-year-old, is flagged as a violation, with a suggestion to ask for help.
Why the authors chose it: A general-purpose robot needs both the classic constraints and the semantic judgment.
Technical details: In ASIMOV-Multimodal the alignment accuracy is 0.86 for Gemini 2.0 Flash, 0.85 for Robotics-ER and 0.88 with the constitution; under adversarial prompts that ask the model to flip what is desirable it falls to 0.28 and recovers to 0.76 with the constitution. On the injury set the values are 0.84, 0.82 and 0.88. These are classification accuracies on questions; the paper reports no measurement of a robot violating or respecting a constraint.
Evidence in the paper: §5 · Figure 29

Check your understanding: Is the safety work a test of the robot’s behaviour?

No. It measures how well the language model recognises unsafe instructions in questions; the robot-level safety layer is described as an interface to controllers from earlier work and is not evaluated here.

Part D

Understand the experiments

Four groups of tests: the embodied-reasoning model on benchmarks and in zero-shot control, the action model out of the box and under distribution shift, the specialisations and the adaptations. This part says what each figure shows and what it does not. Our re-calculations are in the folded examples.

The set-up: trial counts and how to read the charts

How it works

Most results of the action model are bar charts without printed numbers (Figures 16, 17, 26) or with values on the bars (Figures 21, 23, 24, 28). The trial counts are given only in the appendix text for some tests: 20 per specialist task, 12 for the spelling game, 10 per point of the fast-adaptation curves, 20 per task in the new-embodiment tests, 100 trials across the eight reasoning tasks, 50 per task in simulation, and 9 or 10 for the real zero-shot and in-context tests. Many figures give a “progress” score between 0 and 1 in addition to success.

Technical details

Our calculation With 10 trials a bar near 50% has a standard error of 16 points; with 20, 11; with 50, 7. An exact test calls two rates different only when they are about 50 points apart with 10 trials, 35 with 20 and 22 with 50. The pairwise t-test the paper mentions is not reported; only the printed bars are available to a reader.

Run the example: how far the headline gaps can be trusted

Our calculation

What each part does

Take the trial counts used in the paper: 9 to 400.
Compute the standard error and the smallest gap an exact test calls different.
Test the headline gaps.

import math
# How far can the paper's headline gaps be trusted? Trial counts used in the Gemini Robotics report.
def se(p, n): return 100 * math.sqrt(p * (1 - p) / n)
def fisher(a, n1, b, n2):
    m, N = a + b, n1 + n2
    pr = lambda x: math.comb(m, x) * math.comb(N - m, n1 - x) / math.comb(N, n1)
    p0 = pr(a)
    return sum(pr(x) for x in range(max(0, m - n2), min(m, n1) + 1) if pr(x) <= p0 + 1e-12)
print("trials | where                                                | StdErr at 50% | smallest gap an exact test calls different (baseline at 50%)")
for n, where in ((9, "real ALOHA 2 tests of Gemini Robotics-ER (9 or 10)"), (10, "fast-adaptation points of Figure 26"), (12, "spelling game"), (20, "specialist tasks; embodiment tests"), (50, "one simulated task"), (350, "one simulated row (7 tasks x 50)"), (400, "ERQA questions")):
    base = n // 2
    gap = next((100 * (k - base) / n for k in range(base, n + 1) if fisher(k, n, base, n) < 0.05), None)
    print(f"{n:6d} | {where:52s} | {se(0.5, n):9.1f} pts | {gap:.0f} points" if gap is not None else f"{n:6d} | {where:52s} | {se(0.5, n):9.1f} pts | none")
print()
print("headline gaps as exact tests")
for name, a, n1, b, n2 in (("simulation zero-shot: ER 186/350 vs Flash 94/350", 186, 350, 94, 350), ("ERQA: Pro 193/400 vs GPT-4o 188/400 (unpaired approximation)", 193, 400, 188, 400),
                           ("specialists, lunch-box: 20/20 vs multi-task diffusion 7/20", 20, 20, 7, 20), ("specialists, origami: 9/20 vs 1/20", 9, 20, 1, 20), ("specialists, place peas: 11/20 vs single-task diffusion 15/20", 11, 20, 15, 20),
                           ("reasoning-enhanced: 11/14 vs 4/14 (assumed 14 trials)", 11, 14, 4, 14), ("reasoning-enhanced: smallest soda 4/10 vs 3/10 (assumed 10 trials)", 4, 10, 3, 10), ("real ER: in-context 18/28 vs zero-shot 7/28", 18, 28, 7, 28)):
    print(f"  {name:66s} p = {fisher(a, n1, b, n2):.4f}")
print()
print("the paper says its trials were run back to back in random order and compared with a pairwise t-test; no test statistic, p-value or interval is printed in the report")

Output:

trials | where                                                | StdErr at 50% | smallest gap an exact test calls different (baseline at 50%)
     9 | real ALOHA 2 tests of Gemini Robotics-ER (9 or 10)   |      16.7 pts | 56 points
    10 | fast-adaptation points of Figure 26                  |      15.8 pts | 50 points
    12 | spelling game                                        |      14.4 pts | 50 points
    20 | specialist tasks; embodiment tests                   |      11.2 pts | 35 points
    50 | one simulated task                                   |       7.1 pts | 22 points
   350 | one simulated row (7 tasks x 50)                     |       2.7 pts | 8 points
   400 | ERQA questions                                       |       2.5 pts | 7 points

headline gaps as exact tests
  simulation zero-shot: ER 186/350 vs Flash 94/350                   p = 0.0000
  ERQA: Pro 193/400 vs GPT-4o 188/400 (unpaired approximation)       p = 0.7771
  specialists, lunch-box: 20/20 vs multi-task diffusion 7/20         p = 0.0000
  specialists, origami: 9/20 vs 1/20                                 p = 0.0084
  specialists, place peas: 11/20 vs single-task diffusion 15/20      p = 0.3203
  reasoning-enhanced: 11/14 vs 4/14 (assumed 14 trials)              p = 0.0213
  reasoning-enhanced: smallest soda 4/10 vs 3/10 (assumed 10 trials) p = 1.0000
  real ER: in-context 18/28 vs zero-shot 7/28                        p = 0.0066

the paper says its trials were run back to back in random order and compared with a pairwise t-test; no test statistic, p-value or interval is printed in the report

Our calculation assuming independent trials; each experiment is a single run, and the unpaired tests ignore the pairing the authors used.

Evidence in the paper

Appendix C.1 · Appendix D · §3.4

Check your understanding: Why does back-to-back random-order testing help even without printed statistics?

It spreads environmental changes evenly over the models, so large gaps are credible; it does not tell you how large a gap must be to exclude chance, which needs the counts.

Embodied-reasoning benchmarks and control (Tables 1 to 6)

How it works: Gemini 2.0 Flash and Pro Experimental are best in their model classes on ERQA, RealworldQA and BLINK; chain-of-thought prompting lifts ERQA scores by 3 to 10 points (Flash to 50.3, Pro to 54.8). Robotics-ER improves pointing, scores 48.3 on SUN-RGBD, and in zero-shot simulated control reaches 53% against 27% for 2.0 Flash; with ten in-context demonstrations 65% (simulation) and 65% (real, as printed).
Technical details: Our calculation The ERQA lead of Gemini 2.0 Pro Experimental over GPT-4o is 5 questions of 400 (an unpaired exact test would give p of 0.777). With chain of thought GPT-4o (50.5) is level with Gemini 2.0 Flash (50.3). The printed real-robot in-context average of 65% is 64.3% pooled over 28 trials (70, 56 and 67 per task average to 64.1). The simulated gain from Flash to Robotics-ER is large and robust, 94 against 186 successes out of 350.
Evidence in the paper: §2 · Tables 1 to 6

Check your understanding: Which of the Section 2 claims is best supported?

That Robotics-ER improves zero-shot control over Gemini 2.0 Flash in simulation: 350 trials per row, a gap of about 26 points and consistent in six of seven tasks.

Out of the box: 20 tasks and fine-grained instructions (Figures 16 and 17)

How it works

On 20 tasks sampled from the training set, Gemini Robotics exceeds 80% success on half of them, excels at deformable objects (folding cloth, wrapping a wire around headphones), and is described as the only method with non-zero success on some of the hardest ones. On 25 precise instructions in five scenes, both Gemini Robotics and π0 re-implement beat the diffusion baseline in simple scenes, and Gemini Robotics is better with new objects and detailed language such as “top black container”.

Technical details

Our calculation Reading the bars of Figure 16 by eye (error about 0.05), the means over the 20 tasks are about 0.569 for Gemini Robotics, 0.245 for π0 re-implement and 0.205 for multi-task diffusion; 10 tasks are at 0.8 or above. π0 ties on fold pants and does better on open glasses case; multi-task diffusion ties on bottle from holder. The multi-task diffusion bar at “open pink folder” is about 0.1, so Gemini Robotics is not the only method with non-zero success there.

Run the example: the 20 out-of-the-box tasks

Our calculation

What each part does

Type in the bar heights read from the chart.
Count tasks above 0.8 and tasks where a baseline ties or wins.
Check the tasks the text names as non-zero only for Gemini Robotics.

# Figure 16: success rates of the 20 out-of-the-box tasks, READ BY EYE from the bar chart (the paper prints no values; error about 0.05).
tasks = [("close laptop", 1.0, 0.9, 0.7), ("fold pants", 1.0, 1.0, 1.0), ("pour pulses", 1.0, 0.4, 0.3), ("stack measuring cup", 1.0, 0.4, 0.1), ("fold pink cloth", 0.9, 0.6, 0.2), ("pick shoelace", 0.9, 0.8, 0.6),
         ("place grater compartment", 0.9, 0.4, 0.2), ("open pink folder", 0.8, 0.0, 0.1), ("ornaments on table", 0.8, 0.0, 0.1), ("wrap headphone wires", 0.8, 0.0, 0.0), ("fold blue shirt", 0.5, 0.2, 0.4), ("insert red block", 0.5, 0.0, 0.0),
         ("bottle from holder", 0.3, 0.0, 0.3), ("flatten blue shirt", 0.3, 0.0, 0.0), ("pick marker cap", 0.28, 0.0, 0.0), ("unfold mat", 0.2, 0.0, 0.0), ("dress on hanger", 0.1, 0.0, 0.0), ("open glasses case", 0.1, 0.2, 0.1), ("hang loofah", 0.0, 0.0, 0.0), ("insert lace shoe", 0.0, 0.0, 0.0)]
names = ["Gemini Robotics", "pi0 re-implement", "Multi-task diffusion"]
print(f"{len(tasks)} tasks; mean success (our reading): " + ", ".join(f"{n} {sum(t[j + 1] for t in tasks) / 20:.3f}" for j, n in enumerate(names)))
print("tasks at 0.8 or above, Gemini Robotics:", sum(t[1] >= 0.8 for t in tasks), "of 20 (the text: 'half of the tasks ... exceeding 80%')")
print("tasks where a baseline ties or beats Gemini Robotics:", [t[0] for t in tasks if max(t[2], t[3]) >= t[1] and max(t[2], t[3]) > 0])
print("tasks where Gemini Robotics is at zero:", [t[0] for t in tasks if t[1] == 0])
print()
print("the text names 'open pink folder', 'insert red block' and 'wrap the wire around the headphone' as tasks where Gemini Robotics is the only method with non-zero success")
for name in ("open pink folder", "insert red block", "wrap headphone wires"):
    t = next(x for x in tasks if x[0] == name)
    print(f"  {name:22s} Gemini {t[1]:.1f}  pi0 {t[2]:.1f}  multi-task diffusion {t[3]:.1f}  -> baselines at zero: {t[2] == 0 and t[3] == 0}")
print("the multi-task diffusion bars at open pink folder and ornaments on table are small but visible (about 0.1, one trial in ten if 10 trials per task)")
print()
print("what a single bar can mean: with 10 trials per task each trial is 0.1; the paper does not give the number of trials per task of Figure 16")

Output:

20 tasks; mean success (our reading): Gemini Robotics 0.569, pi0 re-implement 0.245, Multi-task diffusion 0.205
tasks at 0.8 or above, Gemini Robotics: 10 of 20 (the text: 'half of the tasks ... exceeding 80%')
tasks where a baseline ties or beats Gemini Robotics: ['fold pants', 'bottle from holder', 'open glasses case']
tasks where Gemini Robotics is at zero: ['hang loofah', 'insert lace shoe']

the text names 'open pink folder', 'insert red block' and 'wrap the wire around the headphone' as tasks where Gemini Robotics is the only method with non-zero success
  open pink folder       Gemini 0.8  pi0 0.0  multi-task diffusion 0.1  -> baselines at zero: False
  insert red block       Gemini 0.5  pi0 0.0  multi-task diffusion 0.0  -> baselines at zero: True
  wrap headphone wires   Gemini 0.8  pi0 0.0  multi-task diffusion 0.0  -> baselines at zero: True
the multi-task diffusion bars at open pink folder and ornaments on table are small but visible (about 0.1, one trial in ten if 10 trials per task)

what a single bar can mean: with 10 trials per task each trial is 0.1; the paper does not give the number of trials per task of Figure 16

Readings by eye from a bar chart with no printed values; they are approximate.

Evidence in the paper

§3.2 · §3.3 · Figures 16 and 17 · Appendix C.1.1

Check your understanding: Why is a figure without printed values less useful?

Readers cannot recompute averages or tests; every statement about the bars depends on how they are read, and trial counts per task are not given.

Generalisation: visual, instruction and action (Figure 21)

How it works

An 85-task benchmark is 20% in-distribution, 28% visual, 28% instruction and 24% action variations. Average progress scores: instruction 0.88 in distribution and 0.65 out of it, visual 0.88 and 0.75, action 0.69 and 0.60, against 0.32 to 0.36 (π0 re-implement) and 0.26 to 0.34 (multi-task diffusion) out of distribution. The model keeps non-zero progress where baselines collapse, for instance with instructions in a new language (0.68 against 0.04 and 0.12).

Technical details

Our calculation The printed out-of-distribution averages are not plain means of the sub-bars: for π0 on the visual axis the printed 0.36 compares with 0.320 as a mean of the three bars, and on the action axis Gemini’s printed 0.60 compares with 0.635, so the averages are weighted by task counts that are not given. On the action axis π0 keeps only 35% of its in-distribution progress, against 87% for Gemini Robotics.

Run the example: the generalisation averages

Our calculation

What each part does

Type in the printed progress scores.
Compare the printed averages with plain means of the sub-bars.
Compute how much of the in-distribution score each model keeps.

# Figure 21: average progress scores (0 to 1) on the 85-task generalisation benchmark. Order: Gemini Robotics, pi0 re-implement, multi-task diffusion.
fig = {
 "instruction": {"in-distribution avg": (0.88, 0.33, 0.55), "out-of-distribution avg": (0.65, 0.32, 0.34), "Rephrasing": (0.79, 0.50, 0.61), "Typo": (0.54, 0.44, 0.36), "New language": (0.68, 0.04, 0.12), "Descriptive": (0.61, 0.25, 0.25)},
 "visual": {"in-distribution avg": (0.88, 0.33, 0.55), "out-of-distribution avg": (0.75, 0.36, 0.34), "Distractors": (0.77, 0.50, 0.29), "New background": (0.75, 0.32, 0.29), "Lighting": (0.71, 0.14, 0.46)},
 "action": {"in-distribution avg": (0.69, 0.31, 0.32), "out-of-distribution avg": (0.60, 0.11, 0.26), "New object instance": (0.39, 0.04, 0.21), "Different position": (0.88, 0.22, 0.33)}}
names = ["Gemini Robotics", "pi0 re-implement", "Multi-task diffusion"]
print("printed out-of-distribution average against the plain mean of its sub-bars")
for axis, d in fig.items():
    subs = [v for k, v in d.items() if "avg" not in k]
    for j, n in enumerate(names):
        plain = sum(s[j] for s in subs) / len(subs); pr = d["out-of-distribution avg"][j]
        print(f"  {axis:11s} {n:21s} printed {pr:.2f}  plain mean {plain:.3f}  difference {pr - plain:+.3f}")
print()
print("the in-distribution drop to out-of-distribution, per axis (progress points)")
for axis, d in fig.items():
    i, o = d["in-distribution avg"], d["out-of-distribution avg"]
    print(f"  {axis:11s} " + "; ".join(f"{n.split()[0]} {i[j]:.2f}->{o[j]:.2f} ({o[j] / i[j]:.0%} kept)" for j, n in enumerate(names)))
print()
print("benchmark composition (Section 3.4): 85 tasks = 20% in distribution + 28% visual + 28% instruction + 24% action")
for k, p in (("in distribution", 20), ("visual", 28), ("instruction", 28), ("action", 24)):
    print(f"  {k:16s} {p}% of 85 = {85 * p / 100:.1f} tasks")
print(f"  the shares add up to {20 + 28 + 28 + 24}%")
print("  the in-distribution average is identical (0.88, 0.33, 0.55) for the instruction and visual axes, so those two axes share their in-distribution tasks")

Output:

printed out-of-distribution average against the plain mean of its sub-bars
  instruction Gemini Robotics       printed 0.65  plain mean 0.655  difference -0.005
  instruction pi0 re-implement      printed 0.32  plain mean 0.307  difference +0.013
  instruction Multi-task diffusion  printed 0.34  plain mean 0.335  difference +0.005
  visual      Gemini Robotics       printed 0.75  plain mean 0.743  difference +0.007
  visual      pi0 re-implement      printed 0.36  plain mean 0.320  difference +0.040
  visual      Multi-task diffusion  printed 0.34  plain mean 0.347  difference -0.007
  action      Gemini Robotics       printed 0.60  plain mean 0.635  difference -0.035
  action      pi0 re-implement      printed 0.11  plain mean 0.130  difference -0.020
  action      Multi-task diffusion  printed 0.26  plain mean 0.270  difference -0.010

the in-distribution drop to out-of-distribution, per axis (progress points)
  instruction Gemini 0.88->0.65 (74% kept); pi0 0.33->0.32 (97% kept); Multi-task 0.55->0.34 (62% kept)
  visual      Gemini 0.88->0.75 (85% kept); pi0 0.33->0.36 (109% kept); Multi-task 0.55->0.34 (62% kept)
  action      Gemini 0.69->0.60 (87% kept); pi0 0.31->0.11 (35% kept); Multi-task 0.32->0.26 (81% kept)

benchmark composition (Section 3.4): 85 tasks = 20% in distribution + 28% visual + 28% instruction + 24% action
  in distribution  20% of 85 = 17.0 tasks
  visual           28% of 85 = 23.8 tasks
  instruction      28% of 85 = 23.8 tasks
  action           24% of 85 = 20.4 tasks
  the shares add up to 100%
  the in-distribution average is identical (0.88, 0.33, 0.55) for the instruction and visual axes, so those two axes share their in-distribution tasks

Arithmetic on the printed bars; progress scores are partial credit, not success rates.

Evidence in the paper

§3.4 · Figures 18 to 21 · Appendix C.1.3

Check your understanding: What is a progress score?

A number from 0 to 1 for the share of a task completed, defined per task in the appendix (for example 0.5 for picking and handing over a banana without placing it). It is more continuous than success but depends on the authors’ rubric.

Specialisation, reasoning and fast adaptation (Figures 23, 24, 26)

How it works: After specialisation Gemini Robotics solves six long-horizon dexterous tasks with a mean of 79%, 100% on the lunch-box and 83% on the spelling game (including 4 of 6 hand-drawn sketches). The reasoning-enhanced variant lifts average success on eight out-of-distribution tasks from 0.40 to 0.73. With 5, 20 and 100 demonstrations, finetuning reaches over 70% on 7 of 8 shorter tasks with at most 100 demonstrations, and 100% on two.
Technical details: Our calculation Against the best baseline the lunch-box (20/20 against 7/20) and spelling (10/12 against 1/12) gaps are far beyond chance and origami (9/20 against 1/20) has p of 0.0084; on scooping nuts (p of 0.4872), playing cards and place peas (11/20 against 15/20, p of 0.3203) it is not distinguishable from the baselines, and single-task diffusion trained from scratch is ahead on place peas. For Figure 24 the rates fit a different trial count for several tasks (for example 14 for matching pile and 15 for same color), so “100 trials across 8 tasks” is not 12.5 per task.
Evidence in the paper: §4.1 · §4.2 · §4.3 · Figures 23, 24 and 26 · Appendix D

Check your understanding: For which specialist tasks is the evidence clearest?

The ones the baselines essentially fail: the lunch-box, the spelling game and origami. On the easier tasks baselines are competitive and with 20 trials the differences are within chance.

New embodiments (Figures 27 and 28)

How it works: Without changing the recipe, the model is finetuned to a two-armed Franka robot and to the Apptronik Apollo humanoid with five-fingered hands. On the Franka, in-distribution success is on par or slightly better than a single-task diffusion policy and averages 63% over its tasks; under visual and action shifts the finetuned model keeps more of its performance than the baseline. For example in new object instances the progress is 0.47 against 0.20 in the figure.
Technical details: Only a single-task diffusion baseline is used, because it is the only one the authors consider as an adaptation baseline; instruction generalisation is not studied because that baseline has no language input. The humanoid has qualitative examples and no number in the main text. The authors call these results preliminary.
Evidence in the paper: §4.4 · Figures 27 and 28 · Appendix D.4

Check your understanding: What does the Franka result show about the base model?

That finetuning can keep some of the generalisation of the base model on a different robot; it does not show zero-shot transfer, which the authors leave for future work.

What the results show

Gemini 2.0 and Robotics-ER are strong at pointing and spatial questions, and the ER model controls simulated robots with no action data about twice as well as 2.0 Flash.
The action model clearly beats a re-implemented π0 and diffusion policies on long-horizon and deformable-object tasks, and keeps more of its performance under language, visual and action shifts.
Finetuning from the generalist works where training from scratch gives 0%, and 100 demonstrations are enough for several shorter tasks.

What they do not show

Which ingredient is responsible: the backbone, the non-action data, the data volume or the latency design. No ablation separates them.
That the gaps on the easier tasks (scooping nuts, playing cards, placing peas) are more than chance.
That the model transfers to a new robot without finetuning, or that its safety work changes robot behaviour.
Anything reproducible: models, data sizes and statistics are not published.

Part E · optional

Engineering deep dive

The output formats, the robot API, the baselines and the evaluation tasks, for readers who want to prompt the models or audit the work. Everything is collapsed; Parts A to D do not depend on it.

Spatial output conventions (Appendix B.1)

Reported by the paper 2D boxes: (y0, x0, y1, x1), with (y0, x0) the top-left corner, normalised to integers from 0 to 1000.
Reported by the paper Points: (y, x) tuples, returned as a JSON list of dictionaries with the keys in_frame, point and label.
Reported by the paper 3D boxes: x, y, z, w, h, l and three Euler angles, as short token sequences truncated to two decimals.
Reported by the paper Top-down grasps: y, x and a rotation angle in integer degrees between −90 and 90; 0 is where the gripper fingers are aligned with the horizontal image axis.
Reported by the paper Pointing benchmarks: accuracy is the share of points inside the ground-truth mask; Pixmo-Point has no masks, so circles of radius 25 around the points are used; GPT and Claude get instruction-based formatting and Molmo’s XML output is parsed.

The zero-shot control loop (Section 2.3)

Reported by the paper The model gets a task-agnostic system prompt, documentation of a robot API (get_grasp_pose, detect_object, open_gripper, close_gripper, move_gripper, and a state readout), and the task. In each step it receives images, the robot state and the execution feedback of the last code, and outputs a plan and code. The tool library exposes grasp point and angle prediction, 2D boxes and 2D pointing. The ALOHA 2 cell has an arm on each side of a 0.8 m by 0.4 m table and the tasks randomise object positions: lift banana, banana in bowl, mug on plate, bowl on rack, banana handover, fruit bowl and pack toy in simulation; banana handover, fold dress and wiping on the real robot.

The baselines (Appendix C.2)

As described in Appendix C.2; the Gemini Robotics batch size and steps are not given. Baselines ran locally on an RTX 4090, Gemini Robotics in the cloud.
Model	Description	Training (generalist stage)	Training (specialist stage)
π0 re-implement	PaliGemma VLM plus a diffusion-transformer action expert, trained on the authors’ mixture	batch 2,048, 300 thousand steps	batch 2,048, 50 thousand steps from the generalist checkpoint
Multi-task diffusion	ALOHA Unleashed diffusion policy with a CLIP text encoder	batch 512, 2 million steps	batch 512, 1 million steps
Single-task diffusion	ALOHA Unleashed diffusion policy, one task	not used	from scratch, batch 512, 2 million steps
π0 openpi	the public checkpoint	weaker than the re-implementation in the authors’ tests	used only as an extra baseline in an appendix figure

The evaluation tasks

Reported by the paper Out of the box (Figure 16): 20 tasks from the dataset in a laundry room, kitchen and office, such as fold pants, stack measuring cup, open pink folder, insert red block and wrap the wire around the headphone.
Reported by the paper Instruction following: 25 instructions in five scenes, including novel scenes with unseen objects and receptacles.
Reported by the paper Generalisation: 85 tasks; four lunch-bag tasks under typos, Spanish, rephrasing and descriptive modifiers; visual changes (distractors, background, lighting); action changes (different initial positions, new object instances such as dress sizes XS and M instead of S).
Reported by the paper Reasoning tasks (Appendix D.2): eight tasks in three groups, one-step reasoning (for example sort the bottom right mouse into the matching pile), semantic generalisation (put the Japanese fish delicacy in the lunch-box) and spatial understanding (pack the smallest coke soda).
Reported by the paper Fast adaptation: draw card, play card, pour lettuce, salad dressing, seal container, put container in lunch-box, zip lunch-box and origami first fold, with 5, 20 and 100 demonstrations and 10 trials per point.
Reported by the paper New embodiments: four dexterous industrial tasks on the bi-arm Franka (such as hanging tape on a wall hook, inserting a plug, placing belts on pulleys), 20 trials per task, with progress scores.

Limitations stated by the authors (Section 6)

Gemini 2.0 may struggle to ground spatial relationships across long videos, and its numeric predictions (points and boxes) may not be precise enough for fine-grained control.
Combining multi-step reasoning with precise dexterous motion in novel situations remains to be done.
Future work: more use of simulation for diverse, contact-rich data, and multi-embodiment experiments aiming at zero-shot transfer to new robots.

Part F

Critical paper review

Everything above reports what the paper says. This part is our independent reading: where its numbers and text disagree, where a comparison changes more than the thing being credited, and what a closed technical report leaves untestable. The system is impressive and the experiments cover a broad range of tasks on a real robot; the findings concern the strength of the evidence for why it works. Each one links back to the concept it affects, and the calculations are reproducible in the examples.

Numbers that do not agree

F1. “The only method with non-zero success” is not what Figure 16 shows for the pink folder Numerical inconsistency

What the paper says. For the more challenging tasks (open pink folder, insert red block, wrap the wire around the headphone), Gemini Robotics is the only method that can achieve non-zero success.

What we found. Read by eye, the multi-task diffusion bar at “open pink folder” is about 0.1, and so is its bar at “ornaments on table”; for insert red block and wrap headphone wires the baselines are at zero. One trial in ten would explain the bar.

Why it matters. The claim holds for two of the three named tasks. It is a small point, but it shows that statements about the bars cannot be checked without printed values and trial counts. Concept: Out of the box · example
F2. “65% in both simulation and the real world” is 64.3% in the real world Numerical inconsistency

What the paper says. Gemini Robotics-ER achieves 65% in both simulation and the real world with in-context learning.

What we found. Simulation: 65.1% over 350 trials. Real robot (Table 6): 70% of 10, 5 of 9 (printed 56) and 6 of 9 (67) pool to 18 of 28, 64.3%, and the mean of the three tasks is 64.1; the printed average is 65. The zero-shot average (25) is consistent.

Why it matters. A rounding-level difference; the point is that a real-robot average over 28 trials is not a stable number to the whole percent, as one trial is worth about 10 points. Concept: In-context learning · example
F3. The printed generalisation averages are not plain means, and the weights are not given Numerical inconsistency

What the paper says. Figure 21 reports in-distribution and out-of-distribution average progress for three models on the three axes.

What we found. The printed out-of-distribution average differs from the plain mean of its sub-bars by up to 0.04 (π0 on the visual axis: 0.36 against 0.320; Gemini Robotics on the action axis: 0.60 against 0.635). Averages weighted by the number of tasks per sub-condition would explain this, but the task counts per bar are not given.

Why it matters. The ranking is not at risk, but the averages cannot be reproduced from the printed bars. Concept: Generalisation · example
F4. “100 trials across 8 tasks” does not give equal trials per task Numerical inconsistency

What the paper says. For the reasoning-enhanced and vanilla models, the authors perform 100 trials across 8 different tasks.

What we found. The printed rates of Figure 24 can be produced by integer counts only for different trials per task: 0.79 against 0.29 needs 14 (or 24, 28), 0.60 against 0.27 needs 15 (or 30), 0.73 against 0.45 needs 11 (22), while 0.50 against 0.20 and 0.40 against 0.30 fit 10. No single equal number per task fits all eight pairs, and 100 over 8 is 12.5.

Why it matters. Trial counts differ between tasks and are not stated; differences such as 0.40 against 0.30 on the smallest-soda task could be one trial. Concept: Reasoning variant · Specialisation · example

How far the claims go

F5. The embodied-reasoning leads over other vendors are small and come from the authors’ own benchmark Method caveat

What the paper says. Gemini 2.0 Flash and Pro Experimental achieve a new state of the art on all three benchmarks in their respective model classes; ERQA is the most challenging.

What we found. On ERQA the lead of 2.0 Pro Experimental over GPT-4o is 48.3 against 47.0, five questions of 400 (standard error about 2.5 points), and with chain of thought GPT-4o (50.5) and Flash (50.3) are level. The models were evaluated in February 2025 with the authors’ own prompts for other vendors’ models, on a benchmark made by the Gemini team. The report describes the training data of Robotics-ER only as embodied-reasoning data, so overlap with the benchmarks’ domains cannot be checked.

Why it matters. The paper supports “strong embodied reasoning”; it does not support a ranking among the leading models on ERQA. Concept: ERQA · Benchmarks and control · example
F6. The 3D detection comparison is not like for like Method caveat

What the paper says. Gemini Robotics-ER achieves a new state of the art on SUN-RGBD 3D detection (48.3 AP@15).

What we found. The strongest expert model (ImVoxelNet, 43.7) was measured on an easier set of 10 categories (the table’s asterisk); Gemini 2.0 Flash and Pro score 30.7 and 32.5, below it. The expert models use closed sets of categories, while Gemini is queried with open-vocabulary prompts, and the paper does not say whether the Robotics-ER training data include SUN-RGBD-like scenes.

Why it matters. The result shows competitive 3D detection from a language model, not a clean win over specialised detectors. Concept: Embodied reasoning · example
F7. The credit for the results goes to the backbone, but the backbone is never isolated Method caveat

What the paper says. Improvements result from the larger and more powerful VLM backbone, including the vision encoder of Gemini 2.0, combined with diverse training data; the key difference with the baselines is the much more powerful Gemini-based backbone.

What we found. The comparison is Gemini Robotics (cloud backbone of unstated size, trained on actions plus web, code and reasoning data) against π0 re-implement (a 3-billion-class PaliGemma backbone, according to the π0 paper) and diffusion policies trained on the action data only, run locally. Backbone size, non-action data, the embodied-reasoning pretraining and the cloud serving all differ at once, and the authors call their explanation a speculation. The only control is Gemini Robotics trained from scratch on specialist data (0%), which tests the generalist stage, not the backbone.

Why it matters. The results show that this whole system beats the baselines; they do not tell a builder whether to invest in a bigger backbone, in more non-action data or in embodied-reasoning pretraining. Concept: Baselines · The data · Out of the box
F8. On the easier specialist tasks the baselines are not beaten Method caveat

What the paper says. Gemini Robotics is the only model that can consistently solve the extremely challenging tasks; baselines are competitive on the easier ones.

What we found. With 20 trials, Gemini Robotics’s 20/20 on lunch-box, 10/12 on spelling and 9/20 on origami are far above the baselines (p of 0.0000, 0.0006 and 0.0084 against the best baseline). On scooping nuts (20/20 against 18/20), playing cards (18/20 against 17/20) and placing peas (11/20 against 15/20, in favour of single-task diffusion trained from scratch) the gaps are within chance. The fine-tuning budgets differ greatly across models (50 thousand steps for π0, 1 million for multi-task diffusion, not given for Gemini Robotics) and the checkpoints are selected by the authors.

Why it matters. The headline of the section, a 79% mean, is driven by three tasks; the paper’s own wording about easier tasks is accurate. Concept: Specialisation · Specialisation results · example
F9. The reasoning-enhanced comparison changes labels and training stage at once Method caveat

What the paper says. After finetuning on a re-labelled dataset that bridges action prediction to embodied reasoning, the model generalises to novel situations combining several distribution shifts.

What we found. The legend contrasts a “reasoning-enhanced specialist” with the vanilla Gemini Robotics generalist. The two differ in the labels (re-labelled data with trajectory reasoning), the training stage (a specialist finetune) and the output format. There is no control that is finetuned on the same data without reasoning outputs, and the tasks (such as smallest soda: 0.40 against 0.30) were designed by the authors with trial counts that are not stated.

Why it matters. The result is that this finetune helps on these tasks; whether reasoning is the cause is not established. Concept: The reasoning variant · Specialisation results

What is not tested

F10. A closed system: sizes, data and statistics are not published Limitation

What the paper says. The report is a technical description of new model families; a model card is given in the appendix.

What we found. No parameter counts for Gemini Robotics or its backbone, no number of episodes or hours beyond “thousands of hours”, no chunk length, no training steps or compute for the Gemini models, no model weights; the model card describes inputs, outputs and hardware (TPU v4, v5p and v6e) but not quantities. The paper says trials are compared with a pairwise t-test, yet no statistic, p-value or interval appears in the report. Only ERQA and the ASIMOV datasets are released.

Why it matters. The results cannot be reproduced or independently re-analysed; the guide’s own calculations are the best a reader can do from the printed bars. Concept: The action model · The data · example
F11. “Generalist” means one robot type plus finetuning to others Limitation

What the paper says. Gemini Robotics is a generalist model that supports different robot embodiments.

What we found. All the action data come from the ALOHA 2 fleet. Out-of-the-box and generalisation tests change objects, scenes, language and positions on that robot; other robots are reached by finetuning (63% on average on the bi-arm Franka, qualitative examples on a humanoid), compared only with a single-task diffusion policy. Zero-shot transfer to a new robot is listed as future work.

Why it matters. The evidence supports strong generalisation within one embodiment and quick finetuning to others, but not cross-embodiment generality in the sense of the Open X-Embodiment collection. Concept: New embodiments · The data
F12. The safety results measure answers to questions, not robot behaviour Ambiguity

What the paper says. We see initial assurances that the rigorous safety standards upheld by non-robotics models also apply to the embodied and robotics-focused models.

What we found. The measurements are classification accuracies on questions (86% or 85% for the base models on ASIMOV-Multimodal, 88% with a constitution; 28% under adversarial prompts without one, 76% with) and a 96% against 20% rejection of bias-inducing pointing queries, with no sample sizes in the main text. Physical action safety is described as the job of low-level controllers, cited from earlier work, and is not tested with the Gemini Robotics model; no robot trial measures a violation.

Why it matters. The benchmarks are useful and released, but a reader should not read “initial assurances” as a demonstration that the robot behaves safely. Concept: Safety

Knowledge check

Try each question first, then reveal the explanation.

Question 1

In your own words: how does Gemini Robotics get from a frontier language model to robot control?

Explanation

Gemini 2.0 is trained for embodied reasoning, so it can point, box, plan paths and grasp; a distilled version becomes a cloud backbone that is trained on robot demonstrations plus web, code and reasoning data; a local decoder turns its output into action chunks and covers the latency.

Question 2

What does the in-context control result show, and what is its limit?

Explanation

With ten demonstrations in the prompt the reasoning model reaches 65% in simulation and about 64% on a real robot, mostly on dexterous bimanual tasks. Nothing is trained, so the skill is not kept, and the real-robot averages are over 9 or 10 trials per task.

Question 3

Which specialist results are clearly beyond chance, and which are not?

Explanation

Lunch-box (20/20 against 7/20 for the best baseline), the spelling game and origami are far beyond chance; scooping nuts, playing cards and placing peas are within chance of the baselines.

Question 4

Name one thing the report does not tell you.

Explanation

For example: the parameter counts, the number of training episodes, the chunk length, the number of trials of most bars, or any statistic from the pairwise t-test the authors say they use.

Glossary

Embodied reasoning (ER): A vision-language model’s ability to ground objects and spatial concepts in the real world and use them for robotics.
ERQA: Embodied Reasoning Question Answering: the paper’s 400-question multiple-choice benchmark.
Gemini Robotics-ER: Gemini 2.0 Flash trained for embodied reasoning; outputs text such as coordinates.
Gemini Robotics: The vision-language-action model built from the embodied-reasoning model; outputs action chunks.
Pointing: Returning the (y, x) position of an object, part, free space or affordance named in language.
Top-down grasp: A grasp given as an image point and a rotation angle.
Multi-view correspondence: Finding where points of one camera view appear in another view.
Code as policy: Controlling a robot by letting the model write code that calls perception and motion functions.
In-context learning: Conditioning the model on demonstrations in the prompt, with no weight updates.
Action chunk: A short sequence of future actions produced together.
Cloud backbone and local decoder: The two parts of the action model: a large model in the cloud and a small decoder on the robot.
ALOHA 2: A low-cost two-armed teleoperation platform on which the robot data were collected.
Progress score: A number from 0 to 1 for the share of a task completed.
Specialist: The generalist checkpoint finetuned on thousands of episodes of one task.
Constitution: A list of safety rules added to the prompt so that the model judges instructions against them.
ASIMOV: The released datasets for semantic action safety: visual and text questions and injury-report scenarios.
Semantic action safety: Respecting everyday safety constraints in open environments, such as not placing a toy on a hot stove.
π0 re-implement: The authors’ own re-implementation of π0, trained on their data; the main baseline.
Diffusion policy: An imitation policy that generates actions by removing noise in several steps.
Chain-of-thought prompting: Asking the model to reason step by step before answering.

References

Numbered notes in the text, like ^[1], point here. Each entry says what it is cited for and how that was checked. Papers and books can be revised or misread, so follow the links before relying on a detail. Code marked “Toy model” is our own illustration of an idea, not the code of the cited system.

Gemini Robotics Team, Google DeepMind (2025). Gemini Robotics: Bringing AI into the physical world. arXiv. arXiv:2503.20020
Cited for:
- Gemini Robotics-ER (Embodied Reasoning) extends Gemini’s multimodal reasoning into the physical world with better spatial and temporal understanding, supporting object detection, pointing, trajectory and grasp prediction, multi-view correspondence and 3D bounding box prediction; Gemini Robotics builds on top of it. Checked against the paper’s abstract.
- Gemini Robotics is a vision-language-action model built on Gemini 2.0 that outputs robot control directly; reported strengths include fluid manipulation of varied objects, robustness to changes in object type and position, operation in unseen settings and open-vocabulary instruction following. Checked against the paper’s abstract.
- With further fine-tuning the model can take on long-horizon, highly dexterous tasks, learn new short-horizon tasks from as few as 100 demonstrations and adapt to new robot embodiments; the authors also discuss safety considerations. Checked against the paper’s abstract.

Go further

All paper guides · Back to the VLA track

How to use this page

Start here: the paper at a glance

The problem

The main contribution

How the model works, in one paragraph

What it learns from, and what it is tested on

What the paper reports

What to keep in mind

Understand the method

1. Embodied reasoning: answers in coordinates

2. Zero-shot control by writing code

3. Few-shot control by in-context demonstrations

4. The action model: a cloud backbone and a local decoder

5. The data: robot actions and everything else

6. A variant that predicts trajectories before acting

Understand the data and the training

ERQA: a benchmark for embodied reasoning

Baselines and the evaluation protocol

The specialisation recipe

Safety: content, semantic action and the low-level layer

Understand the experiments

The set-up: trial counts and how to read the charts

Embodied-reasoning benchmarks and control (Tables 1 to 6)

Out of the box: 20 tasks and fine-grained instructions (Figures 16 and 17)

Generalisation: visual, instruction and action (Figure 21)

Specialisation, reasoning and fast adaptation (Figures 23, 24, 26)

New embodiments (Figures 27 and 28)

What the results show

What they do not show

Engineering deep dive

Critical paper review

Numbers that do not agree

F1. “The only method with non-zero success” is not what Figure 16 shows for the pink folder Numerical inconsistency

F2. “65% in both simulation and the real world” is 64.3% in the real world Numerical inconsistency

F3. The printed generalisation averages are not plain means, and the weights are not given Numerical inconsistency

F4. “100 trials across 8 tasks” does not give equal trials per task Numerical inconsistency

How far the claims go

F5. The embodied-reasoning leads over other vendors are small and come from the authors’ own benchmark Method caveat

F6. The 3D detection comparison is not like for like Method caveat

F7. The credit for the results goes to the backbone, but the backbone is never isolated Method caveat

F8. On the easier specialist tasks the baselines are not beaten Method caveat

F9. The reasoning-enhanced comparison changes labels and training stage at once Method caveat

What is not tested

F10. A closed system: sizes, data and statistics are not published Limitation

F11. “Generalist” means one robot type plus finetuning to others Limitation

F12. The safety results measure answers to questions, not robot behaviour Ambiguity

Knowledge check

Question 1

Question 2

Question 3

Question 4

Glossary

References

Go further