Papers, concepts, architecture blocks, problems and open questions, joined by explained relationships. Each relationship says how strong its evidence is: stated in the paper, cited without a stated dependency, documented in our guide, or our interpretation. A later paper is not shown as building on an earlier one just because they share a technique.
Select any item to see its connections.
-
Flow Matching introduces Conditional flow matching
Proposes training a flow model by regressing a conditional target, with the same gradients as the intractable marginal objective.
Stated in the paper Established
-
Flow Matching uses Flow-based generative modelling
Builds on continuous-time generative flows (continuous normalising flows) and gives them a simulation-free training method.
Stated in the paper Established
-
Flow Matching uses Straight-line paths and rectification
Its optimal-transport conditional path moves each sample along a straight line at constant speed; the guide shows this holds per example, not for the learned marginal field.
Documented in our guide Established
-
Rectified Flow addresses Straight-line paths and rectification
Studies what rectification preserves (both distributions, no larger convex cost) and shows straight fixed points need not be optimal.
Stated in the paper Established
-
Rectified Flow addresses Flow-based generative modelling
A theory of one family of flow-based generative procedures; no robot experiments.
Documented in our guide Established
-
Rectified Flow is related to Flow Matching
Both regress straight-line directions between paired noise and data; the guides treat them as companions. Not a citation claim either way.
Our interpretation Possible connection
-
Flow-based action generation builds on Conditional flow matching
Flow-based action generation trains an action network with the conditional flow-matching objective.
Documented in our guide Established
-
RT-2 uses Discrete action tokens
Writes every action as eight integer tokens (256 bins per dimension), following RT-1's discretisation.
Stated in the paper Established
-
RT-2 introduces Co-fine-tuning on web and robot data
Proposes co-fine-tuning on robot and web data as the recipe; the ablation compares it with fine-tuning and training from scratch.
Stated in the paper Established
-
RT-2 uses Pretrained vision-language backbone
Fine-tunes PaLI-X (5B and 55B) and PaLM-E (12B) so one set of weights does language and action.
Stated in the paper Established
-
RT-2 evaluates Semantic transfer from web knowledge
Tests symbol understanding, reasoning and person recognition with instructions absent from the robot data.
Stated in the paper Established
-
RT-2 evaluates Writing a plan before acting
A chain-of-thought variant writes a plan before the action; shown only in example rollouts, with no success rates.
Documented in our guide Established
-
RT-2 addresses Generalising beyond the robot data
The aim of the paper: use web knowledge to generalise to new objects, backgrounds, environments and instructions.
Stated in the paper Established
-
π0 uses Pretrained vision-language backbone
Starts from a PaliGemma vision-language backbone.
Stated in the paper Established
-
π0 introduces Separate action expert
Adds a second set of weights for the state and action tokens; the guide notes the paper gives no numbers for this choice.
Stated in the paper Established
-
π0 uses Flow-based action generation
Its action expert refines noisy action tokens with flow matching to produce 50-step chunks.
Stated in the paper Established
-
π0 builds on Flow Matching
Uses the flow-matching training method; the guide links the two.
Documented in our guide Established
-
π0 uses Action chunking
Generates 50-step chunks and executes part of each before replanning.
Stated in the paper Established
-
π0 uses Cross-embodiment pretraining
Pre-trains on a mixture of seven robot configurations and 68 tasks.
Stated in the paper Established
-
π0 uses Writing a plan before acting
A high-level language policy can split a task into steps for the low-level model.
Stated in the paper Established
-
π0 addresses Smooth continuous actions from a language-model backbone
The flow-matching expert is the paper's answer to smooth, high-rate continuous control.
Documented in our guide Established
-
π0 addresses Scarce and fragmented robot data
Pre-trains on a large mixed collection and post-trains on curated data.
Stated in the paper Established
-
GR00T N1 introduces Dual-system model
A slower vision-language module (System 2) and a faster flow-matching module (System 1), trained together.
Stated in the paper Established
-
GR00T N1 uses Flow-based action generation
System 1 is a diffusion transformer trained with flow matching, producing a chunk of 16 actions in four steps.
Stated in the paper Established
-
GR00T N1 builds on Flow Matching
Trains its action module with flow matching.
Documented in our guide Established
-
GR00T N1 contrasts with π0
The paper contrasts its cross-attention bridge between the two modules with π0's mixture-of-experts bridge, without testing a swap.
Stated in the paper Established
-
GR00T N1 uses Pretrained vision-language backbone
System 2 is a pretrained vision-language model (Eagle-2).
Stated in the paper Established
-
GR00T N1 uses Cross-embodiment pretraining
Per-robot encoders and decoders map each body's numbers into a shared model.
Stated in the paper Established
-
GR00T N1 introduces Data pyramid and generated data
Mixes real robot, simulated, generated and human-video data, giving unlabelled parts latent or inverse-dynamics pseudo-actions.
Stated in the paper Established
-
GR00T N1 uses Action chunking
Produces chunks of 16 motor commands.
Stated in the paper Established
-
GR00T N1 addresses Different robot bodies
Per-robot adapters and a shared model target data that does not add up across robot designs.
Stated in the paper Established
-
GR00T N1 is related to Writing a plan before acting
The System 2 / System 1 split separates understanding from acting; it is not the same as writing a plan in words.
Our interpretation Possible connection
-
AsyncVLA introduces Selective regeneration (asynchronous flow matching)
A confidence rater scores each action of a draft chunk and only the low-scoring ones are regenerated while the others stay fixed as context.
Stated in the paper Established
-
AsyncVLA extends Flow-based action generation
Adds a second selective pass after a standard synchronous flow-matching draft.
Stated in the paper Established
-
AsyncVLA builds on Flow Matching
Its draft stage is standard flow-matching generation; its second stage changes the noise schedule per action.
Documented in our guide Established
-
AsyncVLA contrasts with π0
Compared against π0 and π0.5 in the result tables; the guide notes the comparisons are not like for like.
Stated in the paper Established
-
AsyncVLA addresses One bad action spoils a chunk
Targets a single bad action spoiling a long task, by checking the plan before acting.
Stated in the paper Established
-
AsyncVLA uses Pretrained vision-language backbone
A Qwen2.5-VL-3B backbone.
Stated in the paper Established
-
SmolVLA uses Pretrained vision-language backbone
A small frozen SmolVLM-2, cut to half its language layers.
Stated in the paper Established
-
SmolVLA uses Separate action expert
A flow-matching action expert of about 100 million parameters, with interleaved cross- and causal self-attention.
Stated in the paper Established
-
SmolVLA uses Flow-based action generation
Generates chunks of 50 actions with ten flow-matching steps.
Stated in the paper Established
-
SmolVLA builds on π0
Takes the action-expert idea and sampling of the flow-matching time from π0, and compares against it.
Stated in the paper Established
-
SmolVLA builds on Flow Matching
Trains its expert with the flow-matching objective, citing the Flow Matching paper.
Documented in our guide Established
-
SmolVLA introduces Queue-based asynchronous inference
A robot client and policy server with an action queue and a threshold g; the speed result is clear, the accuracy cost is not established.
Stated in the paper Established
-
SmolVLA introduces Efficient VLA design
Fewer visual tokens, half the layers, a small expert; the paper states cost claims that the guide finds lightly measured.
Stated in the paper Established
-
SmolVLA introduces Community-collected datasets
Pretrains on 481 community datasets of low-cost robots with machine-regenerated task descriptions.
Stated in the paper Established
-
SmolVLA addresses Size, cost and deployability
Aims at a VLA that trains on one GPU and runs on consumer hardware.
Stated in the paper Established
-
SmolVLA addresses Inference latency and idle robots
Queue-based asynchronous inference removes idle time while the model computes.
Stated in the paper Established
-
SmolVLA addresses Scarce and fragmented robot data
Pools many small community datasets.
Stated in the paper Established
-
SmolVLA is related to RT-2
Cites RT-2 in its account of how VLAs began; no technical dependency is stated.
Cited, no stated dependency Established
-
SmolVLA contrasts with π0
Compared head to head: a 0.45-billion-parameter model against a model about seven times larger, run by the authors.
Stated in the paper Established
-
AsyncVLA contrasts with SmolVLA
Both use “asynchronous” for different mechanisms: AsyncVLA regenerates selected actions at different noise levels; SmolVLA overlaps chunk prediction with execution. Conceptual comparison, not a citation.
Our interpretation Possible connection
-
Flow-based action generation contrasts with Discrete action tokens
Continuous chunks from a flow model against integer tokens written one at a time. SmolVLA's related-work section says tokenisation poses limitations for continuous control; no paper in the collection compares the two under identical data.
Stated in the paper Established
-
RT-2 contrasts with π0
Token-writing policy against a flow-matching action expert on a similar vision-language starting point. Different robots, data and tasks.
Our interpretation Possible connection
-
Separate action expert contrasts with Pretrained vision-language backbone
RT-2 fine-tunes all weights of the backbone for actions; π0 and SmolVLA add separate expert weights. The guides find no test that isolates this choice.
Documented in our guide Established
-
Octo introduces Flexible token interface and readout tokens
Designs the sequence of task, observation and readout tokens with block-wise masking so that inputs and heads can be added at finetuning without touching the pretrained weights.
Stated in the paper Established
-
Octo uses Diffusion action head
Decodes chunks of continuous actions with a diffusion head; the paper says the components were known and the combination is its contribution.
Stated in the paper Established
-
Octo uses Action chunking
The head predicts a chunk of consecutive actions; the paper found chunking helpful and temporal ensembling unnecessary.
Stated in the paper Established
-
Octo uses Cross-embodiment pretraining
Pretrains on 25 Open X-Embodiment datasets from many robots, then finetunes to new robots with new encoders or heads.
Stated in the paper Established
-
Octo addresses Scarce and fragmented robot data
Pools many labs' robot data and shows finetuning from about 100 demonstrations.
Stated in the paper Established
-
Octo addresses Different robot bodies
A token interface and new encoders or heads let one pretrained model serve robots with different sensors and action formats.
Stated in the paper Established
-
Octo contrasts with Discrete action tokens
Its ablation scores discretised 256-bin actions at 18% against 83% for the diffusion head on four WidowX tasks (single runs, 40 trials each); the authors describe the discrete head as decisive but imprecise.
Stated in the paper Established
-
Octo contrasts with Pretrained vision-language backbone
Has no vision-language backbone: a frozen text encoder and a shallow image stem feed a transformer trained on robot data only.
Documented in our guide Established
-
Octo is related to RT-2
Compares zero-shot with RT-2-X, a 55-billion-parameter vision-language model finetuned on Open X-Embodiment data; the WidowX numbers are copied from another paper and the values are shown only in a chart.
Stated in the paper Established
-
Flow-based action generation is related to Diffusion action head
Both generate an action chunk by iterative denoising from noise. Octo trains a noise-prediction head (DDPM, 20 steps); π0, GR00T N1, SmolVLA and AsyncVLA train with flow matching. The collection has no like-for-like comparison of the two objectives.
Documented in our guide Established
-
π0 contrasts with Octo
π0 uses Octo (a 93-million-parameter diffusion-based generalist) as a baseline; the guide notes the baselines are not trained like π0.
Stated in the paper Established
-
SmolVLA contrasts with Octo
SmolVLA's LIBERO table lists Octo (0.09B) at 75.1 against 87.3, and its related work describes Octo as trained from scratch on robot data; the guide notes baseline scores for several rows come from other papers.
Stated in the paper Established
-
OpenVLA builds on RT-2
Follows RT-2’s recipe of writing each action number as a token and finetuning a vision-language model, with open weights, and uses RT-2-X as its main baseline; RT-2-X is queried for its second most likely action because the most likely one is often all zeros.
Stated in the paper Established
-
OpenVLA uses Discrete action tokens
Cuts each of seven action numbers into 256 bins between the 1st and 99th percentile and overwrites the 256 least-used Llama tokens; trained with next-token prediction on the action tokens.
Stated in the paper Established
-
OpenVLA uses Pretrained vision-language backbone
Starts from Prismatic-7B (Llama 2 plus two image encoders) and finetunes all of it on robot episodes.
Stated in the paper Established
-
OpenVLA uses Fused image encoders
Uses Prismatic’s joined SigLIP and DINOv2 features and credits the second encoder with better spatial control; its own ablation of the second encoder is 5 points on eight tasks, within chance.
Stated in the paper Established
-
OpenVLA uses Parameter-efficient finetuning and quantised serving
Applies LoRA (rank 32, 1.4% of the weights) and 4-bit quantisation, which it calls the first demonstration for VLAs; the tests use a smaller variant of the model and 33 to 80 rollouts.
Stated in the paper Established
-
OpenVLA uses Cross-embodiment pretraining
Pretrains on 970 thousand episodes from 27 Open X-Embodiment datasets and finetunes to a Franka arm; removing the Open X-Embodiment data costs 30 points on eight Bridge tasks.
Stated in the paper Established
-
OpenVLA addresses Scarce and fragmented robot data
Builds a larger, cleaned training mixture from Open X-Embodiment (it lists 27 datasets, 970 thousand episodes) and shows finetuning from 10 to 150 demonstrations.
Stated in the paper Established
-
OpenVLA addresses Different robot bodies
One model controls two robots out of the box and is finetuned to a Franka arm with a single camera.
Stated in the paper Established
-
OpenVLA addresses Size, cost and deployability
Shows LoRA finetuning on one A100 card and 4-bit serving in 7.0 GB; the “8x” compute saving is a range from about 3 to 12.
Stated in the paper Established
-
OpenVLA addresses Generalising beyond the robot data
Tests visual, motion, physical and semantic generalisation and language grounding on 17 Bridge tasks and out-of-distribution Google robot tasks; RT-2-X remains ahead on the semantic tasks.
Stated in the paper Established
-
OpenVLA builds on Octo
Reuses Octo’s dataset weights for the training mixture and compares with a finetuned Octo; the two are level on the multi-instruction Franka tasks.
Stated in the paper Established
-
OpenVLA contrasts with Octo
Octo composes pretrained text and image pieces with new components trained from scratch; OpenVLA finetunes a whole vision-language model end to end.
Stated in the paper Established
-
OpenVLA contrasts with Action chunking
Predicts one action per model call with no chunk; the authors say chunking and temporal smoothing may help it match Diffusion Policy on dexterous tasks.
Stated in the paper Established
-
OpenVLA contrasts with Diffusion action head
Compared with Diffusion Policy, which generates action chunks by denoising: Diffusion Policy is better on narrow single-instruction tasks, OpenVLA on tasks with several objects and instructions.
Stated in the paper Established
-
π0 contrasts with OpenVLA
π0 uses OpenVLA (7 billion parameters, tokens, no chunk) as a baseline; the π0 guide notes that training data, steps and size differ.
Stated in the paper Established
-
SmolVLA contrasts with OpenVLA
SmolVLA’s LIBERO table lists OpenVLA (7B) at 76.5 against 87.3; the row is copied from the OpenVLA paper’s Table 12.
Stated in the paper Established
-
DROID introduces Diverse in-embodiment data collection
Defines a protocol (shared station, scene change about every 20 minutes, randomly drawn tasks, scene augmentations) and five axes along which a robot dataset can be diverse.
Stated in the paper Established
-
DROID addresses Scarce and fragmented robot data
Releases 76 thousand episodes in 564 scenes with language, calibration and a build guide; the experiments use the first 40 thousand annotated ones.
Stated in the paper Established
-
DROID addresses Generalising beyond the robot data
Out of distribution (distractors, new objects, a camera shift) a co-trained diffusion policy scores 0.53 against 0.17 without co-training; the gap to Open X-Embodiment is not clearly beyond chance.
Stated in the paper Established
-
DROID uses Diffusion action head
Its test policy is a diffusion policy (ResNet-50 encoders, U-Net head, DDIM sampling).
Stated in the paper Established
-
DROID uses Action chunking
The policy predicts 16-step action sequences and executes 8 before replanning.
Stated in the paper Established
-
DROID contrasts with Cross-embodiment pretraining
Single-robot data from many scenes against the pooled multi-robot Open X-Embodiment collection; the tests run on DROID’s own hardware, so scene diversity and embodiment match are confounded.
Documented in our guide Established
-
OpenVLA uses DROID
Adds DROID to its training mixture at a 10% weight, then removes it for the last third of training because the action-token accuracy on it stayed low.
Stated in the paper Established
-
Open X-Embodiment introduces Pooled multi-robot datasets in a common format
Pools 60 datasets in the RLDS format with a coarse shared layout (one canonical camera view, a 7-number end-effector action plus a stop flag).
Stated in the paper Established
-
Open X-Embodiment evaluates Cross-embodiment pretraining
Tests whether co-training on 9 robots’ data improves each robot: RT-1-X beats the original method on 4 of 5 small datasets; all tested robots are in the training data.
Stated in the paper Established
-
Open X-Embodiment uses Discrete action tokens
Both RT-1-X and RT-2-X write actions as 256 bins in eight dimensions, normalised per dataset.
Stated in the paper Established
-
Open X-Embodiment uses Co-fine-tuning on web and robot data
RT-2-X is co-fine-tuned on about equal parts web vision-language data and robotics data; here co-fine-tuning and plain fine-tuning perform similarly (44.4 against 48.7).
Stated in the paper Established
-
Open X-Embodiment uses Pretrained vision-language backbone
RT-2-X is built on the PaLI-X vision-language model; web pretraining is critical (0.0 from scratch against 48.7).
Stated in the paper Established
-
Open X-Embodiment builds on RT-2
Trains RT-2 on the multi-robot mixture as RT-2-X; unseen-object generalisation is on par with RT-2 (61 against 62) and emergent-skill success is 75.8 against 27.3.
Stated in the paper Established
-
Open X-Embodiment addresses Scarce and fragmented robot data
Pools many labs’ datasets so that small datasets can borrow from large ones; the gains are largest on Cable Routing and uneven across domains.
Stated in the paper Established
-
Open X-Embodiment addresses Different robot bodies
Shows positive transfer between known robots without any mechanism for the embodiment gap; says nothing about a new robot.
Stated in the paper Established
-
Octo builds on Open X-Embodiment
Trains on 25 datasets of Open X-Embodiment (800 thousand episodes) and finetunes to new robots, which the Open X-Embodiment paper does not test.
Stated in the paper Established
-
OpenVLA builds on Open X-Embodiment
Trains on 970 thousand episodes of Open X-Embodiment (27 datasets listed) and compares with RT-2-X; removing the Open X-Embodiment data costs 30 points on eight Bridge tasks.
Stated in the paper Established
-
DROID contrasts with Open X-Embodiment
Single-robot data from many scenes against the pooled multi-robot collection: DROID cites 311 scenes and 217 verbs for Open X-Embodiment and uses its Octo-curated subset (about 400 thousand trajectories) as the baseline.
Stated in the paper Established
-
Gemini Robotics introduces Embodied reasoning as a model capability
Defines embodied reasoning for VLMs, releases the ERQA benchmark and trains Gemini Robotics-ER; the report’s ER results are on its own and public benchmarks.
Stated in the paper Established
-
Gemini Robotics uses Pretrained vision-language backbone
Builds the action model on a distilled Gemini Robotics-ER backbone, credited by the authors for the gains; the backbone is never isolated in an experiment.
Stated in the paper Established
-
Gemini Robotics introduces Cloud backbone with a local action decoder
A cloud backbone under 160 ms and a local decoder, 250 ms end to end and 50 Hz effective through action chunks; chunk length and model sizes are not given.
Stated in the paper Established
-
Gemini Robotics uses Action chunking
Action chunks make 50 Hz effective control possible despite a 250 ms pipeline.
Stated in the paper Established
-
Gemini Robotics extends Writing a plan before acting
A reasoning-enhanced variant predicts keypoint trajectories for the next second before acting; its comparison with the vanilla model changes labels and training stage together.
Stated in the paper Established
-
Gemini Robotics uses Cross-embodiment pretraining
Finetunes a model trained on one platform to a two-armed Franka and a humanoid; zero-shot transfer is future work.
Stated in the paper Established
-
Gemini Robotics addresses Generalising beyond the robot data
An 85-task benchmark of visual, instruction and action variations on one robot platform, where baselines collapse on a new language.
Stated in the paper Established
-
Gemini Robotics addresses Smooth continuous actions from a language-model backbone
Dexterous bimanual control with action chunks and specialist finetuning; the hardest tasks are solved by the specialists, not the generalist.
Stated in the paper Established
-
Gemini Robotics addresses Inference latency and idle robots
Distils the backbone and adds a local decoder to reach 250 ms end to end and an effective 50 Hz.
Stated in the paper Established
-
Gemini Robotics contrasts with π0
Uses its own re-implementation of π0, trained on the same data, as the main VLA baseline; the re-implementation beats the public checkpoint in the authors’ tests.
Stated in the paper Established
-
Gemini Robotics is related to GR00T N1
Both split a slow, large model from a fast, small action module; GR00T N1 runs both on the robot and Gemini Robotics hosts the backbone in the cloud.
Documented in our guide Established
-
Gemini Robotics builds on RT-2
Cites RT-2 and OpenVLA as VLAs that struggle to keep abstract reasoning in their actions, which motivates the reasoning variant.
Stated in the paper Established