10.1 — NVIDIA GR00T N1 and N1.5

Learning objectives

After this lesson, you should be able to:

• Describe the System 2 and System 1 modules of GR00T N1.
• Compute the age of the latent used by the fast system.
• Explain training with partial labels.
• State what open release does and does not provide.

The one-sentence idea

A slow system that understands, a fast system that moves.

Each runs at its own rate; the price is a slightly old plan.

A foundation model for humanoids

NVIDIA's GR00T N1 is described as an open foundation model for generalist humanoid robots. By its abstract it is a vision-language-action model with two cooperating modules: a vision-language module (“System 2”) that interprets the environment through vision and language instructions, and a diffusion transformer module (“System 1”) that generates fluid motor actions in real time. The two are tightly coupled and trained jointly, end-to-end.^[1] The naming borrows from the psychology of slow deliberate thinking and fast reflexes: System 2 understands, System 1 acts.

The training data are deliberately heterogeneous: real-robot trajectories, human videos and synthetically generated datasets. The authors report results on simulation benchmarks across several robot embodiments and deployment on the Fourier GR-1 humanoid for language-conditioned bimanual manipulation, and the GR00T-N1-2B checkpoint, training data and simulation benchmarks are public.^[1] GR00T N1.5 is a later release; I did not look it up for this course, so check NVIDIA's own release notes for what changed.

You have met each ingredient: a pretrained vision-language model for understanding (lesson 6.3 for π0's version), a diffusion-style action generator (lessons 4.6 and 4.7), and heterogeneous co-training (lesson 5.4). What is new here is the humanoid target and the explicit two-system split.

Why split into two systems

Understanding a scene and an instruction is expensive and does not need to be redone every few milliseconds. Producing smooth motor actions needs to happen frequently. A dual-system design lets each run at its own rate: the slow system produces a compact summary (a latent vector) now and then, and the fast system uses the most recent summary at every step. A time-line simulation with assumed rates (these are not GR00T's published rates):

slow_hz, fast_hz = 8, 100                      # assumed rates for illustration
seconds = 1.0

latent_updates = {round(k / slow_hz, 4) for k in range(int(seconds * slow_hz))}      # times when the slow system publishes a new latent
newest = None
age_ms = []
for step in range(int(seconds * fast_hz)):
    t = step / fast_hz
    due = [u for u in latent_updates if u <= t + 1e-9]
    newest = max(due)                                                               # the most recent latent available now
    age_ms.append((t - newest) * 1000)

print("fast steps per second:", len(age_ms))
print("fast steps per slow update:", fast_hz // slow_hz, "(about)")
print("age of the latent used by the fast system: average", round(sum(age_ms) / len(age_ms), 1), "ms, worst", round(max(age_ms), 1), "ms")

fast steps per second: 100
fast steps per slow update: 12 (about)
age of the latent used by the fast system: average 60.0 ms, worst 120.0 ms

Toy model. Assumed rates and a perfectly regular schedule. Real systems run on a GPU with jitter, and the published rates differ by model.

The fast system acts on a latent that is on average about 60 ms old and at worst 120 ms old. That is the price of the split (compare lesson 8.6): the plan reflects the scene as it was a moment ago, but the actions that carry it out are as fresh as the fast loop. For a tabletop task that is fine; for a moving target, less so.

The rates above are assumed round numbers. The GR00T N1 paper reports its own: the vision-language module at 10 Hz and the action module at 120 Hz, which would make the latent at most about 100 ms old. For a guided tour of the whole paper (architecture, training data, experiments and a critical review), see the GR00T N1 paper guide. GR00T N1 borrows its action-generation recipe from π0, which has its own guide: π0 paper guide.

Training on data without actions

Human videos contain no robot actions, and synthetic data may have different labels from real data. A model can still learn from every source if the loss is computed only where the label exists. A toy of that logic (generic, not GR00T's actual training code):

import numpy as np
rng = np.random.default_rng(0)

sources = ["real robot", "human video", "synthetic"]
has_action_label = {"real robot": True, "human video": False, "synthetic": True}

batch = rng.choice(sources, size=12, p=[0.4, 0.4, 0.2])
action_loss_terms = [s for s in batch if has_action_label[s]]
print("batch composition:", {s: int(np.sum(batch == s)) for s in sources})
print("examples that contribute an action loss:", len(action_loss_terms), "of", len(batch))
print("examples that can contribute only a perception or language loss:", len(batch) - len(action_loss_terms))

batch composition: {'real robot': 4, 'human video': 4, 'synthetic': 4}
examples that contribute an action loss: 8 of 12
examples that can contribute only a perception or language loss: 4

Data without actions can still teach the vision-language side about objects and scenes (the System 2 half) even if it cannot directly teach motor commands; how a particular model uses such data is a design choice described in its paper.

What “open” gives you

Because the checkpoint, training data and simulation benchmarks are public,^[1] you can do what lesson 7.4 describes: fine-tune on your own humanoid's demonstrations, evaluate in simulation (lesson 7.7) and compare against other models on the same tasks (lesson 6.6). What “open” does not give you is a calibrated robot, a safe action adapter or a safety monitor: those remain your job (lessons 9.2 to 9.7), perhaps more so on a humanoid, as the next lessons explain.

Common misconceptions

Engineering takeaways

1. System 2 understands, System 1 acts.
2. Heterogeneous training data.
3. Latent age trades against rate.
4. Open checkpoint, your safety.

Think slowly, move quickly.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

Next: Gemini Robotics, built on Gemini.

Back to the VLA roadmap