11.7 — Current Open Research Problems

Learning objectives

After this lesson, you should be able to:

• List open problems with the evidence for each.
• Audit a result claim for missing information.
• Explain why extrapolated learning curves are uncertain.
• Carry the course’s habits into new reading.

The one-sentence idea

The open problems are the next lessons; the habits are how to read them.

Hold every claim, including confident ones, to the same checks.

What the sources say is unsolved

You have now seen the main ideas of VLA research. This last lesson collects what is not solved, using only statements that earlier lessons cite, so you can tell open problems from opinion. A survey of foundation models in robotics names the scarcity of robot-relevant training data, safety guarantees, uncertainty quantification and real-time execution as the main open problems.^[1] The RT-2 authors note that the model's physical skills remain limited to the distribution of skills seen in the robot data.^[2] A taxonomy study reports that current VLAs struggle with several kinds of semantic generalisation even though they are pre-trained on Internet-scale language data.^[3] The FAST authors say simple per-dimension, per-timestep binning typically performs poorly for dexterous skills from high-frequency data.^[4] SIMPLER starts from the observation that real-world evaluation is not scalable and has reproducibility problems.^[5] Policies trained in simulation can perform worse on real hardware because simulation and reality differ.^[6] And the AI-safety list of concrete problems includes safe exploration and robustness to distributional shift.^[7]

problems = [
    ("Robot data is scarce and costly",          "5.1, 5.3", "130k episodes took 13 robots 17 months"),
    ("Physical skills stay within the data",     "5.2, 5.5", "RT-2 paper: limited to skills seen in robot data"),
    ("Semantic generalisation is weak",          "5.5, 8.3", "taxonomy study of current VLAs"),
    ("Fine, fast motions are hard to tokenise",  "4.2, 4.8", "FAST: binning performs poorly on dexterous data"),
    ("Evaluation does not scale",                "8.1, 8.2", "real-world evaluation is costly and hard to reproduce"),
    ("Sim-to-real gap",                          "7.7",      "policies can do worse on real hardware"),
    ("Safety guarantees and uncertainty",        "9.7, 11.6","survey lists both as open"),
    ("Real-time execution",                      "8.6",      "latency versus control rate"),
]
print(f"{'problem':<44}{'lessons':<11}evidence")
for name, lessons, evidence in problems:
    print(f"{name:<44}{lessons:<11}{evidence}")

problem                                     lessons    evidence
Robot data is scarce and costly             5.1, 5.3   130k episodes took 13 robots 17 months
Physical skills stay within the data        5.2, 5.5   RT-2 paper: limited to skills seen in robot data
Semantic generalisation is weak             5.5, 8.3   taxonomy study of current VLAs
Fine, fast motions are hard to tokenise     4.2, 4.8   FAST: binning performs poorly on dexterous data
Evaluation does not scale                   8.1, 8.2   real-world evaluation is costly and hard to reproduce
Sim-to-real gap                             7.7        policies can do worse on real hardware
Safety guarantees and uncertainty           9.7, 11.6  survey lists both as open
Real-time execution                         8.6        latency versus control rate

Toy model. A summary table assembled from lessons of this course. Each row restates a claim that was cited where it first appeared.

Some of these are engineering problems with known directions (better tokenisers, async inference); some are scientific questions where the field does not yet agree.

A tool: audit any claim you read

The most useful skill the course can leave you with is reading results critically. Here is a small audit that turns the questions of lessons 5.5, 8.1, 8.2 and 8.3 into code. Give it the facts a paper states about a result and it lists what is missing:

def audit(claim):
    missing = []
    if claim.get("trials") is None:              missing.append("number of trials not stated")
    elif claim["trials"] < 50:                   missing.append(f"only {claim['trials']} trials: wide uncertainty")
    if not claim.get("baseline"):                missing.append("no baseline named")
    if not claim.get("success_criterion"):       missing.append("success criterion not defined")
    if claim.get("generalization") and not claim.get("axis"):    missing.append("generalisation claimed without saying which axis (visual, semantic, behavioural)")
    if claim.get("generalization") and claim.get("seen_in_pretraining") is None: missing.append("unclear whether the 'new' items were in the pretraining data")
    return missing

claims = {
    "headline claim": {"trials": 20, "baseline": None, "success_criterion": None, "generalization": True, "axis": None, "seen_in_pretraining": None},
    "careful claim":  {"trials": 400, "baseline": "policy A", "success_criterion": "within 2 cm", "generalization": True, "axis": "semantic", "seen_in_pretraining": False},
}
for name, claim in claims.items():
    problems = audit(claim)
    print(name, "->", "complete" if not problems else "missing:")
    for p in problems: print("   -", p)

headline claim -> missing:
   - only 20 trials: wide uncertainty
   - no baseline named
   - success criterion not defined
   - generalisation claimed without saying which axis (visual, semantic, behavioural)
   - unclear whether the 'new' items were in the pretraining data
careful claim -> complete

Use it as a checklist, not a verdict: a paper may state some of these facts elsewhere, and an incomplete claim is not a false one. But a claim that passes none of them cannot be compared with anything.

A caution about extrapolation

Much of the excitement about VLAs rests on the idea that more data and larger models keep helping. Whether and how far that extrapolates to robot manipulation is itself an open question, and you can see why extrapolation is delicate with a small experiment. Fit a power law to four noisy points of an error-versus-data curve, then ask: how uncertain is the prediction at ten times more data?

import numpy as np
rng = np.random.default_rng(0)

sizes = np.array([100, 200, 400, 800])
true_error = 0.8 * sizes ** -0.3                          # the (unknown) true curve
measured = true_error * np.exp(rng.normal(0, 0.08, 4))    # four noisy measurements

def predict(sample, at):
    slope, intercept = np.polyfit(np.log(sizes), np.log(sample), 1)
    return float(np.exp(intercept + slope * np.log(at)))

predictions = [predict(true_error * np.exp(rng.normal(0, 0.08, 4)), 8000) for _ in range(500)]     # repeat the whole experiment
low, high = np.percentile(predictions, [5, 95])
print("true error at 8000 demonstrations:      ", round(float(0.8 * 8000 ** -0.3), 4))
print("prediction from the four points:        ", round(predict(measured, 8000), 4))
print("5th to 95th percentile over repetitions:", round(float(low), 4), "to", round(float(high), 4))

true error at 8000 demonstrations:       0.054
prediction from the four points:         0.0563
5th to 95th percentile over repetitions: 0.04 to 0.0726

Toy model. A made-up power law with 8% measurement noise. Real learning curves are less regular and the noise is larger.

Even in this tame case, with a perfectly regular curve and only 8% noise, the range of predictions ten times beyond the data runs from about a quarter below the truth to a third above it, and a real curve may bend. Treat any statement of the form “with N times more data, performance will…” as a hypothesis.

Where you are now

You started with what physical AI is (phase 1), built the foundations of transformers, vision-language models and robot learning (phases 2 and 3), saw how actions are represented and generated (phase 4), followed the research that established VLAs and the modern models (phases 5 and 6), built, evaluated and deployed policies (phases 7 to 9) and looked at humanoids and the frontier (phases 10 and 11). The field is moving quickly, so the habits matter more than the model names: write the action definition down, split by episode, test one factor at a time, report trials and baselines, put monitors around the policy, and cite your sources. Every lesson cited what it states, and said where its checking stopped. Hold new claims to the same standard, including ones you read in places that sound confident.

Common misconceptions

Engineering takeaways

1. Data, safety, uncertainty, real time, generalisation and evaluation remain open.
2. Audit claims.
3. Extrapolations are hypotheses.
4. Keep the habits.

Cite, check, and say where your checking stopped.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

That completes the course. Revisit any lesson, change the code and see what happens.

Back to the VLA roadmap