M1.3 — Probability and Basic Statistics

Learning objectives

After this lesson, you should be able to:

• Compute the expectation and variance of a simple distribution.
• Use the 68-95-99.7 rule and explain standard error.
• Apply Bayes’ rule to update a belief from evidence.
• Fuse two noisy measurements by precision weighting.
• Explain sampling from a softmax distribution.
• Explain why squared error corresponds to Gaussian noise and maximum likelihood.

The one-sentence idea

Probability describes uncertain quantities with distributions, lets you combine evidence correctly, and, through maximum likelihood, is where common training losses come from.

Uncertainty is information. Handle it with the rules.

Why probability shows up everywhere

Almost nothing in robot learning is certain. Sensors are noisy (M5.1). Demonstrators do not repeat themselves exactly (M6.2). A language model chooses its next token from a distribution (M4.4), a classifier outputs probabilities (M3.1), and a success rate from ten trials is only an estimate (M6.4). Probability is the language that makes these statements precise. This lesson covers the small set of ideas those uses depend on: random variables, expectation and variance, the Gaussian, conditional probability and Bayes’ rule, and the link between probability and the squared-error loss.

Random variables, expectation and variance

A random variable takes different values with different probabilities, listed by its distribution. Two numbers summarise it. The expectation (mean) is the probability-weighted average of its values, E[X] = Σ p(x)·x. The variance measures spread: the expected squared distance from the mean, Var[X] = E[(X − E[X])²], and its square root is the standard deviation σ. For a fair six-sided die:

import numpy as np

values = np.arange(1, 7)
p = np.full(6, 1 / 6)
mean = float((p * values).sum())
var = float((p * (values - mean) ** 2).sum())
print(round(mean, 3), round(var, 3), round(var ** 0.5, 3))

rolls = np.random.default_rng(0).integers(1, 7, size=100000)       # simulate many rolls
print(round(float(rolls.mean()), 2), round(float(rolls.var()), 2))

3.5 2.917 1.708
3.5 2.91

The exact mean is 3.5 and the variance 2.917. Averaging 100,000 simulated rolls reproduces both (2.91 against the exact 2.917), which is the law of large numbers: sample averages settle on the true expectation as the sample grows. That is why estimating a loss on a batch works at all.

The Gaussian distribution

The Gaussian (or normal) distribution is the bell curve, described by its mean μ and standard deviation σ. It is the default model of noise, because many small independent disturbances add up to something close to Gaussian. A useful rule of thumb: about 68% of values fall within one σ of the mean, 95% within two, and 99.7% within three.

rng = np.random.default_rng(1)
z = rng.normal(loc=0.0, scale=1.0, size=200000)
print([round(float((np.abs(z) < k).mean()), 3) for k in (1, 2, 3)])

[0.684, 0.954, 0.997]

In robotics a joint encoder reading might be modelled as the true angle plus Gaussian noise of 0.01 rad. A quantity computed from n noisy readings has less noise than a single one. Its standard deviation, the standard error, shrinks as σ/√n, which is why averaging helps and why it helps ever more slowly:

for n in (1, 10, 100, 1000):
    means = rng.normal(0, 1, size=(5000, n)).mean(axis=1)           # 5000 experiments, each averaging n readings
    print(n, round(float(means.std()), 3), round(1 / n ** 0.5, 3))

1 0.996 1.0
10 0.31 0.316
100 0.101 0.1
1000 0.032 0.032

The measured spread of the averages (second column) matches the σ/√n prediction (third). To halve the noise you need four times as many readings. The same square-root law is behind the confidence intervals of M6.4.

Conditional probability and Bayes’ rule

The conditional probability P(A | B) is the probability of A once you know B happened. Bayes’ rule turns it around, combining what you believed before (the prior) with new evidence:

A robot’s object detector shows how counter-intuitive this can be. Suppose a particular object is rare in the scene, present only 1% of the time. The detector fires 90% of the time when the object is there, and also 5% of the time when it is not. If it fires, how likely is the object to be there?

prior, tpr, fpr = 0.01, 0.90, 0.05
posterior = tpr * prior / (tpr * prior + fpr * (1 - prior))
print(round(posterior, 3))

rng = np.random.default_rng(2)
present = rng.random(1000000) < prior
fires = np.where(present, rng.random(1000000) < tpr, rng.random(1000000) < fpr)
print(round(float(present[fires].mean()), 3))                      # among the times it fired, how often was the object there?

0.154
0.152

Despite a 90% detection rate, a firing detector means the object is present only about 15% of the time. The object is rare, so the false alarms (5% of the 99% of scenes without it) outnumber the real detections (90% of the 1% with it). The simulation, which just counts, gives 0.152, in agreement with the formula up to sampling noise. Updating a belief by multiplying by the likelihood of the evidence is the idea behind the belief updates of the Physical AI lesson on belief under uncertainty.

The same logic combines two noisy measurements of one quantity. For Gaussian errors the best estimate weights each reading by its precision, 1/σ², so the more reliable one counts for more, and the combined estimate is more certain than either alone:

z1, s1 = 1.0, 0.1                       # first sensor: reading and noise level
z2, s2 = 1.2, 0.2                       # second sensor: less precise
w1, w2 = 1 / s1 ** 2, 1 / s2 ** 2
fused = (w1 * z1 + w2 * z2) / (w1 + w2)
fused_std = (1 / (w1 + w2)) ** 0.5
print(round(fused, 3), round(fused_std, 3))

1.04 0.089

The fused value, 1.04, sits close to the better sensor’s 1.0, and its uncertainty, 0.089, is smaller than either sensor’s 0.1 or 0.2. This is the core of how filters such as the Kalman filter blend a prediction with a measurement.

Probability and learning

Two links to the rest of the course complete the picture. First, a classifier’s softmax output (M3.1) is a probability distribution over classes, and so is a language model’s output over tokens. Generating from them is sampling. Drawing many samples reproduces the probabilities:

logits = np.array([2.0, 1.0, 0.1])
p = np.exp(logits) / np.exp(logits).sum()
samples = np.random.default_rng(3).choice(3, size=10000, p=p)
freq = np.array([(samples == k).mean() for k in range(3)])    # how often each option was drawn
print(p.round(3), freq.round(2))

[0.659 0.242 0.099] [0.66 0.24 0.1 ]

Second, the loss functions of training come from probability. Assume the target is the model’s prediction plus Gaussian noise, and ask which prediction makes the observed data most likely, the maximum likelihood principle. The negative log-likelihood of that Gaussian is, up to constants, the squared error. So squared error is not an arbitrary choice, it corresponds to assuming Gaussian noise, and cross-entropy (M3.5) likewise corresponds to a categorical distribution:

data = np.array([1.0, 2.0, 2.5, 3.5])
sigma = 1.0

def nll(mu):
    return float(np.sum(0.5 * ((data - mu) / sigma) ** 2 + np.log(sigma * np.sqrt(2 * np.pi))))

grid = np.linspace(0, 5, 501)
best = grid[np.argmin([nll(m) for m in grid])]
print(best, data.mean())

2.25 2.25

Searching for the value of μ that makes this data most likely lands on 2.25, the sample mean, which is also where the squared error is smallest. This is why a policy trained with squared error (M6.2) effectively predicts the average of the demonstrated actions, and why, when demonstrations are multimodal, the average can be the wrong answer.

Common misconceptions

Engineering takeaways

1. A random variable has a distribution, summarised by its mean and variance.
2. Sample averages converge to the expectation.
3. About 68%, 95% and 99.7% of a Gaussian lies within 1, 2 and 3 standard deviations.
4. The standard error of an average of n readings is σ/√n.
5. Bayes’ rule: P(A|B) = P(B|A) P(A) / P(B), and rare events dominate false alarms.
6. Precision-weighted fusion gives an estimate more certain than either reading.
7. Softmax outputs are distributions to sample from.
8. Squared error corresponds to Gaussian noise, and cross-entropy to a categorical distribution.

To act under uncertainty, first be precise about what you do and do not know.

Knowledge check

Try each question first, then reveal the explanation.

What’s next

You now have the language of uncertainty. The last lesson of this module covers optimization: how a loss is actually minimised.

Back to the VLA roadmap prerequisites