M9-T1 — Introduction to Uncertainty

Learning objectives

After this lesson, you should be able to:

• Describe a noisy measurement with its mean, variance and standard deviation, and compute them from data.
• Distinguish random noise from systematic bias and from model error, and say how each affects an estimate.
• Show that averaging N independent readings shrinks the standard deviation by √N.
• Combine two independent estimates by inverse-variance weighting and compute the fused variance.

Introduction and motivation

A mobile robot has wheel odometry that drifts, a LiDAR that is accurate but sometimes fooled by glass, and an IMU that is smooth but biased. No sensor is right and none is useless. The only sensible way to combine them is to know how uncertain each is and weight them accordingly. That requires numbers for uncertainty, and, since states have several coordinates that interact, matrices of them (the covariance matrix of the next lessons).

The intuition

Measure the same distance five times and you get five slightly different numbers. They scatter around some centre, the mean, with some spread, the standard deviation. A narrow scatter means a precise sensor. When you merge two independent measurements, the better one deserves more weight, and the merged result is always more certain than either alone, because each reading carries some information the other lacks.

Geometric and physical interpretation

Geometric view. A noisy scalar is a point drawn from a bell curve; a noisy position in the plane is a point drawn from a cloud whose shape is an ellipse. Spread along each direction is the uncertainty in that direction. Physical view. Uncertainty has causes: electronic noise and quantisation (random, averages out), calibration error and bias (systematic, does not average out), and wrong models such as an assumption of flat ground (neither). Only the first kind behaves like the statistics below.

The mathematics

x₁, …, x_N

repeated measurements of the same quantity.

μ

the mean (expected value), estimated by x̄ = (1/N) Σ xᵢ.

σ², σ

the variance and standard deviation: the mean squared deviation from μ, and its square root.

w = 1/σ²

the weight (precision) of a measurement.

Mean and variance

x̄ = (1/N) Σ xᵢ        s² = (1/(N − 1)) Σ (xᵢ − x̄)²        σ = √variance

What it means. The mean is the centre of the scatter; the variance measures its squared spread. Dividing by N − 1 (not N) corrects for the fact that the mean was estimated from the same data.

Averaging

Var( x̄ ) = σ²/N        std( x̄ ) = σ/√N

What it means. The standard deviation of the average falls like one over the square root of N: four times the data halves the uncertainty. It does nothing for bias.

Inverse-variance fusion

x̂ = ( w₁x₁ + w₂x₂ ) / ( w₁ + w₂ ),     σ̂² = 1 / ( w₁ + w₂ ),   wᵢ = 1/σᵢ²

What it means. Weights are the precisions. The fused precision is the sum of the precisions, so the fused variance is smaller than either. This is the scalar Kalman update.

Describe quantities by mean and variance. Average to cut noise by √N. Fuse independent estimates with weights 1/σ²: precisions add.

Step-by-step derivation: variance of an average and the optimal fusion weights

1. Variance of an average. For independent readings with equal variance, Var(Σxᵢ) = Σ Var(xᵢ) = Nσ², so Var(x̄) = Var(Σxᵢ)/N² = σ²/N and std(x̄) = σ/√N.
2. Fusion as a weighted average. Let x̂ = a x₁ + (1 − a) x₂ with 0 ≤ a ≤ 1; if both are unbiased, so is x̂ (the weights add to 1).
3. Its variance. By independence Var(x̂) = a²σ₁² + (1 − a)²σ₂².
4. Minimise over a. Differentiate: 2aσ₁² − 2(1 − a)σ₂² = 0, so a = σ₂²/(σ₁² + σ₂²) = (1/σ₁²)/(1/σ₁² + 1/σ₂²): the weight is proportional to the precision.
5. Fused variance. Substituting, Var(x̂) = σ₁²σ₂²/(σ₁² + σ₂²), equivalently 1/Var(x̂) = 1/σ₁² + 1/σ₂²: precisions add. The result is smaller than the smaller of σ₁², σ₂².
6. Same as a gain form. Write K = σ₁²/(σ₁² + σ₂²); then x̂ = x₁ + K(x₂ − x₁) and σ̂² = (1 − K)σ₁². This is exactly the Kalman update in one dimension: the prior x₁ is corrected by the measurement x₂ with gain K.
7. What averaging cannot fix. If every reading has the same bias b, E[x̄] = μ + b however large N is. Calibration, redundancy with independent sensors, or a model of the bias is needed.

Units and scale. The standard deviation has the units of the measurement; the variance has squared units. Quote sensor noise as σ (a range sensor with σ = 2 cm), and compute with variances when adding independent contributions: variances add, standard deviations do not.

From scalars to vectors. A state with several components (x, y, heading) has a variance for each component and covariances between them, collected into the covariance matrix, which the next lesson introduces. All of the above (averaging, weighting by precision, K) generalises with matrices in place of numbers.

A worked example: five readings and two sensors

• Five range readings in metres: 4.02, 3.98, 4.05, 3.95, 4.00. The mean is x̄ = 4.000.
• Variance. Deviations 0.02, −0.02, 0.05, −0.05, 0: squares sum to 0.0004 + 0.0004 + 0.0025 + 0.0025 + 0 = 0.0058, so s² = 0.0058/4 = 0.00145, s = 0.0381 m.
• Uncertainty of the mean. s/√5 = 0.0170 m: the average is about 2.2 times better than a single reading.
• Two sensors. Sensor 1 reads 4.00 with σ₁ = 0.05; sensor 2 reads 4.06 with σ₂ = 0.10. Weights w₁ = 400, w₂ = 100.
• Fusion. x̂ = (400·4.00 + 100·4.06)/500 = 4.012, σ̂² = 1/500 = 0.002, σ̂ = 0.0447: better than the 0.05 of the best sensor. The fused value sits close to the more precise reading (0.012 from it, 0.048 from the other).
• Gain form. K = 0.05²/(0.05² + 0.10²) = 0.2, x̂ = 4.00 + 0.2·0.06 = 4.012 ✓, σ̂² = 0.8·0.0025 = 0.002 ✓.
• Bias. If sensor 1 has a hidden 3 cm bias, every one of its readings is 3 cm high; averaging 100 of them still leaves 3 cm of error.

Robotics application

Mobile robots. Odometry, LiDAR matching and a compass each have a variance; the localisation filter weights them by precision; see probability and uncertainty in SLAM and Bayesian estimation.

UAVs. GNSS (noise of metres, no drift), barometer (smooth, drifts), IMU (very precise short term, integrates bias); the filter fuses them with their variances; see Gaussian distributions and covariance, IMU, barometer, magnetometer and GNSS and GNSS fundamentals.

LiDAR and point clouds. Range noise typically grows with distance; weighting residuals by 1/σ² in registration is the weighted least squares of Module 5; see LiDAR point clouds for SLAM.

Manipulators. Encoder noise at the joints propagates to hand-position uncertainty through the Jacobian (covariance propagation lesson).

Engineering implications and limitations

Practical rules

• Characterise each sensor. Record the sensor at rest and compute mean and standard deviation (and the Allan variance for IMUs); use the datasheet as a starting point, not a final answer.
• Use variances when adding. Independent errors add in variance, not in standard deviation: two 3 cm errors combine to 4.2 cm, not 6 cm.
• Check independence. Averaging and inverse-variance fusion assume independent errors. Repeated readings with the same bias, or two sensors sharing a clock, violate this and make the fused estimate over-confident.
• Do not hide bias in noise. If a sensor is consistently off, calibrate it or estimate the bias as a state; do not inflate σ and hope.
• Prefer conservative variances. An over-confident filter ignores good measurements; a slightly pessimistic one recovers. Tune with measured residuals (the innovations of the Kalman filter lesson).
• Report σ with units and with the confidence it implies. “1σ = 2 cm” covers 68 % of the samples for a Gaussian; 95 % needs about 2σ.

Limits

Mean and variance describe a distribution fully only if it is Gaussian. Multimodal ambiguity (a corridor that looks the same in two places), heavy-tailed outliers and strongly nonlinear sensors need richer tools (particle filters, robust losses); see robust optimisation.

Common misconceptions

Engineering takeaways

1. Every measurement is a random variable with a mean and a variance; the standard deviation has the measurement's units.
2. Averaging N independent readings reduces the standard deviation by √N but leaves bias untouched.
3. Inverse-variance fusion x̂ = (w₁x₁ + w₂x₂)/(w₁ + w₂) with w = 1/σ² minimises variance; precisions add.
4. The gain form x̂ = x₁ + K(x₂ − x₁), K = σ₁²/(σ₁² + σ₂²), is the one-dimensional Kalman update.
5. Characterise sensors, keep errors independent, and never hide bias in the noise.

References and further reading

Primary papers and authoritative documentation.

• Barfoot, State Estimation for Robotics (author-hosted first edition, free to read): chapter 2, primer on probability theory
• UCSD ECE276A: Sensing and Estimation in Robotics: probability and Gaussian models for estimation
• Mathematics for Machine Learning: chapter 6, probability and distributions
• Boyd and Vandenberghe, Introduction to Applied Linear Algebra (VMLS): statistics background in the least-squares chapters; the site did not load when we last checked

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

A robot state has several coordinates, and their errors are usually related: a heading error tilts the whole position estimate. The covariance matrix is the table of all variances and correlations at once.

Related content

Nothing below is required to finish this lesson.

• Before
  M8-T10: Redundant Manipulators

  Analyse an arm with more joints than task dimensions and choose among its infinitely many solutions.