M9-T3 — Multivariate Gaussian Distributions

Learning objectives

After this lesson, you should be able to:

• Write the multivariate Gaussian density and identify the roles of μ, Σ, the normalising constant and the quadratic form.
• Interpret level sets as ellipses and compute the probability mass inside a k-sigma ellipse.
• State and use the marginal and conditional distributions of a partitioned Gaussian.
• Sample from a Gaussian with the Cholesky factor of Σ.

Introduction and motivation

Almost every probabilistic robotics algorithm (EKF, UKF, pose-graph SLAM, Gaussian processes) represents beliefs as Gaussians. The reason is practical: operations the robot needs, such as predicting through a linear model, combining with a measurement, ignoring variables it does not care about, and conditioning on what it has observed, all keep the Gaussian form, so a belief is just a vector and a matrix. The conditional formula in particular is the Kalman update in disguise.

The intuition

A Gaussian cloud is the unit round cloud pushed through a linear map: stretched along the principal axes by the square roots of the eigenvalues, rotated, and shifted to the mean. Its density falls off with distance from the centre measured in units of the cloud itself (Mahalanobis distance), so points on the same ellipse have equal density. Looking at just some coordinates gives a Gaussian; fixing some coordinates and looking at the rest also gives a Gaussian.

Geometric and physical interpretation

Geometric view. The density is a bell-shaped hill over the plane whose horizontal slices are the ellipses (x − μ)ᵀΣ⁻¹(x − μ) = c; the peak sits above μ. The hill is wide where Σ has large eigenvalues and narrow where it has small ones, and its total volume is 1. Physical view. A robot belief with a long thin ellipse is sure of its position across the corridor and unsure along it, and the density reflects exactly that: probability concentrated in a thin strip.

The mathematics

N(μ, Σ)

the Gaussian with mean μ and covariance Σ (positive definite).

n

the dimension of x.

d² = (x − μ)ᵀΣ⁻¹(x − μ)

the squared Mahalanobis distance.

x = (x₁; x₂)

a partition of the vector, with μ = (μ₁; μ₂) and Σ = [[Σ₁₁, Σ₁₂], [Σ₂₁, Σ₂₂]].

The density

p(x) = (2π)^(−n/2) · det(Σ)^(−1/2) · exp( −½ (x − μ)ᵀ Σ⁻¹ (x − μ) )

What it means. The exponent is −½ times the squared Mahalanobis distance; the prefactor makes the total probability 1. The determinant is the volume scale of the ellipse: wider clouds have lower peaks.

Probability inside an ellipse

P( d² ≤ k² ) = 1 − e^(−k²/2)   (n = 2),        d² ~ χ²ₙ in general

What it means. The squared Mahalanobis distance of a Gaussian vector follows a chi-square distribution with n degrees of freedom. In 2D the 1σ, 2σ and 3σ ellipses contain 39 %, 86 % and 99 % of the probability, not 68, 95, 99.7 as in 1D.

Marginal and conditional

x₁ ~ N(μ₁, Σ₁₁)        x₁ | x₂ ~ N( μ₁ + Σ₁₂Σ₂₂⁻¹ (x₂ − μ₂),  Σ₁₁ − Σ₁₂Σ₂₂⁻¹Σ₂₁ )

What it means. The marginal is the sub-block. The conditional mean moves toward the observed x₂ in proportion to the correlation (the gain Σ₁₂Σ₂₂⁻¹), and the conditional covariance is always smaller than Σ₁₁ (a Schur complement).

Gaussian = exp(−½ Mahalanobis²) normalised. Level sets are ellipses. Marginals take sub-blocks; conditionals shift the mean by Σ₁₂Σ₂₂⁻¹(x₂ − μ₂) and subtract Σ₁₂Σ₂₂⁻¹Σ₂₁ from the covariance.

Step-by-step derivation: normalisation, probability in an ellipse and the conditional

1. Standard case. For μ = 0, Σ = I the density is (2π)^(−n/2) exp(−½‖x‖²), a product of n one-dimensional bells, each integrating to 1.
2. General case by a linear map. Let x = μ + Lz with z ~ N(0, I) and Σ = LLᵀ (Cholesky). The change of variables multiplies densities by 1/|det L| = det(Σ)^(−1/2) and replaces ‖z‖² = ‖L⁻¹(x − μ)‖² by (x − μ)ᵀΣ⁻¹(x − μ), giving the density above and explaining the normalisation factor.
3. Level sets. The density depends on x only through d², so its level sets are the ellipses d² = c, with axes along the eigenvectors of Σ (spectral theorem).
4. Probability in an ellipse. d² = ‖z‖² is the sum of squares of n independent standard normals: chi-square with n degrees of freedom. For n = 2 its density is ½e^(−s/2), so P(d² ≤ k²) = 1 − e^(−k²/2). For 95 %, k² = −2 ln 0.05 = 5.99, k = 2.448.
5. Marginal. Integrating out x₂ leaves x₁; because x₁ = [I 0] x is a linear map of a Gaussian, it is Gaussian with mean μ₁ and covariance [I 0]Σ[I 0]ᵀ = Σ₁₁.
6. Conditional. Write x₁ = μ₁ + Σ₁₂Σ₂₂⁻¹(x₂ − μ₂) + r, where r = x₁ − μ₁ − Σ₁₂Σ₂₂⁻¹(x₂ − μ₂) is the leftover part. Then Cov(r, x₂) = Σ₁₂ − Σ₁₂Σ₂₂⁻¹Σ₂₂ = 0: r is uncorrelated, hence (being jointly Gaussian) independent of x₂. So given x₂, x₁ is that shifted constant plus r, with Cov(r) = Σ₁₁ − 2Σ₁₂Σ₂₂⁻¹Σ₂₁ + Σ₁₂Σ₂₂⁻¹Σ₂₂Σ₂₂⁻¹Σ₂₁ = Σ₁₁ − Σ₁₂Σ₂₂⁻¹Σ₂₁.
7. Link to Kalman. If x₂ is a measurement of x₁ (plus noise), the conditional mean formula is the Kalman update with gain K = Σ₁₂Σ₂₂⁻¹ and the conditional covariance is the posterior covariance P − KSKᵀ (the last lesson of this module).
8. Sampling. Draw z ~ N(0, I) and form x = μ + Lz with L = chol(Σ): the sample has covariance LLᵀ = Σ.

Why Gaussians are everywhere. They are closed under linear maps (Ax ~ N(Aμ, AΣAᵀ), next lesson), sums of independent Gaussians, marginalisation and conditioning. The central limit theorem makes sums of many small independent errors approximately Gaussian, which is why sensor noise often is.

Not Gaussian. Outliers, multi-modal beliefs (symmetric corridors) and angles on a circle are not well modelled by a single Gaussian; robust losses, mixtures and particle filters take over.

A worked example: density, ellipse probabilities and a conditional

Let μ = (0, 0), Σ = [[4, 2], [2, 3]].

• Determinant and peak. det Σ = 12 − 4 = 8. Peak density 1/(2π√8) = 0.0563.
• Density elsewhere. Σ⁻¹ = (1/8)[[3, −2], [−2, 4]]. At x = (2, 0): d² = 2·3·2/8 = 1.5, so p = 0.0563 exp(−0.75) = 0.0266.
• Probabilities. Inside the 1σ, 2σ and 3σ ellipses: 1 − e^(−1/2) = 39.3 %, 1 − e^(−2) = 86.5 %, 1 − e^(−4.5) = 98.9 %. The 95 % ellipse has k = 2.448.
• Marginal. x₁ ~ N(0, 4), x₂ ~ N(0, 3): read the diagonal.
• Conditional. Observe x₂ = 3. The conditional mean of x₁ is Σ₁₂Σ₂₂⁻¹ x₂ = (2/3)·3 = 2; the variance drops from 4 to 4 − 2·(1/3)·2 = 4 − 4/3 = 2.667.
• Sampling. L = chol Σ = [[2, 0], [1, 1.414]] and z = (1, 1) gives x = (2, 2.414); LLᵀ = [[4, 2], [2, 3]] ✓.

Robotics application

SLAM and localisation. The belief of an EKF is exactly N(μ, Σ); conditioning on a LiDAR range changes the pose and every correlated landmark; see Bayesian estimation, probability and uncertainty and EKF-SLAM.

UAV state estimation. Navigation filters carry Gaussian beliefs over position, velocity, attitude error and biases; innovation gating uses the chi-square test above; see Gaussian distributions and covariance, Bayesian estimation and the error-state Kalman filter.

LiDAR registration. Normal-distributions transform (NDT) models each map cell as a Gaussian and scores points by the density; the score is a sum of exp(−d²/2) terms; see scan registration.

Mobile robots and manipulators. Sampling-based motion models (sampling a Gaussian over wheel odometry noise via the Cholesky factor) and sensor simulation both use x = μ + Lz; see robot motion models for navigation.

Engineering implications and limitations

Practical rules

• Work with log-density. The density underflows quickly in high dimensions. Use −½(d² + ln det Σ + n ln 2π) and obtain ln det Σ as 2 Σ ln Lᵢᵢ from the Cholesky factor.
• Never invert Σ explicitly. Solve with the Cholesky factor: d² = ‖L⁻¹(x − μ)‖² by one triangular solve.
• Use the right gate. A 2D 95 % gate is d² ≤ 5.99, not 4 (which is 2σ in 1D and only 86 % in 2D). Use chi-square quantiles for the dimension at hand.
• Check that Σ is positive definite before using it. A singular Σ has no density (use the information form or the pseudo-determinant).
• Sample via Cholesky, not by independent components. Sampling x and y separately ignores correlation.
• Respect the units in conditionals. Σ₁₂Σ₂₂⁻¹ converts units of x₂ into units of x₁; mixed units are fine, but the numbers must be consistent.

Limits

One Gaussian cannot represent two separate hypotheses or heavy tails. Nonlinear transformations of a Gaussian are not Gaussian (the next lesson shows the banana), so linearisation or sigma points are approximations.

Common misconceptions

Engineering takeaways

1. N(μ, Σ) has density (2π)^(−n/2) det(Σ)^(−1/2) exp(−½ d²), with d² the squared Mahalanobis distance.
2. Level sets are ellipses with axes along the eigenvectors of Σ; in 2D, P(d² ≤ k²) = 1 − e^(−k²/2) (39 %, 86 %, 99 % for 1, 2, 3σ; 95 % at k = 2.45).
3. Marginals read sub-blocks; the conditional x₁|x₂ has mean μ₁ + Σ₁₂Σ₂₂⁻¹(x₂ − μ₂) and covariance Σ₁₁ − Σ₁₂Σ₂₂⁻¹Σ₂₁.
4. Sample with x = μ + Lz where Σ = LLᵀ; compute d² with a triangular solve.
5. Gaussians are closed under linear maps, marginalisation and conditioning, which is why filters use them.

References and further reading

Primary papers and authoritative documentation.

• Barfoot, State Estimation for Robotics (author-hosted first edition, free to read): chapter 2.2, Gaussian probability density functions
• Mathematics for Machine Learning: chapter 6.5, Gaussian distribution and its marginals and conditionals
• UCSD ECE276A: Sensing and Estimation in Robotics: Gaussian models, conditioning and the Kalman filter
• Boyd and Vandenberghe, Introduction to Applied Linear Algebra (VMLS): statistics background and least squares; the site did not load when we last checked

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Gaussians survive linear maps: if x ~ N(μ, Σ) and y = Ax then y ~ N(Aμ, AΣAᵀ). The next lesson makes that rule explicit, extends it to nonlinear functions through the Jacobian, and shows where the approximation breaks.

Related content

Nothing below is required to finish this lesson.

• Before
  M9-T2: Covariance Matrices

  Compute and read a covariance matrix, including correlation and the ellipse it describes.