M6-T1 — Eigenvalues and Eigenvectors

Learning objectives

After this lesson, you should be able to:

• Define an eigenvector and eigenvalue and verify a candidate by multiplication.
• Derive Aⁿv = λⁿv and use it to predict long-term behaviour and stability of x_k+1 = A x_k.
• Show that eigenvectors belonging to distinct eigenvalues are independent, and write A = PDP⁻¹ when there is a full set.
• Interpret eigenvalues geometrically (stretch, flip, collapse) and recognise when no real eigenvector exists.

Introduction and motivation

Run a system forward in time and the matrix is applied again and again: a filter's covariance prediction, a discrete controller, a rotating body. What happens in the long run is governed by the directions that the matrix does not turn, because along those the effect of many steps is easy to see. The same directions reveal the principal axes of a covariance ellipse (symmetric case, Module 6) and the natural modes of a mechanical structure. Eigenvectors are the key that unlocks repeated application.

The intuition

Most vectors are both stretched and turned by a matrix. A few special directions are only stretched (or flipped, or squashed): after multiplying, the vector still points along the same line. Such a vector is an eigenvector, and the stretch factor is its eigenvalue: A v = λ v.

Geometric and physical interpretation

Geometric view. The eigenvalue says what the matrix does along its eigenvector: |λ| > 1 stretches, |λ| < 1 shrinks, λ < 0 also flips the direction, and λ = 0 squashes the direction to zero (an eigenvector for λ = 0 is exactly a null-space vector). Some matrices (a rotation by 90°) have no real eigenvector at all: every direction is turned. Physical view. For a discrete linear system, an eigenvector is a mode: a pattern that keeps its shape while its amplitude is multiplied by λ at each step. A mode with |λ| < 1 dies out, with |λ| > 1 grows without bound.

The mathematics

v

an eigenvector: a non-zero vector with A v = λ v.

λ

the corresponding eigenvalue (a real or complex number).

P, D

P has eigenvectors as columns, D is the diagonal matrix of eigenvalues.

Definition

A v = λ v,     v ≠ 0

What it means. Applying A to v gives back a multiple of v: the direction is unchanged and the length is multiplied by λ. Any non-zero multiple of v is also an eigenvector, so only the direction matters.

Powers

A^n v = λ^n v          x(k) = A^k x(0)   with   x(0) = c1 v1 + c2 v2   ⇒   x(k) = c1 λ1^k v1 + c2 λ2^k v2

What it means. Expressed in eigenvectors, each step multiplies the components by their own eigenvalue. In the long run the component with the largest |λ| dominates; the system is stable when every |λ| < 1.

Diagonalisation

A = P D P⁻¹,     P = [v1 v2 ... vn],     D = diag(λ1, ..., λn)          A^n = P D^n P⁻¹

What it means. If A has n independent eigenvectors, then in the basis of eigenvectors A acts as a pure diagonal scaling. Powers of A are then powers of the numbers on the diagonal.

Eigenvectors are directions A only stretches, eigenvalues the stretch factors. Aⁿv = λⁿv, so eigenvalues govern what happens over many steps.

Step-by-step derivation: powers, independence of eigenvectors and diagonalisation

Powers. If A v = λ v then A²v = A(λv) = λ(Av) = λ²v, and by induction Aⁿv = λⁿv.

Eigenvectors of distinct eigenvalues are independent. Take two, A v₁ = λ₁v₁ and A v₂ = λ₂v₂ with λ₁ ≠ λ₂, and suppose c₁v₁ + c₂v₂ = 0.

1. Apply A: c₁λ₁v₁ + c₂λ₂v₂ = 0.
2. Multiply the original relation by λ₂: c₁λ₂v₁ + c₂λ₂v₂ = 0.
3. Subtract: c₁(λ₁ − λ₂)v₁ = 0. Since v₁ ≠ 0 and λ₁ ≠ λ₂, we get c₁ = 0.
4. Then c₂v₂ = 0 gives c₂ = 0 (as v₂ ≠ 0). Only the trivial combination vanishes: the eigenvectors are independent.

The same argument extends to any number of distinct eigenvalues, so an n × n matrix with n distinct eigenvalues has a full set of eigenvectors.

Diagonalisation. Put the eigenvectors in the columns of P. Then AP = [λ₁v₁ … λnvn] = PD by the column view of matrix products. If the columns of P are independent, P is invertible, giving A = PDP⁻¹. Therefore Aⁿ = PDⁿP⁻¹, because the inner P⁻¹P factors cancel.

The zero eigenvalue. A v = 0·v = 0 means v is in the null space; a matrix is singular exactly when 0 is an eigenvalue.

A worked example: a matrix, its modes, and its tenth power

Let A = [[2, 1], [1, 2]].

1. Check a candidate. A (1, 1) = (3, 3) = 3·(1, 1), so (1, 1) is an eigenvector with λ = 3. A (1, −1) = (1, −1) = 1·(1, −1), an eigenvector with λ = 1.
2. Diagonalise. P = [[1, 1], [1, −1]], D = diag(3, 1), and A = P D P⁻¹ with P⁻¹ = ½ [[1, 1], [1, −1]].
3. A tenth power. Write v = (2, 0) = 1·(1, 1) + 1·(1, −1). Then A¹⁰v = 3¹⁰(1, 1) + 1¹⁰(1, −1) = (59049 + 1, 59049 − 1) = (59050, 59048). The component along the eigenvector with λ = 3 has taken over: the result is almost parallel to (1, 1).
4. Stability. The discrete system x(k + 1) = [[0.9, 0.2], [0, 0.5]] x(k) is triangular, so its eigenvalues are the diagonal entries 0.9 and 0.5, both smaller than 1 in magnitude: every state decays to zero. Changing the first entry to 1.1 gives an eigenvalue 1.1 and an unstable system.
5. Trace and determinant. For A: trace = 4 = 3 + 1 and determinant = 3 = 3·1. (This always holds; the next lesson proves it.)
6. No real eigenvectors. The 90° rotation [[0, −1], [1, 0]] turns every direction, so it has no real eigenvector; its eigenvalues are ±i.

Robotics application

Stability of discrete models. A linearised controller or filter step x(k + 1) = A x(k) is stable if and only if all eigenvalues satisfy |λ| < 1. The UAV track examines the stability of a discrete filter in extended Kalman filter design. For continuous-time systems ẋ = A x the condition becomes: real parts of the eigenvalues are negative.

Uncertainty growth. A covariance prediction Σ ← A Σ Aᵀ stretches the uncertainty ellipse along the eigen-directions of A; a dynamics model with an eigenvalue above 1 makes the corresponding uncertainty grow every step unless measurements correct it (Module 9).

Modes of structures. The vibration modes of a flexible robot arm or a camera gimbal are eigenvectors of its mass-stiffness matrix; the eigenvalues give the squared natural frequencies. A controller must avoid exciting them.

Principal axes. For symmetric matrices (covariances, inertia tensors, Hessians) the eigenvectors are mutually perpendicular axes along which the matrix acts as a pure scaling: the content of the next lessons and of PCA.

Engineering implications and limitations

Practical rules

• Never compute eigenvalues from the characteristic polynomial for large matrices. Root-finding on the polynomial is numerically unstable; library routines use the QR algorithm (eig) or, for symmetric matrices, eigh.
• Use eigh for symmetric matrices. It is faster, returns real eigenvalues and perfectly orthogonal eigenvectors, and avoids spurious complex results.
• Eigenvectors are only defined up to sign and scale. Never compare two eigenvectors for equality; compare directions (absolute dot product near 1) or use sign-invariant quantities.
• Non-symmetric matrices can be badly behaved. A matrix with nearly parallel eigenvectors is sensitive: small changes to the matrix change its eigenvalues a lot. Check the condition number of P, not just the eigenvalues.
• Stability from eigenvalues is asymptotic. A system with all |λ| < 1 decays eventually but can grow a lot before it does (non-normal transients); check the actual response.

Limits

Not every matrix has a full set of eigenvectors: [[2, 1], [0, 2]] has the single eigenvector direction (1, 0) (a “defective” matrix). Symmetric matrices always do (next lessons), which is one reason they are so pleasant. For general matrices the SVD, which exists for every matrix, is often the better tool.

Common misconceptions

Engineering takeaways

1. An eigenvector v satisfies A v = λ v: the direction is kept and the length is multiplied by the eigenvalue λ.
2. Aⁿv = λⁿv; expressed in eigenvectors, repeated application multiplies each component by its own λⁿ.
3. Eigenvectors for distinct eigenvalues are independent; a full set gives A = PDP⁻¹ and Aⁿ = PDⁿP⁻¹.
4. A discrete system x(k+1) = Ax(k) is stable when all |λ| < 1; λ = 0 corresponds to the null space.
5. Eigenvectors are defined only up to scale and sign, and general matrices can lack real or complete sets.

References and further reading

Primary papers and authoritative documentation.

• MIT 18.06SC Linear Algebra (OCW): lectures on eigenvalues and eigenvectors, diagonalisation and powers
• 3Blue1Brown, Essence of Linear Algebra: eigenvectors and eigenvalues
• Immersive Linear Algebra: chapter 10, Eigenvalues and Eigenvectors
• Mathematics for Machine Learning: the chapter on matrix decompositions: eigendecomposition

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

You know what eigenvalues are for. The next lesson gives the equation that finds them, including the cases of repeated and complex eigenvalues.

Related content

Nothing below is required to finish this lesson.

• Before
  M5-T8: Numerical Conditioning and Stability

  Use the condition number to predict how much error a problem amplifies, and tell a bad problem from a bad algorithm.