M6-T7 — The Moore-Penrose Pseudoinverse

Learning objectives

After this lesson, you should be able to:

• Build the pseudoinverse A⁺ = VΣ⁺Uᵀ from the SVD and state the four Penrose conditions.
• Show that x = A⁺b is the minimum-norm least-squares solution.
• Use the closed forms (AᵀA)⁻¹Aᵀ and Aᵀ(AAᵀ)⁻¹ for full-rank tall and wide matrices.
• Explain why small singular values must be truncated or damped, and connect this to Jacobian-based control.

Introduction and motivation

A 7-joint arm reaching a 3D point has infinitely many joint solutions; a velocity command for its hand can be met by infinitely many joint-velocity vectors. Which one should the controller pick? The shortest one is smooth and energy-efficient, and it is exactly q̇ = J⁺ ẋ. In the opposite situation, a calibration with more measurements than unknowns, there is no exact solution and we want the closest. One object, the pseudoinverse, answers both.

The intuition

An ordinary inverse undoes the matrix completely, which is only possible if nothing was squashed. The matrix A maps its row space onto its column space, one stretch σᵢ at a time, and annihilates the null space. The pseudoinverse undoes exactly that part: it divides by each non-zero σᵢ and throws away the rest. Anything b has outside the column space cannot be reached, so it is dropped (that is the least-squares projection); anything that x could add in the null space is not needed, so it is set to zero (that is the minimum norm).

Geometric and physical interpretation

Geometric view. A⁺b is built in two projections. First project b onto the column space of A (the least-squares step), obtaining the nearest reachable output. Then, of all x that map there (a whole affine subspace, the particular solution plus the null space), pick the one perpendicular to the null space, which is the shortest. Physical view. For a redundant arm, the shortest joint velocity does the task with the least motion; the leftover freedom (null space) is kept for secondary goals in Module 8.

The mathematics

A⁺

the pseudoinverse (Moore-Penrose inverse) of A, size n × m for an m × n matrix.

Σ⁺

the transpose-shaped diagonal matrix with entries 1/σᵢ for σᵢ > τ and 0 otherwise.

x⁺

the pseudoinverse solution x⁺ = A⁺ b.

Definition by SVD

A = U Σ Vᵀ   ⇒   A⁺ = V Σ⁺ Uᵀ = Σᵢ (1/σᵢ) vᵢ uᵢᵀ   (sum over σᵢ > τ)

What it means. Replace each non-zero singular value by its reciprocal, and zero stays zero. For an invertible square matrix this is exactly A⁻¹.

Closed forms for full rank

tall, full column rank:  A⁺ = (AᵀA)⁻¹ Aᵀ        wide, full row rank:  A⁺ = Aᵀ (AAᵀ)⁻¹

What it means. The tall form is the normal-equation solution of least squares (Module 5). The wide form picks the solution lying in the row space, which is the minimum-norm one. Use the SVD in code, not these formulas, for accuracy.

What it solves

x⁺ = A⁺ b minimises ‖A x − b‖, and among all minimisers has the smallest ‖x‖

What it means. It combines the two cases: when no exact solution exists it minimises the residual, and when many solutions exist it picks the shortest.

A⁺ = VΣ⁺Uᵀ. It always exists and is unique, and x = A⁺b is the shortest vector among all least-squares solutions.

Step-by-step derivation: minimum-norm least squares from the SVD

Let A = UΣVᵀ have rank r, and change variables y = Vᵀx, c = Uᵀb. Orthogonal matrices preserve lengths, so

1. ‖Ax − b‖² = ‖UΣVᵀx − b‖² = ‖Σy − c‖².
2. Since Σ is diagonal, this equals Σᵢ≤ᵣ (σᵢyᵢ − cᵢ)² + Σᵢ>ᵣ cᵢ²: the first sum can be made zero by yᵢ = cᵢ/σᵢ; the second does not depend on x (it is the unavoidable residual outside the column space).
3. The components yᵢ for i > r are free (they multiply zero) and do not change the residual. They correspond to the null space.
4. Among all minimisers the length is ‖x‖ = ‖y‖, smallest when the free components are zero: yᵢ = 0 for i > r.
5. Return to x: x = Vy = V Σ⁺ c = V Σ⁺ Uᵀ b = A⁺ b.

The Penrose conditions. The matrix A⁺ is the unique matrix satisfying (1) A A⁺ A = A, (2) A⁺ A A⁺ = A⁺, (3) (AA⁺)ᵀ = AA⁺ and (4) (A⁺A)ᵀ = A⁺A. Using the SVD, AA⁺ = U (ΣΣ⁺) Uᵀ is the orthogonal projector onto the column space and A⁺A = V (Σ⁺Σ) Vᵀ the projector onto the row space; both are symmetric, and (1) and (2) follow because ΣΣ⁺Σ = Σ.

Full-rank forms. For tall full-column-rank A, AᵀA = VΣᵀΣVᵀ is invertible and (AᵀA)⁻¹Aᵀ = VΣ⁻²VᵀVΣUᵀ = VΣ⁻¹Uᵀ = A⁺. For wide full-row-rank A, Aᵀ(AAᵀ)⁻¹ reduces to the same expression.

Damping and truncation. The map b ↦ A⁺b amplifies noise along uᵢ by 1/σᵢ, unbounded as σᵢ → 0. The cure is either truncation (drop σᵢ < τ, as in Σ⁺) or damping, replacing 1/σᵢ by σᵢ/(σᵢ² + λ²), the same factor as ridge regression, which is smooth and gives the damped least-squares method of Module 8.

A worked example: three kinds of system, one formula

• Rank-deficient square. A = [[1, 1], [1, 1]], b = (2, 2). The singular values are 2 and 0, v₁ = (1, 1)/√2, u₁ = (1, 1)/√2, so A⁺ = (1/2) v₁u₁ᵀ = (1/4)[[1, 1], [1, 1]] and x⁺ = A⁺b = (1, 1). All solutions of x₁ + x₂ = 2 work; (1, 1) has the smallest length √2.
• Wide (underdetermined). A = [[1, 1, 1], [1, −1, 0]], b = (3, 1). Rows are independent, so A⁺ = Aᵀ(AAᵀ)⁻¹ with AAᵀ = [[3, 0], [0, 2]]; (AAᵀ)⁻¹b = (1, 0.5) and x⁺ = Aᵀ(1, 0.5) = (1.5, 0.5, 1). Check: 1.5 + 0.5 + 1 = 3 ✓, 1.5 − 0.5 = 1 ✓. The null space is spanned by (1, 1, −2) (since both rows are perpendicular to it), and x⁺ is perpendicular to it: 1.5 + 0.5 − 2 = 0 ✓ so no shorter solution exists.
• Tall (overdetermined). Fit a line y = c₀ + c₁t to points (0, 0), (1, 1), (2, 1): X = [[1, 0], [1, 1], [1, 2]], y = (0, 1, 1). Then XᵀX = [[3, 3], [3, 5]], Xᵀy = (2, 3), giving c = (1/6, 1/2): X⁺y = (0.1667, 0.5), the least-squares line.
• Zero singular value left alone. The pseudoinverse of diag(2, 0) is diag(0.5, 0), not an error: it never divides by zero.

Robotics application

Manipulators. q̇ = J⁺ ẋ gives the minimum joint-speed solution for a redundant arm, and (I − J⁺J) projects onto the joint motions that leave the hand still (Module 8, null-space control). See also frames and footprints for the mobile-base analogue.

SLAM and calibration. When the information matrix is rank-deficient because of gauge freedoms (the whole map can slide or rotate), the pseudoinverse gives the minimum-norm update, a practical step used in pose-graph solvers; see pose graphs and nonlinear least squares.

UAV sensor fusion. Allocating thrust and torque among more than four rotors is an underdetermined problem; the pseudoinverse of the allocation matrix gives the minimum-effort motor command. IMU and GNSS calibration with unexcited parameters is handled by truncating the spectrum; see IMU preintegration.

LiDAR registration. A degenerate scan (featureless corridor) has a rank-deficient normal matrix; the pseudoinverse leaves the unconstrained direction at zero update instead of letting noise decide it.

Engineering implications and limitations

Practical rules

• Use numpy.linalg.pinv or lstsq, not inv(AᵀA). Both use the SVD and handle rank deficiency; the explicit closed form squares the condition number.
• Set the cutoff rcond deliberately. pinv(A, rcond) treats σ below rcond·σ₁ as zero; the default is tiny. Set it from the sensor noise level so noise is not amplified.
• The pseudoinverse is not continuous. A tiny change that turns a zero singular value into 1e-12 makes 1/σ explode. Damping (σ/(σ² + λ²)) is the smooth alternative.
• Does not equal the inverse of a nearby matrix. Never assume A⁺ ≈ (A + δ)⁻¹; they can differ wildly near rank changes.
• It solves for x only for the metric you used. The “shortest” solution depends on the units and weighting of x. Scale variables (or use a weighted pseudoinverse) so that a metre and a radian are comparable.
• Check the residual and the norm. Report both ‖Ax − b‖ (how well it fits) and ‖x‖ (how large); a huge ‖x‖ signals amplified noise.

Limits

The pseudoinverse solves a linear problem. For a nonlinear problem it is applied to the local Jacobian, repeatedly, and near singularities its jumps cause large joint velocities, which is why damped variants are used in practice.

Common misconceptions

Engineering takeaways

1. A⁺ = VΣ⁺Uᵀ: invert the non-zero singular values, leave zeros at zero.
2. x = A⁺b is the least-squares solution of smallest norm, uniting the over- and underdetermined cases.
3. AA⁺ projects onto the column space and A⁺A onto the row space; the four Penrose conditions define it uniquely.
4. For full-rank tall or wide matrices it equals (AᵀA)⁻¹Aᵀ or Aᵀ(AAᵀ)⁻¹, but compute it with the SVD.
5. Small singular values must be truncated or damped, which is the basis of robust Jacobian-based control.

References and further reading

Primary papers and authoritative documentation.

• MIT 18.06SC Linear Algebra (OCW): lecture on the pseudoinverse and least squares
• Boyd and Vandenberghe, Introduction to Applied Linear Algebra (VMLS): the pseudo-inverse and least-norm solutions; the site did not load when we last checked
• Modern Robotics (Lynch and Park), free preprint: the use of the pseudoinverse in velocity kinematics
• Mathematics for Machine Learning: the Moore-Penrose pseudoinverse

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

The last lesson of this module uses the SVD of a centred data matrix to find the directions in which data varies most: principal component analysis, the tool behind surface-normal estimation and many dimensionality-reduction steps in perception.

Related content

Nothing below is required to finish this lesson.

• Before
  M6-T6: Rank and Condition Number

  Read the numerical rank and the condition number from the singular values.

• Lab
  M6-L1: Point Cloud PCA

  Find the surface normal and principal axes of a point cloud with the SVD, and tell planes, edges and volumes apart.