Localization & SLAM · Theory · M4

M4-T1 — Front-End and Back-End SLAM

One half decides what the data mean. The other half decides what they add up to.

Lesson 1 of 8
Module 4: Graph-Based SLAM
Theory
About 7 minutes to read

Nearly every modern SLAM system, from a lidar mapper to a visual-inertial tracker, is built from two parts with a clean interface between them. The front end turns raw sensor data into constraints; the back end finds the trajectory and map that satisfy them best. Understanding the split tells you where to look when a map goes wrong, and why graph optimisation scales where filters do not.

Learning objectives

After this lesson, you should be able to:

Describe the two halves of a SLAM system and list what each is responsible for.
State the interface between them: nodes, edges, measurements and information matrices.
Explain why this split makes failures easier to diagnose and the back end generic.
Estimate the size and sparsity of the optimisation problem for a given trajectory.

The intuition

Think of an accountant and a detective. The detective looks at the evidence (photographs, receipts, witness statements) and decides what each piece says: “this receipt shows a payment of 40 on Monday.” The accountant takes all those statements, however imperfect, and finds the account book that contradicts them least. The detective can make mistakes that the accountant cannot undo; the accountant can only weigh what he is given.

The front end interprets data. The back end reconciles interpretations. A wrong constraint from the front end is invisible to the back end unless it contradicts the others.

The standard architecture. The constraints (relative poses with uncertainties) are the interface; the back end does not need to know which sensor produced them.

The mathematics

What each half owns

Front end and back end compared
	Front end (perception)	Back end (estimation)
Input	Raw sensor data: scans, images, IMU, wheel ticks	Constraints: relative poses with information matrices
Main jobs	Odometry and tracking; scan or feature matching; loop-closure detection; data association; keyframe selection	Build the graph; minimise the total error; return the optimised poses and map; report uncertainty
Depends on	Sensor, environment and appearance	Mathematics only: independent of the sensor
Typical failure	False loop closure; lost tracking; wrong association; degenerate geometry	Bad initial guess; local minimum; ill-conditioned problem; outliers it cannot reject
Rate	Sensor rate (10–100 Hz) for tracking; slower for loop closure	Per keyframe or per loop closure (0.1–10 Hz), or incrementally
Code	Cartographer’s local SLAM; ORB-SLAM3 tracking; LIO-SAM’s feature extraction	Sparse pose-graph optimisation (g2o, Ceres, GTSAM, SPA)

The interface

The back end sees a graph. Each node is an unknown (a robot pose at a keyframe, perhaps a landmark). Each edge is a measurement that relates two nodes: a relative pose z_ij with an information matrix Ω_ij (the inverse of its covariance), saying how much to trust it. Odometry and scan matching give edges between consecutive nodes; a loop closure gives an edge between nodes that are far apart in time.

The back-end objective

\text{minimise}\,F(x)=\sum _{(i,\,j)}\,e_{ij}(x_{i},\,x_{j})^{\top}\,\Omega _{ij}\,e_{ij}(x_{i},\,x_{j})

x_i: the unknown pose of node i.
z_ij, Ω_ij: the measured relative pose from i to j and its information matrix.
e_ij: the error: how far the current estimate of the relative pose is from the measurement (M4-T2).
F: the total weighted squared error: the cost the optimiser minimises.

The back end does not care whether z_ij came from a wheel encoder, ICP, an IMU pre-integration or a camera: that is the point of the split.

Front-end decisions that the back end inherits

Keyframes. Not every scan becomes a node. Choosing nodes by distance or rotation keeps the graph small.
Information matrices. The front end must say how reliable each constraint is. A matching score and the geometry (degeneracy) set Ω.
Loop-closure acceptance. A candidate is added only after verification (M4-T5). The back end can only down-weight what survives.
Initial guess. The odometry chain provides the starting poses; nonlinear optimisation is only as good as its starting point.

A worked example: how big is the problem?

A robot drives a corridor loop and keeps one node every 0.5 m. After 50 m there are 100 nodes, 99 odometry edges and two loop closures.

Unknowns. Each node is a planar pose (x, y, θ): 3 numbers. The state has 300 unknowns.
The system matrix. The optimiser repeatedly solves H Δ = −b with H a 300 × 300 matrix built from 3 × 3 blocks. A block is non-zero only if two nodes share an edge, or for a node with itself.
Count the blocks. 100 diagonal blocks, plus 2 per edge (the matrix is symmetric): 2 × 99 = 198 for odometry and 2 × 2 = 4 for the loop closures. In total 302 non-zero blocks × 9 entries = 2,718 non-zero entries of 90,000: 3%.
Compare with a filter. EKF-SLAM with the same 100 poses in its state would hold a dense 300 × 300 covariance and update all 90,000 entries on every measurement. Graph SLAM stores only the 3% and solves with a sparse Cholesky factorisation.
Scale up. A 10 km route at 0.5 m has 20,000 nodes and 60,000 unknowns: a dense matrix would need 29 GB, the sparse one a few megabytes, and a solve takes milliseconds to a second.

This sparsity is why SLAM moved from filters to graphs: the information is local, and the structure is kept instead of marginalised away.

Engineering implications and limitations

Engineering implications

Debug by layer. If the optimised map is bent, plot the constraints first: is one loop closure wrong (front end) or is the optimiser failing to converge (back end)? Residuals after optimisation point to the first; a large final cost without bad edges, to the second.
Keep the interface clean and logged. Save the graph (nodes, edges, information) to a file. Almost every SLAM bug can be reproduced offline from the graph, without re-running the sensors.
Run them at different rates. Tracking must be real time; the back end can lag. Systems like Cartographer and ORB-SLAM3 run local tracking continuously and global optimisation in a background thread.
Choose the optimiser for the problem. Batch optimisation (g2o, Ceres) for offline refinement; incremental (iSAM2) for online use; robust variants for unreliable front ends (M4-T6).
Information is a design choice. Over-confident information makes the back end distort the map to satisfy one edge; under-confident information lets odometry dominate. Calibrate Ω from data.

Limits

The clean split is an idealisation. In practice the back end’s result improves the front end (better poses make matching easier and a re-linearised map improves later scan-to-map matching), and a good system closes that loop. Tightly coupled systems (visual-inertial odometry, M6) merge the two for accuracy. The split nevertheless remains the right mental model for diagnosis.

Common misconceptions

“The optimiser fixes bad data.”

It finds the best compromise among the constraints. A single confident but wrong constraint can bend the whole map unless it is rejected or down-weighted.

“The back end needs to know the sensor.”

It needs only relative poses and information matrices. That is what lets one optimiser serve laser, camera and radar front ends.

“More nodes are always better.”

More nodes cost computation and add little information between close poses; keyframes keep the graph compact.

“SLAM = optimisation.”

Optimisation is half of it. Most failures in deployed systems are front-end failures: wrong matches, lost tracking, false loops.

Engineering takeaways

A SLAM system splits into a front end (data to constraints) and a back end (constraints to trajectory and map).
The interface is a graph: nodes are poses, edges are relative-pose measurements with information matrices.
The back end is sensor-independent and exploits sparsity: only a few percent of the matrix is non-zero.
Most failures are front-end failures; the back end can only compromise among the constraints it gets.
Save and inspect the graph: it is the best debugging tool in SLAM.

References and further reading

Primary papers and authoritative documentation.

Cadena, C. et al. (2016). Past, Present, and Future of SLAM. IEEE Transactions on Robotics 32(6): the front-end and back-end architecture.
Grisetti, G., Kümmerle, R., Stachniss, C. and Burgard, W. (2010). A tutorial on graph-based SLAM. IEEE Intelligent Transportation Systems Magazine 2(4).
Kümmerle, R., Grisetti, G., Strasdat, H., Konolige, K. and Burgard, W. (2011). g2o: A general framework for graph optimization. IEEE International Conference on Robotics and Automation.
Hess, W., Kohler, D., Rapp, H. and Andor, D. (2016). Real-time loop closure in 2D LIDAR SLAM. IEEE International Conference on Robotics and Automation (Cartographer).
Mur-Artal, R. and Tardós, J. D. (2017). ORB-SLAM2: an open-source SLAM system for monocular, stereo and RGB-D cameras. IEEE Transactions on Robotics 33(5).

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

Question 1

A robot’s optimised map shows one corridor bent into an arc after a loop closure. Which half of the system do you check first, and what would you look for?

Explanation

The front end: look at the loop-closure constraint that was added just before the bend and verify it (is the place really the same? is the relative pose consistent with the odometry within its uncertainty?). A bent map after one closure is the typical signature of a false constraint that the back end tried to satisfy.

Question 2

Why can the same optimiser serve a lidar system and a camera system?

Explanation

It only sees relative poses and their information matrices, not the raw data. Any front end that can produce such constraints can be combined with it; this independence is the purpose of the interface.

Question 3

For 1,000 nodes in a corridor with a handful of loop closures, roughly what fraction of the system matrix entries is non-zero?

Explanation

Almost all non-zero blocks are on the diagonal or next to it (odometry edges): about 3 blocks of 9 entries per node, so about 27,000 entries in a 3000 × 3000 matrix of 9 million, around 0.3%. The sparsity grows with the number of nodes, which is why sparse solvers scale.

What’s next

Next, the data structure at the interface: the pose graph, its nodes and edges, and how to compute the error of an edge.

Practice this lesson

Visualize a Pose Graph

Build a pose graph from odometry and loop-closure measurements and inspect its edges, residuals and information.

Next lesson

M4-T2 — Pose Graphs

Define nodes, relative-pose edges and information matrices and build a pose graph from odometry and loop closures.

Nothing below is required to finish this lesson.

Before
M3-T8: Particle Filters and FastSLAM
Explain Monte Carlo localization, resampling and the Rao-Blackwellised idea behind FastSLAM.
Lab
M4-L1: Visualize a Pose Graph
Build a pose graph from odometry and loop-closure measurements and inspect its edges, residuals and information.

Learning objectives

The intuition

The mathematics

What each half owns

The interface

Front-end decisions that the back end inherits

A worked example: how big is the problem?

Engineering implications and limitations

Engineering implications

Limits

Common misconceptions

“The optimiser fixes bad data.”

“The back end needs to know the sensor.”

“More nodes are always better.”

“SLAM = optimisation.”

Engineering takeaways

References and further reading

Knowledge check

Question 1

Question 2

Question 3

What’s next

Visualize a Pose Graph

M4-T2 — Pose Graphs

Related content

Get notified when the next lesson and new labs are released.