M4-T5 — Loop Closure

Learning objectives

After this lesson, you should be able to:

• Describe the stages of loop-closure detection: retrieval, gating, verification and consistency checking.
• Explain why place recognition alone is not enough and what geometric verification adds.
• Compute precision and recall for a set of candidate loop closures and say which matters more.
• Estimate the effect of one false loop closure on a trajectory.

The problem and the intuition

You walk into a hotel corridor and think “I have been here before.” Is it because it looks the same, or because it is the same? In a hotel every corridor looks the same, so looking the same proves nothing. You would check: is this about where I would be if I had walked straight back (consistent with my path)? Does the pattern of doors match door by door (geometry)? Only if both say yes do you act. A robot does the same, in stages.

Appearance retrieves candidates; geometry and consistency decide.

The algorithm: detect, gate, verify, check

Problem. Given the current keyframe and all earlier ones, decide whether the robot is at (nearly) the same place as an earlier keyframe, and if so measure the relative pose between them.

Assumptions. The place looks similar enough on revisit (appearance or structure) to be recognised; the environment is mostly static; the odometry drift is not so large that the true candidate is out of the search region.

The stages

1. Retrieve candidates (place recognition). Describe each keyframe by a compact descriptor and look up the most similar earlier ones. Lidar: Scan Context, histograms, learned descriptors. Camera: bag of visual words (FAB-MAP, DBoW2), learned global descriptors (NetVLAD). Geometry only: submap matching within an uncertainty radius (Cartographer).
2. Gate. Reject candidates that are recent in time (the immediate neighbours are always similar: require, say, at least 30 s or 50 keyframes of separation) and those that are far from the current position estimate when odometry is reliable.
3. Verify geometrically. Match the two scans or feature sets (ICP, correlative matching, RANSAC with a minimal solver). Accept only if the fit is good: high inlier ratio, low residual, and a well-conditioned geometry. This also gives the relative pose z_ij and its covariance.
4. Check consistency. Compare z_ij with what the current trajectory predicts: the difference should be explainable by the accumulated drift (a Mahalanobis test with a chi-square threshold, using the drift covariance), and a group of closures should agree with each other (pairwise consistency, M4-T6).

Judging a loop-closure detector

TP

true positives: accepted closures between genuinely the same place.

FP

false positives: accepted closures between different places (perceptual aliasing).

FN

false negatives: real revisits that the system failed to detect.

SLAM needs precision close to 100%: operating points are chosen at the highest recall that still gives no false positives (Lowry et al., 2016). A recall of 50% is acceptable: the next revisit provides another chance, while one false closure persists in the optimised map.

Why false positives are so costly

A loop-closure edge has high information, and the optimiser tries to honour it. If it connects two places that are D metres apart in the true world, the chain of N poses between them must absorb the contradiction: roughly D/N metres of extra displacement on each odometry edge, rotating and bending the whole segment. For D = 3 m and 100 poses in between, every step is distorted by 3 cm in a consistent direction, producing a visibly curved corridor and wrong poses everywhere in between. All later scans placed with those poses blur the map.

Accept a loop closure only if the evidence would still hold after you removed the descriptor: geometry and consistency, not looks.

A worked example: five candidates

A robot’s retrieval step returns five candidates for the current keyframe. After the time gate, each gets a geometric verification (inlier ratio of the scan match) and a consistency test: the difference between the matched relative pose and the pose predicted by the trajectory, as a squared Mahalanobis distance d² against a drift covariance (3 degrees of freedom, 99% threshold χ² = 11.34).

1. Descriptor similarity alone would have accepted B and C above a threshold of 0.85. Both are false positives: the same layout in a different corridor and on a different floor. Both look right to a descriptor.
2. Verification removes B (only 35% of the scan points agree); consistency removes C (the match says the robot is 6 m from where odometry puts it, with a drift covariance that allows 1 m: d² = 38.7).
3. Result. TP = 2 (A, D), FP = 0, FN = 1 (E): precision 2/2 = 100%, recall 2/3 = 67%. A system that accepted all similarities above 0.85 would have TP = 2, FP = 2: precision 50%.
4. The cost of C. Suppose C had been accepted. It claims that the robot is 6 m from where the trajectory says. With 120 poses between, the optimiser puts 6/120 = 5 cm of displacement on every odometry edge, bending the entire 60 m run; the resulting map is unusable.
5. The cost of E. E is missed: the drift of about 0.4 m stays in the map until the next detection. An annoyance, not a disaster.

Engineering implications and limitations

Engineering implications

• Tune for precision first. Pick thresholds on a validation set at 100% precision; accept the recall that results. Add recall later with better descriptors and by retrying.
• Verify with a different kind of evidence than the one used for retrieval: appearance to retrieve, geometry to verify. Independent errors multiply the protection.
• Require temporal consistency: accept a closure only if several consecutive keyframes match the same earlier region, or if a group of closures is mutually consistent.
• Gate by uncertainty. Search for candidates where the odometry-based covariance allows them. That is cheap and also rejects aliasing across the building.
• Prefer robust back ends as a second line of defence (M4-T6): switchable constraints and robust kernels can reduce the damage of a false closure that slipped through.
• Log closures. Record every accepted and rejected candidate with its scores; they are the dataset for tuning and the evidence when a map goes wrong.

Limits

Perceptual aliasing (hospital corridors, warehouse aisles, orchards) defeats appearance and sometimes geometry; only a strong consistency test saves you. Dynamic and seasonal change, lighting, weather and viewpoint change break recognition across long periods, which is why long-term and lifelong SLAM (Module 8) is hard. Learned descriptors improve recall but not the fundamental need to verify.

Common misconceptions

Engineering takeaways

1. Loop closure runs in stages: retrieve candidates, gate, verify geometrically, check consistency with the trajectory.
2. Precision must be near 100%; recall can be modest. A false closure costs far more than a missed one.
3. Appearance finds candidates, geometry and consistency decide.
4. One false closure of D metres spreads about D/N of displacement over each of the N edges between the two places.
5. Log every candidate with its scores; use robust back ends as the second line of defence.

References and further reading

Established fundamentals. Learned place recognition and lifelong loop closure are active research (Module 8).

• Cummins, M. and Newman, P. (2008). FAB-MAP: Probabilistic localization and mapping in the space of appearance. International Journal of Robotics Research 27(6).
• Gálvez-López, D. and Tardós, J. D. (2012). Bags of binary words for fast place recognition in image sequences. IEEE Transactions on Robotics 28(5).
• Kim, G. and Kim, A. (2018). Scan Context: Egocentric spatial descriptor for place recognition within 3D point cloud map. IEEE/RSJ IROS.
• Lowry, S. et al. (2016). Visual place recognition: a survey. IEEE Transactions on Robotics 32(1).
• Hess, W., Kohler, D., Rapp, H. and Andor, D. (2016). Real-time loop closure in 2D LIDAR SLAM. IEEE International Conference on Robotics and Automation.
• Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T. and Sivic, J. (2016). NetVLAD: CNN architecture for weakly supervised place recognition. IEEE CVPR.

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Some false closures will get through. The next lesson shows how to make the back end robust to them: robust costs, switchable constraints and consistency checks.

Related content

Nothing below is required to finish this lesson.

• Before
  M4-T4: Graph Optimization

  Apply Gauss-Newton to SE(2) pose graphs, with the sparse structure, gauge freedom and an anchor.

• Lab
  M4-L3: Diagnose a Broken Loop Closure

  One loop closure is wrong and bends the map: find it with residual tests and make the optimisation robust.