M4-T3: ORB-SLAM3

Learning objectives

After this lesson, you should be able to:

• List the sensor configurations ORB-SLAM3 supports and what each adds.
• Explain ORB features, binary descriptors and Hamming-distance matching with a ratio test.
• Describe the tracking, local mapping and loop closing threads and the Atlas.
• Run it on a public dataset and read its configuration and outputs.

Conceptual explanation

This lesson describes a published research system and its open-source implementation; behaviour depends on the dataset, configuration and software version, and details here follow the paper and README at the time of writing: consult them for the commit you build.

ORB-SLAM3 combines ORB features (oriented FAST keypoints with rotated BRIEF binary descriptors), a DBoW2 bag-of-words place recognition database, covisibility and essential graphs, three parallel threads (tracking, local mapping, loop closing and map merging) and an Atlas: a set of disconnected maps that are created when tracking is lost and merged when places are recognised. The visual-inertial mode adds IMU preintegration with a maximum-a-posteriori initialisation.

Engineering intuition

ORB features are cheap binary strings: matching is a Hamming distance (count of differing bits), thousands of times faster than floating-point descriptors, and they are rotation-tolerant. The price is sparsity: ORB-SLAM3 maps are sparse point clouds, good for localisation, not for planning obstacle avoidance. And as a feature-based system it needs texture; it can lose track in blur or featureless regions, which is why the Atlas exists: it starts a new map and tries to merge later, instead of dying.

For a UAV the practical question is initialisation and rate: the IMU-aided versions need initial motion, and the whole pipeline must run at the camera rate on your computer (lesson M7-T3). Check your compute budget before choosing the resolution and the number of features.

Mathematical formulation

Matching: the Hamming distance between two 256-bit descriptors and the ratio test that rejects ambiguous matches (Lowe):

Tracking minimises the reprojection error of matched map points over the camera pose (motion-only BA); local BA and global optimisation follow lesson M4-T2, with IMU preintegration factors (lesson M3-T8) and inertial initialisation as a maximum-a-posteriori estimate of scale, gravity direction and biases (Campos et al.). Loop closure: DBoW2 retrieves candidates; a similarity transformation Sim(3) (monocular) or SE(3) is verified by RANSAC and geometric checks; correction by essential-graph optimisation and global BA.

ORB

oriented FAST keypoint with a rotated BRIEF 256-bit descriptor.

$\tau$

ratio-test threshold.

Atlas

set of maps with a merging mechanism.

Architecture and data flow

Run (EuRoC stereo-inertial example, as in the repository README; verify the arguments against the README of your version):

./Examples/Stereo-Inertial/stereo_inertial_euroc ./Vocabulary/ORBvoc.txt ./Examples/Stereo-Inertial/EuRoC.yaml \
    PATH_TO_EuRoC/MH01 ./Examples/Stereo-Inertial/EuRoC_TimeStamps/MH01.txt dataset-MH01_stereoi
# outputs a keyframe trajectory file (for evo) such as f_dataset-MH01_stereoi.txt and kf_dataset-MH01_stereoi.txt

# excerpt of a settings file (key names as in the EuRoC examples; copy a shipped file and edit it)
Camera.type: "PinHole"
Camera1.fx: 458.654        # intrinsics from your calibration, not these example values
Camera.fps: 20
IMU.NoiseGyro: 1.7e-4      # from an Allan analysis of YOUR IMU (lesson M3-T7)
IMU.NoiseAcc: 2.0e-3
IMU.GyroWalk: 1.9393e-5
IMU.AccWalk: 3.e-3
IMU.Frequency: 200.0
ORBextractor.nFeatures: 1200

Real-world UAV applications

• A visual or visual-inertial SLAM baseline for Capstone 1.
• Mapping and relocalisation in feature-rich indoor spaces.
• Comparing against VINS-Fusion and OpenVINS on the same dataset.
• Research starting point for multi-map and multi-session SLAM.

Implementation example

This runs as shown: ORB-style binary descriptor matching: 256-bit descriptors with bit-flip noise are matched by Hamming distance with a ratio test, which rejects ambiguous matches while keeping the correct ones.

import numpy as np
rng = np.random.default_rng(0)
N, bits = 300, 256
A = rng.integers(0, 2, (N, bits), dtype=np.uint8)                   # descriptors in image 1
B = A.copy(); flip = rng.random(B.shape) < 0.10; B[flip] ^= 1       # the same features in image 2 with 10% of the bits flipped
B[200:] = rng.integers(0, 2, (N - 200, bits), dtype=np.uint8)        # 100 features have no counterpart (unrelated descriptors)

def hamming_matrix(X, Y):                                            # (n, m) matrix of Hamming distances
    return (X[:, None, :] != Y[None, :, :]).sum(-1)

D = hamming_matrix(A, B); order = np.argsort(D, axis=1)
best, second = D[np.arange(N), order[:, 0]], D[np.arange(N), order[:, 1]]
accept = best / second < 0.8                                         # ratio test
correct = accept & (order[:, 0] == np.arange(N))
print(f"accepted {accept.sum()} of {N}; correct {correct.sum()}; wrong {np.sum(accept & ~correct)}; true matches kept {correct[:200].sum()} of 200; unmatched features accepted {accept[200:].sum()} of 100")
assert correct[:200].sum() > 190 and np.sum(accept & ~correct) <= 2 and accept[200:].sum() <= 2

Common engineering mistakes

Failure modes

Debugging approach

1. Check feature counts per frame and tracking state in the viewer.
2. Verify calibration and IMU noise against the dataset’s documentation.
3. Evaluate with evo on a dataset with ground truth.
4. Profile per-thread CPU use.

Practical exercises

1. Run the stereo-inertial EuRoC example and evaluate ATE with evo.
2. Change the number of ORB features and record accuracy and processing rate.
3. Disable the IMU on the same data and compare scale and drift.
4. Explain what the Atlas does when tracking is lost.

Interview questions

Write an answer in your own words, then reveal the model answer. Nothing is sent anywhere.

References and further reading

Primary papers, official documentation and standards. Each claim above that is specific to a tool or version points here.

• C. Campos et al., “ORB-SLAM3: an accurate open-source library for visual, visual-inertial and multi-map SLAM”, IEEE T-RO, 2021
• ORB-SLAM3 on arXiv:2007.11898
• ORB-SLAM3 repository and README
• D. Gálvez-López, J. Tardós, “Bags of binary words for fast place recognition in image sequences”, IEEE T-RO, 2012

What’s next