P4-T5 — Geometric Consistency

Learning objectives

After this lesson, you should be able to:

• List the common geometric models (similarity, affine, homography, epipolar geometry) with their minimal sample sizes and when each applies.
• Apply a homography and compute the transfer error of a match.
• State the epipolar constraint, compute the distance from a match to its epipolar line, and say what the constraint cannot detect.
• Recognise degenerate configurations and use motion priors from odometry or an IMU.

The idea: two views of one world are not independent

If you know how the camera moved, or how a plane appears in two images, you can predict where a point seen in the first image has to appear in the second. A match that is far from the prediction is not a different view of the same point: it is a mistake. How strongly a model constrains matches depends on how much of the second image it allows:

The more a model constrains where a match can be, the more wrong matches it rejects, and the more it needs the scene to obey its assumptions.

The mathematics: homography and epipolar constraints

Homography

For points on a plane (or for a rotating camera), corresponding image points are related by a 3 × 3 matrix H in homogeneous coordinates:

The transfer error of a match (p, q) is the distance between q and the prediction from p. H has 8 degrees of freedom (it is defined up to scale), so four point matches (each gives two equations) determine it.

Epipolar geometry

For two views of a rigid scene, with x = (u, v, 1)ᵀ a point in the first image and x′ its match in the second,

F

the fundamental matrix: 3 × 3 with rank 2, determined by the relative camera pose and both cameras’ intrinsics. It maps a point x to the line l′ = F x in the second image on which its match must lie.

E

the essential matrix: the same for calibrated cameras, E = [t]× R, built from the relative rotation R and translation t. It has 5 degrees of freedom (the translation scale is unobservable).

K, K′

the intrinsic matrices of the two views (the same matrix for a single moving camera).

The distance from the match x′ to the epipolar line l′ = (a, b, c) = F x is

This is the usual test (Sampson distance is a refinement that takes both images into account). For a stereo pair that has been rectified, so that the cameras are side by side and aligned, the epipolar lines are the horizontal rows: a match must have the same row, v′ = v, and the test is simply |v′ − v| ≤ t.

How much does each constraint reject?

Suppose a wrong match lands uniformly at random in an image of area A = W H, and the threshold is t pixels. A homography or similarity accepts a disc of area πt²; the epipolar constraint accepts a strip of width 2t along the line, about 2t W in area:

For a 640 × 480 image and t = 1 pixel, that is π/307 200 ≈ 1.0 × 10⁻⁵ for the point model and 2/480 ≈ 4 × 10⁻³ for the epipolar line, about 400 times weaker. So with 200 wrong matches the point model lets in 0.002 of them on average and the epipolar constraint lets in 0.8: the epipolar test is far weaker, and a few wrong matches always remain that happen to lie along their epipolar line (for example because they slid along the line to a similar-looking feature).

Degeneracies

• All points on a plane: the fundamental matrix is not uniquely determined (a homography explains the data); the estimate is unstable. Test for it by also fitting a homography and comparing inlier counts.
• Pure rotation or no translation: the epipolar geometry is undefined (there is no baseline); a homography is correct, and translation direction cannot be estimated.
• Small baseline: matches differ by a few pixels, and noise is comparable to the parallax. Depth and translation are unreliable.
• Collinear sample points: a sample of 4 points on a line does not determine a homography. Reject degenerate samples in RANSAC.

Motion priors

A robot knows something about how it moved: wheel odometry, an IMU or a motion model gives an approximate pose change. Predict where each feature should appear and accept matches within a gate around the prediction. This is a weaker version of the epipolar constraint but needs no estimation, and it also narrows the descriptor search (Lesson P4-T2).

A worked example: using the constraints on real numbers

Homography transfer

A plane is described by H = [[1.02, 0.03, 4], [-0.02, 0.99, -2], [0.0004, -0.0002, 1]]. A feature at (200, 150) in the first image is predicted at

The detector found a match at (202.98, 135.31): a transfer error of 0.72 pixel, so with a threshold of 2 pixels it is an inlier. Another candidate at (216.38, 144.71) has a transfer error of 16.6 pixels and is rejected. The homography also shows why a homography is not a pure shift: at (200, 150) the displacement is (2.4, -14.3), at (600, 400) it would be (-58.6, -70.7).

Epipolar test on a rectified stereo pair

Six matches between the left and right images of a rectified pair. The test is |v′ − v| ≤ 1 pixel:

Match 4 is clearly wrong: its row differs by 11.5 pixels and it is rejected whatever sensible threshold is used. Matches 3 and 6 (rows differing by 1.2 and 1.1 pixels) fail a threshold of 1 pixel and pass a threshold of 1.5: whether they are kept depends on the noise and the quality of the rectification, which is why the threshold must come from measurement (Lesson P4-T4). Note what the test cannot do: if match 2 has paired the left feature with a different but similar post of a fence in the same row, it passes, and so does any other wrong match along that row. The disparity (u − u′ = 19 pixels) would then be wrong, and with it the depth. Further checks are needed: disparity range, ordering and the ratio test.

Choosing the model for a robot

A mobile robot with one camera moving forward in a corridor: a homography is wrong (the scene is not planar), the fundamental or essential matrix with RANSAC is right, and odometry gives a prior that gates matches before RANSAC. A drone looking straight down at the ground: the scene is nearly planar, so a homography explains the data and the fundamental matrix is degenerate. A conveyor belt: a pure translation, one parameter pair. A rotating pan-tilt camera: a homography (rotation only). Matching the model to the motion and the scene is part of the design.

Engineering implications and limitations

Use the simplest model that fits. A model with fewer degrees of freedom needs fewer RANSAC tries and rejects more wrong matches. If the motion is planar or the camera only rotates, use a homography; use the essential matrix only for general motion in a 3D scene.

Fit competing models and compare. Run both a homography and an essential matrix on the same matches. If the homography explains almost as many matches as the essential matrix, the scene is near-planar or the baseline small, and the essential matrix pose is unreliable: handle it as a degenerate case.

Gate with priors. Use odometry or an IMU to predict feature positions and reject matches outside a gate. It is cheap, it uses information the vision system does not have, and it scales with uncertainty: a wider gate when the motion estimate is poor.

Stack the checks. Descriptor distance, ratio test, cross-check, epipolar or homography gating, and a minimum spread of inliers each remove different errors. None is enough alone, and together they are cheap compared with the cost of a wrong pose.

Keep the calibration in the loop. The epipolar constraint uses the intrinsics and, for stereo, the relative pose. A calibration error of a pixel makes a threshold of one pixel meaningless. Lesson P3-T6’s reprojection error is the check.

Look at the failures. Draw the rejected matches. A systematic pattern (all rejected matches on one side) points to a model or calibration error, not to bad matches.

Common misconceptions

Engineering takeaways

1. Models of increasing generality: similarity (s = 2), affine (3), homography (4, planes and pure rotation), epipolar geometry (E: 5, F: 7 or 8, general 3D scenes).
2. Homography: (u′, v′) = H (u, v, 1) divided by w′; transfer error is the distance to the prediction.
3. Epipolar constraint x′ᵀ F x = 0: a match must lie on a line; for rectified stereo the lines are image rows.
4. A point model rejects far more wrong matches than an epipolar line (about 10⁻⁵ against 4 × 10⁻³ pass probability at 1 pixel, 640 × 480); wrong matches along the line survive.
5. Watch for degeneracies (planar scene, pure rotation, small baseline, collinear samples), compare models and gate with motion priors.

References and further reading

Standard references; the homography and stereo examples were computed for this lesson.

• Hartley, R. and Zisserman, A. (2004). Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press: homographies (chapter 4), epipolar geometry and the fundamental matrix (chapters 9 and 11).
• Nistér, D. (2004). An efficient solution to the five-point relative pose problem. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(6): the essential matrix from 5 points.
• Longuet-Higgins, H. C. (1981). A computer algorithm for reconstructing a scene from two projections. Nature 293: the essential matrix and the eight-point idea.
• Szeliski, R. (2022). Computer Vision: Algorithms and Applications, 2nd ed. Springer: two-frame structure from motion and geometric verification.

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Next: where all of this fails in practice: repetitive structure, textureless surfaces and motion blur, and how to notice before the pose goes wrong.

Related content

Nothing below is required to finish this lesson.

• Before
  P4-T4: Robust Estimation and RANSAC

  Fit a model robustly with RANSAC and choose its parameters.

• Lab
  P4-L3: Reject Incorrect Correspondences and Measure the Improvement

  Add RANSAC and a geometric check to a matcher and quantify the gain.