Extrinsic calibration is wrong

1. Symptoms

• Projected LiDAR points do not line up with image edges.
• The misalignment changes with distance or only appears when the robot moves.
• Fusion output is biased in a consistent direction.

2. Possible causes

• A rotation error in the extrinsic (constant pixel shift at all ranges).
• A translation error (shift larger at close range).
• A time offset between the sensors (shift grows with speed).
• Intrinsic or distortion error (shift grows toward the image edges).
• A mount that moved after calibration (temperature, impact).

3. Measurements to collect

• Pixel offset of a known edge at 5, 10 and 20 m.
• The same offset with the robot still and moving.
• Offset at the centre and at the edge of the image.

4. Diagnostic commands

Commands are not executed by the site’s automated tests (they need a ROS 2 Jazzy installation or the robot).

ros2 run tf2_ros tf2_echo base_link camera
ros2 bag record /points /camera/image_raw /camera/camera_info /tf_static

5. Step-by-step investigation

1. Record a still scene with a flat target at three ranges.
2. Measure the pixel offset at each range: constant means rotation; decreasing with range means translation.
3. Repeat while moving: an offset that appears only in motion is a time offset.
4. Compare centre and edge: growth toward edges points at intrinsics.
5. Re-calibrate the identified term and validate on held-out data.

6. Corrective actions

• Re-calibrate the extrinsic with well-spread poses.
• Correct the time offset in the driver or the synchronisation setup.
• Re-calibrate intrinsics if the edge error is large.
• Stiffen the mount and re-check after thermal cycles.

7. Validation after the fix

• Pixel error below the requirement at 5, 10 and 20 m, still and moving.
• Repeated checks over days agree.

References

• Sensitivity numbers are worked in the extrinsic calibration guide.