FixTheRobot Explore Robotics / Systems Engineering / Engineering Guides / Module 06

Engineering Guides · Module 06

Transforming points between camera, LiDAR and base frames

Name every transform as target-from-source, compose by chaining, invert for the other direction, and check with a round trip.

  • Intermediate
  • Module 06: Camera–LiDAR Calibration & Coordinate Transformations
  • Written for ROS 2 Jazzy where ROS 2 is used
  • Sources checked 2026-11-06

Assumptions: Numbers in the examples are illustrative and the code is executed by the site’s tests. Facts taken from a cited source are marked; other statements are general engineering knowledge and must be confirmed on the datasheet of the part you use.

1. Engineering problem

Sensors are mounted at different places and orientations. To use their data together every point must be expressed in one frame. Most calibration bugs are a transform applied in the wrong direction or composed in the wrong order.

2. When is this needed?

  • Fusing LiDAR and camera data.
  • Writing a static transform for a new sensor.
  • Debugging data that appears mirrored, shifted or rotated.

3. Selection and design decisions

A rigid transform is a 4×4 matrix [R t; 0 1]. Writing TBL for “base from LiDAR” means: multiply a point in LiDAR coordinates to get its coordinates in the base frame. Chains compose by matching inner names: TCL = TCB · TBL = (TBC)⁻¹ · TBL. The ROS frame conventions are in REP 103 (x forward, y left, z up for body frames; camera optical frames use x right, y down, z forward).

MistakeSymptomCheck
Inverse applied instead of forwardSensor appears mirrored through the originRound trip a known point
Wrong multiplication orderOffsets rotate with the wrong frameNames must chain: C←B then B←L
Body frame used where optical frame is expectedImage axes swappedUse the optical frame for projection (x right, y down, z forward)
Degrees and radians mixedLarge angular errorUse one unit and name it
Parent and child swapped in tf2Transform inverted on the treeCompare with tf2_echo

4. Advantages and disadvantages

  • Store the mounting transforms to a common base (base←sensor) and derive the sensor-to-sensor transforms; this keeps one source of truth per sensor.
  • Direct sensor-to-sensor calibration measures the quantity you need most accurately and must be re-chained to the base.
  • Quaternions avoid gimbal problems; matrices are easier to read.

5. Requirements and constraints

  • Each transform has a documented direction (target from source) and a unit.
  • Rotations are orthonormal with determinant +1.
  • The same transform chain is used in code, URDF and tf2.

6. Architecture

lidar ──T_BL──▶ base ◀──T_BC── camera      T_CL = inv(T_BC) · T_BL

7. Implementation

  1. Measure or calibrate the base←LiDAR and base←camera transforms and record their direction and units.
  2. Compose the camera←LiDAR transform by chaining (below).
  3. Transform a known point and check the result by hand.
  4. Check the round trip (forward then inverse) gives the point back.
  5. Publish the same values as static transforms.
import numpy as np
def T(R, t):
    M = np.eye(4); M[:3, :3] = R; M[:3, 3] = t; return M
def yaw(deg):
    a = np.radians(deg); c, s = np.cos(a), np.sin(a)
    return np.array([[c, -s, 0], [s, c, 0], [0, 0, 1.0]])
T_BL = T(yaw(0), [0.30, 0.00, 0.50])             # base <- lidar
T_BC = T(yaw(5), [0.45, 0.05, 0.40])             # base <- camera (small yaw for illustration)
T_CL = np.linalg.inv(T_BC) @ T_BL                # camera <- lidar = (base <- camera)^-1 (base <- lidar)
p_L = np.array([10.0, 0.0, 0.0, 1.0])
p_B = T_BL @ p_L
p_C = T_CL @ p_L
print("point in base:", np.round(p_B[:3], 3))
print("point in camera body frame:", np.round(p_C[:3], 3))
print("round trip error:", float(np.abs(np.linalg.inv(T_BL) @ p_B - p_L).max()))

Printed (checked by the site’s tests):

point in base: [10.3  0.   0.5]
point in camera body frame: [ 9.808 -0.908  0.1  ]
round trip error: 0.0

Reading it: a point 10 m ahead of the LiDAR is 10.3 m ahead and 0.5 m above the base (the LiDAR is mounted 0.30 m forward and 0.50 m up). In the camera body frame it appears at about (9.81, −0.91, 0.10): the 5° camera yaw and 0.05 m offset shift it to the right of the camera axis. The result is in the camera body frame (x forward); converting to the optical frame needs the fixed axis permutation from REP 103. The mounting values are illustrative.

8. Real examples

  • AMR: LiDAR 0.30 m ahead of and 0.50 m above base_link, camera 0.45 m ahead and 0.40 m above; the camera←LiDAR transform is derived, not measured twice.

9. Failure scenarios

  • The static transform published as child←parent instead of parent←child.
  • The optical-frame permutation forgotten, so the image looks rotated 90°.
  • Degrees given to a function that expects radians.

10. Troubleshooting

  1. Put a known object at a known place and compare predicted and observed positions.
  2. Print T and its inverse and check the translation sign.
  3. Use tf2_echo between the frames and compare with your matrix.

11. Validation

  • The example prints the values shown (executed by the site tests).
  • A known object appears at the predicted position within the calibration accuracy.

12. Engineering checklist

13. Further references

Related on FixTheRobot