Engineering Guides · Module 06

LiDAR–IMU and camera–IMU calibration

Before the spatial calibration, the sensors must agree on time. Cross-correlating the rotation rate seen by each sensor reveals the offset.

Intermediate
Module 06: Camera–LiDAR Calibration & Coordinate Transformations
Written for ROS 2 Jazzy where ROS 2 is used
Sources checked 2026-11-06

Assumptions: Numbers in the examples are illustrative and the code is executed by the site’s tests. Facts taken from a cited source are marked; other statements are general engineering knowledge and must be confirmed on the datasheet of the part you use.

1. Engineering problem

Fusing IMU data with LiDAR or camera data needs two things: the rigid transform between the sensors, and the time offset between their clocks. An offset of tens of milliseconds produces errors on a turning robot even with a perfect spatial calibration.

2. When is this needed?

Setting up LiDAR-inertial or visual-inertial odometry.
When odometry diverges only during turns.
After changing drivers, clocks or the sync setup.

3. Selection and design decisions

The time offset can be estimated by comparing a quantity both sensors observe: the magnitude of the angular rate. The IMU measures it directly; the LiDAR or camera gives it through registered or tracked motion. The offset is the lag that maximises their correlation. The rotation between the sensors is estimated by aligning rotation axes over a motion that excites all axes. Dedicated tools (for example Kalibr for camera–IMU) also estimate the transform and time offset; read the tool’s documentation for its inputs and assumptions.

Overview
Item	What to estimate	How
Time offset	Seconds by which one stream lags the other	Cross-correlate angular-rate magnitude; refine in the estimator
Rotation	Fixed rotation between sensor frames	Align rotation axes over rotation about all three axes
Translation (lever arm)	Offset between sensor origins	Needs accelerating motion; matters less at low acceleration
IMU biases and noise	Gyro and accelerometer bias, noise densities	Static recording and Allan variance

4. Advantages and disadvantages

Excite all axes during calibration, which a ground robot rarely does; use a hand-held rig or a motion table.
Fixed offset vs estimated online: online estimation absorbs drift and can mask a wrong calibration.
Sample rates limit the resolution of a correlation; interpolate or refine.

5. Requirements and constraints

Calibration motion includes rotation about all three axes and enough acceleration.
Both streams have trustworthy time stamps (see the timing guides).
Offset resolution is better than a fraction of the sample period.

6. Architecture

IMU |ω| and sensor-derived |ω| ─▶ cross-correlation ─▶ time offset ─▶ rotation/lever-arm estimation ─▶ validate on new data

7. Implementation

Record a motion with rotation about all axes and moderate acceleration.
Compute the angular-rate magnitude from each sensor.
Find the lag that maximises correlation (below) and refine by interpolation.
Apply the offset, estimate the rotation and lever arm, and check the residual.
Validate by running the odometry and checking drift on a new trajectory.

import numpy as np
rate = 100.0
n = 400
t = np.arange(n) / rate
imu = np.exp(-((t - 2.0) / 0.3) ** 2) + 0.6 * np.exp(-((t - 3.2) / 0.2) ** 2)   # |angular rate| from the IMU
shift = 0.037                                                                 # true offset 37 ms: the other sensor's copy is later
other = np.interp(t - shift, t, imu)
lags = np.arange(-30, 31)
score = [float(np.dot(imu[max(0, -k):n - max(0, k)], other[max(0, k):n - max(0, -k)])) for k in lags]
k = int(lags[int(np.argmax(score))])
print("best lag:", k, "samples =", k / rate * 1000, "ms (true 37 ms; resolution 10 ms)")

Printed (checked by the site’s tests):

best lag: 4 samples = 40.0 ms (true 37 ms; resolution 10 ms)

Reading it: a synthetic 37 ms lag is found as 4 samples = 40 ms because the integer-lag correlation has 10 ms resolution at 100 Hz; interpolating around the peak or optimising the offset inside the estimator recovers the remainder. The signals are synthetic.

8. Real examples

LiDAR-inertial odometry: a 20–40 ms offset between driver stamps and sensor time makes the map smear on turns until it is corrected.

9. Failure scenarios

Calibration on a straight-line drive that never excites yaw.
An integer-sample offset treated as exact.
Sync changed later without recalibrating.

10. Troubleshooting

If drift appears only in turns, suspect time offset.
If the correlation has no clear peak, the motion was too small or a stream is noisy.
Repeat with a different motion to confirm.

11. Validation

The example prints the values shown (executed by the site tests).
Residual after correction is below the requirement.
The odometry drift on a validation run improves.

13. Further references

The method is a standard one; check the documentation of your calibration tool (for example Kalibr or the LiDAR-inertial package you use) for its assumptions.
ROS 2 sensor_msgs/CameraInfo and Image (Jazzy): ros2/common_interfaces, read 2026-11-05.