Engineering Guides · Module 10

Diagnosing camera–IMU calibration and time offset

A time offset of tens of milliseconds is invisible when still and fatal in a fast turn. Estimate it with the rotation rate seen by both sensors.

Intermediate
Module 10: Visual-Inertial SLAM & LiDAR-Inertial Integration
Written for ROS 2 Jazzy where ROS 2 is used
Sources checked 2026-11-06

Assumptions: Numbers in the examples are illustrative and the code is executed by the site’s tests. Facts taken from a cited source are marked; other statements are general engineering knowledge and must be confirmed on the datasheet of the part you use.

1. Engineering problem

The camera and the IMU have different clocks and delays. Even a perfect rotation and translation calibration fails if the time offset is wrong, because the estimator compares the image motion with the IMU motion at the wrong instant.

2. When is this needed?

VIO drifts only during motion.
After changing the camera driver or exposure settings.
Before a full calibration.

3. Selection and design decisions

Kalibr (README read 2026-11-06) provides spatial and temporal calibration of an IMU with respect to a camera system, together with IMU intrinsic parameters; it needs a calibration target and an excited motion. A quick check without a target: compare the angular-rate magnitude from the gyro with the rate estimated from the images (for example by a visual odometry front end) and find the lag that maximises correlation, refining the integer-sample lag with a parabola around the peak.

Overview
Method	Needs	Gives
Cross-correlation of rates	Rotation motion, an image-based rate	Time offset only
Kalibr camera–IMU	Target (for example an AprilGrid), IMU noise model, excited motion	Extrinsics, time offset, intrinsics
Online estimation in the VIO	The estimator supports it	Tracks drift of the offset
Hardware trigger	Camera with trigger input	Removes most of the offset

4. Advantages and disadvantages

Offline calibration is more accurate; online estimation tracks changes and can hide a wrong initial value.
Excitation must include fast rotations on all axes.
Rolling-shutter cameras add a per-row time term.

5. Requirements and constraints

Calibration data includes all-axis rotation and moderate acceleration.
Exposure settings are fixed during calibration and use.
The offset is re-checked after any sync, exposure or driver change.

6. Architecture

gyro |ω| ─┐
           ├─ cross-correlate ─▶ lag (refined) ─▶ time offset ─▶ estimator parameter
visual |ω| ─┘

7. Implementation

Record motion with fast rotations.
Compute the angular-rate magnitude from the gyro and from the images.
Find the lag and refine it (below).
Apply the offset in the driver or estimator and re-test.
Run a full calibration tool for extrinsics and intrinsics.

import numpy as np
fs = 200.0
t = np.arange(0, 6, 1 / fs)
gyro = np.exp(-((t - 2) / 0.4) ** 2) + 0.7 * np.exp(-((t - 4) / 0.3) ** 2)
shift = 0.0137                                        # camera-derived rate lags the IMU by 13.7 ms
cam = np.interp(t - shift, t, gyro)
lags = np.arange(-20, 21)
n = len(t)
score = np.array([np.dot(gyro[max(0, -k):n - max(0, k)], cam[max(0, k):n - max(0, -k)]) for k in lags])
i = int(np.argmax(score))
y0, y1, y2 = score[i - 1], score[i], score[i + 1]
frac = 0.5 * (y0 - y2) / (y0 - 2 * y1 + y2)           # parabolic refinement around the peak
est = (lags[i] + frac) / fs
print(f"integer lag {lags[i]} samples = {lags[i] / fs * 1000:.1f} ms; refined {est * 1000:.1f} ms (true 13.7 ms)")

Printed (checked by the site’s tests):

integer lag 3 samples = 15.0 ms; refined 13.7 ms (true 13.7 ms)

Reading it: the integer-sample correlation at 200 Hz finds 3 samples (15.0 ms); the parabolic refinement gives 13.7 ms, equal to the synthetic true value. Real data are noisier, so refine and repeat on several recordings.

8. Real examples

Handheld VIO rig: a 14 ms offset corrected, after which turning drift drops.

9. Failure scenarios

A calibration with only slow motion.
A fixed offset applied after the driver changed buffering.
Assuming the datasheet latency.

10. Troubleshooting

Check that the correlation peak is clear and repeatable.
Repeat with a different motion.
Check exposure time: the stamp should be the exposure start or centre, as the driver documents.

11. Validation

The example prints the line shown (executed by the site tests).
Turning drift is below the requirement after the offset is applied.

13. Further references

Kalibr README: ethz-asl/kalibr, read 2026-11-06 (it states that Kalibr performs visual-inertial calibration: spatial and temporal calibration of an IMU with respect to a camera system, along with IMU intrinsic parameters).