Timestamps in ROS 2 and message_filters

1. Engineering problem

Fusing two sensor streams needs pairs of messages from the same time. Real sensors never produce identical stamps unless they are triggered together. The synchroniser must decide how close is close enough.

2. When is this needed?

• Matching camera and LiDAR messages.
• When a synchronised callback never fires.
• When fused results jitter because pairs are poorly aligned.

3. Selection and design decisions

From the Jazzy message_filters documentation: TimeSynchronizer passes messages only when all channels have messages with the same timestamp; the policies for the Synchronizer are ExactTime (identical stamps required), ApproximateEpsilonTime (identical within an epsilon) and ApproximateTime (an adaptive algorithm matching by stamp); TimeSequencer passes messages in timestamp order after a queue delay (C++ only: it is not implemented in Python); Cache keeps the last N messages for lookup by time. Messages need a header.

4. Advantages and disadvantages

• Small slop gives better alignment and fewer pairs; large slop gives more pairs with misaligned data.
• Queue size must hold messages long enough for the slowest sensor’s latency.
• Interpolating the faster stream to the slower stream’s stamp can beat pairing.

5. Requirements and constraints

• The slop is derived from the maximum allowed timing error (speed × slop below the spatial tolerance).
• The queue size covers the latency difference between streams.
• Stamps are capture times (see the hardware versus software guide).

6. Architecture

lidar ─┐
        ├─▶ ApproximateTime(slop) ─▶ callback(lidar_msg, camera_msg)
camera ─┘

7. Implementation

1. Decide the allowed timing error and set the slop below it.
2. Choose the queue size from the latency difference and rates.
3. Use ApproximateTime for independent sensors; ExactTime only for triggered ones.
4. Log the stamp difference of each matched pair and watch for drift.
5. Count unmatched messages.

def approx_sync(a, b, slop):
    out, used = [], set()
    for ta in a:
        best = min(((abs(ta - tb), j) for j, tb in enumerate(b) if j not in used), default=None)
        if best and best[0] <= slop:
            used.add(best[1]); out.append((ta, b[best[1]]))
    return out
lidar = [0.00, 0.10, 0.20, 0.30]
camera = [0.013, 0.118, 0.215, 0.331]
for slop in (0.0, 0.02, 0.05):
    pairs = approx_sync(lidar, camera, slop)
    print(f"slop {slop}: {len(pairs)} pairs", [(round(x, 3), round(y, 3), round((y - x) * 1000)) for x, y in pairs])

Printed (checked by the site’s tests):

slop 0.0: 0 pairs []
slop 0.02: 3 pairs [(0.0, 0.013, 13), (0.1, 0.118, 18), (0.2, 0.215, 15)]
slop 0.05: 4 pairs [(0.0, 0.013, 13), (0.1, 0.118, 18), (0.2, 0.215, 15), (0.3, 0.331, 31)]

Reading it: with the stamps offset by 13 to 31 ms, an exact match (slop 0) finds nothing, 20 ms finds three of four pairs, and 50 ms finds all four but with a 31 ms misalignment in the last pair. A simple greedy matcher stands in for the adaptive ApproximateTime algorithm, so the numbers illustrate the trade-off and are not the library’s output. The stamps are illustrative.

8. Real examples

• Camera–LiDAR fusion: an ApproximateTime synchroniser with a slop of 20 ms and a queue of 10.

9. Failure scenarios

• ExactTime used for independent sensors: the callback never fires.
• A huge slop that matches frames 100 ms apart.
• A queue too small for a delayed stream, so matches are dropped silently.

10. Troubleshooting

1. Print the stamp differences of each pair.
2. Check that both streams have header stamps from the same clock.
3. Increase the queue size if messages are dropped.

11. Validation

• The example prints the lines shown (executed by the site tests).
• The distribution of stamp differences of matched pairs is below the requirement.

12. Engineering checklist

• Policy chosen
• Slop derived from the requirement
• Queue size set
• Stamp differences logged
• Unmatched counted
• Same clock for all stamps

13. Further references

• ROS 2 message_filters documentation (Jazzy branch): doc/index.rst, read 2026-11-06.

Related on FixTheRobot