FixTheRobot Explore Robotics / Systems Engineering / Troubleshooting

Troubleshooting · Compute and ROS 2

ROS 2 messages are delayed

Delay is queueing somewhere. Find the queue: publisher, middleware, network, or subscriber.

  • Related modules: 17
  • ROS 2 Jazzy

1. Symptoms

  • Messages arrive late but intact.
  • The delay grows over time.
  • Delay appears only on a wireless link or with large messages.

2. Possible causes

  • A subscriber slower than the publisher, with a deep reliable queue building a backlog.
  • A single-threaded executor or a blocking callback delaying others.
  • Large messages fragmented and retransmitted over a lossy link.
  • Kernel buffers too small for the message size.
  • Clock offset between machines, so the measured delay is not real.
  • Stamp set at arrival instead of acquisition.

3. Measurements to collect

  • ros2 topic delay and hz at the subscriber.
  • Queue depth and QoS on both ends.
  • CPU of the subscriber and its callback time.
  • Network drops and kernel buffer statistics.

4. Diagnostic commands

Commands are not executed by the site’s automated tests (they need a ROS 2 Jazzy installation or the robot).

ros2 topic delay /scan
ros2 topic hz /scan
ros2 topic info /scan --verbose
sysctl net.core.rmem_max net.ipv4.ipfrag_time

5. Step-by-step investigation

  1. Check clocks on both machines first.
  2. Check QoS and queue depth on both ends.
  3. Check the subscriber’s callback time and CPU.
  4. Check message size and the link (fragmentation, Wi-Fi).
  5. Check kernel buffers.

6. Corrective actions

  • Use sensor-data QoS with a small queue for streams.
  • Speed up or parallelise the subscriber; move heavy work out of callbacks.
  • Reduce message size or use a wired link.
  • Tune kernel and DDS buffers as in the ROS 2 DDS tuning guide.
  • Synchronise clocks.

7. Validation after the fix

  • Measured delay is stable and within the budget under full load.

References

  • ROS 2 Jazzy, DDS tuning: docs.ros.org, read 2026-11-06.
  • ROS 2 Jazzy, About Quality of Service settings: docs.ros.org, read 2026-11-06.