FixTheRobot Explore Robotics / Systems Engineering / Troubleshooting

Troubleshooting · Sensors

Sensor data drops intermittently

Find where in the chain (sensor, link, driver, middleware, subscriber) the messages disappear, by counting at each stage.

  • Related modules: 02, 04
  • ROS 2 Jazzy

1. Symptoms

  • A topic’s rate dips or stops for a moment and recovers.
  • Gaps in ros2 topic delay or bag timestamps at irregular intervals.
  • The sensor disconnects and reappears in dmesg.

2. Possible causes

  • Power or cable: brownout, loose connector, USB hub or undersized supply (see the electrical guides).
  • Link overload: total bandwidth above what the interface sustains (see the throughput guide).
  • Driver: buffer overflow, blocked read loop, or a slow callback.
  • CPU or memory pressure starving the driver or the DDS threads.
  • QoS mismatch (reliable publisher with a best-effort subscriber or the reverse) or network loss on Wi-Fi.
  • Interference: motors switching on the same supply or cable run.

3. Measurements to collect

  • Rate and the largest gap between messages, at the sensor driver output and at the consumer.
  • Kernel log (dmesg) and driver log around each gap.
  • CPU, memory and link utilisation during gaps.
  • Supply voltage at the sensor during a gap.

4. Diagnostic commands

Commands are not executed by the site’s automated tests (they need a ROS 2 Jazzy installation or the robot).

ros2 topic hz /scan
ros2 topic delay /scan
ros2 topic info /scan --verbose          # QoS of every publisher and subscriber
dmesg -w | grep -i -E "usb|reset|disconnect|eth"
ros2 bag record /scan --max-bag-duration 600

5. Step-by-step investigation

  1. Count at the source and at the consumer. If the driver output already has the gaps, the problem is at or before the driver; if only the consumer sees them, look at QoS, DDS and CPU.
  2. Check the kernel log for resets and disconnects at the gap times.
  3. Compare gap times with events: motor starts, CPU spikes, other sensors starting.
  4. Check QoS compatibility with ros2 topic info --verbose.
  5. Measure the supply voltage and cable at the sensor with an oscilloscope during a gap; swap the cable and port.
  6. Reduce load (rate, resolution, other subscribers) and see whether the gaps disappear.

6. Corrective actions

  • Fix power, connectors and cables; use a powered hub or a dedicated supply.
  • Reduce bandwidth or move to a faster interface.
  • Fix QoS settings so publishers and subscribers are compatible.
  • Give the driver real-time priority or its own core; fix a blocking callback.
  • Separate the sensor supply and cable route from the motors.

7. Validation after the fix

  • Zero gaps longer than twice the nominal period over a run at least as long as the longest mission.
  • No resets in the kernel log.
  • The result holds with all other nodes running and the motors active.

References