CPU usage is too high

1. Symptoms

• One or more cores at 100 % in htop and the pipeline falls behind (rates drop, latency rises).
• Frame rate of a node is below its input rate.
• The system is fine at idle and slow when the robot moves or sensors are busy.

2. Possible causes

• An algorithm stage whose cost exceeds the frame period (one core saturated).
• Data volume: raw images or dense point clouds processed at a higher rate or resolution than needed.
• Serialisation and copies between processes for large messages.
• Busy-waiting or a tight loop without sleep; too-fast timers.
• Too many nodes subscribing to the same large topic.
• Thermal or power throttling lowering clock speeds.
• Logging or visualisation (RViz, image viewers) running on the robot.

3. Measurements to collect

• Per-process and per-thread CPU use over time at full load.
• Message rates and bandwidth of the big topics.
• Per-stage processing time per frame.
• CPU frequency and temperature.

4. Diagnostic commands

Commands are not executed by the site’s automated tests (they need a ROS 2 Jazzy installation or the robot).

htop                                               # per-core use; F5 shows the process tree
top -H -p <PID>                                    # per-thread CPU of one process
ps -eLo pid,tid,pcpu,comm --sort=-pcpu | head -20  # busiest threads on the machine
ros2 topic hz /points
ros2 topic bw /points
free -h                                            # memory pressure also shows up as CPU (swapping)

On NVIDIA Jetson, tegrastats also reports CPU frequency, GPU use and temperatures; it is not available on other platforms.

5. Step-by-step investigation

1. Find the owner: in htop sort by CPU and note the process and whether it is one saturated core or spread across cores.
2. Is it data volume? Compute the expected bandwidth (see the throughput guide) and compare with ros2 topic bw. Reduce resolution, rate or crop before processing and see whether CPU falls proportionally.
3. Is it an algorithm stage? Time each stage per frame; compare ms per frame × frames per second with 1000 (a stage above that cannot keep up on one core). Use the CPU/GPU Resource Estimator to see which stage dominates.
4. Is it copying? Large messages between processes cost CPU; consider composing nodes into one process or using shared-memory transport.
5. Rule out throttling: check frequency and temperature during the run.
6. Remove diagnostics: stop viewers and heavy logging and re-measure.

6. Corrective actions

• Reduce data early: downsample point clouds (voxel filter), crop to the region of interest, lower image resolution or rate.
• Move the dominant stage to the GPU only if it parallelises well and the data transfer is cheaper than the saving.
• Run heavy viewers on a separate machine.
• Reduce the number of subscribers to large topics or compose nodes into one process.
• Parallelise independent stages across cores (separate nodes or threads).
• Fix throttling with cooling and a power profile; do not rely on peak clocks.

7. Validation after the fix

• At full load every stage satisfies ms per frame × fps below about 700 (70 % of one core) with margin.
• Total CPU stays below about 70 % across cores during the worst-case mission segment.
• Rates and latency are stable over a 30-minute run, with no thermal throttling.
• Output quality (detections, localisation accuracy) is unchanged after data reduction.

References

• Module 5 and Module 17 overviews for compute sizing and optimisation.

• Real-Time Performance and Resource Constraints SLAM · Lesson