Optimizing point-cloud pipelines

1. Engineering problem

A 3D LiDAR produces tens to hundreds of megabits per second. Every subscriber in another process receives its own copy, every stage may copy again, and large messages stress the transport. Reducing and shaping the data early costs less than buying a faster computer.

2. When is this needed?

• Point-cloud topics saturate CPU or network.
• Adding more consumers of the same cloud.
• Large messages are delayed or dropped.

3. Selection and design decisions

Levers: downsample and crop early (see the point-cloud workflow guide); compose nodes that share clouds into one process and publish with unique pointers (zero-copy in one process, per the Jazzy documentation read 2026-11-06); use loaned messages only for plain fixed-size types with middleware support; tune the transport. The ROS 2 DDS-tuning guide (read 2026-11-06) describes two real problems: over lossy (Wi-Fi) links a missing UDP fragment leaves the received fragments in the kernel buffer (default 256 KB) until a 30 s timeout, making the connection seem to hang, mitigated by lowering net.ipv4.ipfrag_time; and with Cyclone DDS large messages over a wired network may need a larger net.core.rmem_max and a minimum socket receive buffer.

4. Advantages and disadvantages

• Downsampling trades detail for speed; do it as early as the algorithm allows.
• Compression trades CPU for bandwidth.
• Composition trades isolation for performance.

5. Requirements and constraints

• Bandwidth per subscriber is computed and fits the link at 70 % or less.
• The number of copies in the pipeline is known.
• Kernel buffer settings are applied on every machine that receives the cloud.

6. Architecture

driver ─▶ crop/downsample ─▶ (same process) filters ─▶ consumers
          large raw cloud only to nodes that need it

7. Implementation

1. Compute the bandwidth of each cloud topic and its subscribers (below).
2. Move cropping and downsampling to the front.
3. Compose nodes that exchange clouds and pass by unique pointer.
4. Tune kernel and DDS buffers if large messages drop.
5. Re-measure bandwidth, CPU and latency.

def cloud_mbps(points, point_step, hz):
    return points * point_step * hz * 8 / 1e6
raw = cloud_mbps(131072, 16, 10)
print(f"raw cloud: {raw:.0f} Mbit/s on the wire per subscriber")
for sub in (1, 3, 6):
    print(f"  {sub} cross-process subscriber(s): {raw * sub:.0f} Mbit/s if each receives its own copy (unicast)")
for voxel_keep in (0.5, 0.2, 0.1):
    print(f"  after downsampling to {voxel_keep * 100:.0f} % of the points: {raw * voxel_keep:.0f} Mbit/s per subscriber")
mem_copy_gbs = 5.0
mb_per_msg = 131072 * 16 / 1e6
print(f"one message is {mb_per_msg:.1f} MB; a memory copy at {mem_copy_gbs} GB/s (illustrative) takes {mb_per_msg / (mem_copy_gbs * 1000) * 1000:.2f} ms, times 10 Hz times each copy made in the pipeline")

Printed (checked by the site’s tests):

raw cloud: 168 Mbit/s on the wire per subscriber
  1 cross-process subscriber(s): 168 Mbit/s if each receives its own copy (unicast)
  3 cross-process subscriber(s): 503 Mbit/s if each receives its own copy (unicast)
  6 cross-process subscriber(s): 1007 Mbit/s if each receives its own copy (unicast)
  after downsampling to 50 % of the points: 84 Mbit/s per subscriber
  after downsampling to 20 % of the points: 34 Mbit/s per subscriber
  after downsampling to 10 % of the points: 17 Mbit/s per subscriber
one message is 2.1 MB; a memory copy at 5.0 GB/s (illustrative) takes 0.42 ms, times 10 Hz times each copy made in the pipeline

Reading it: a 131 072-point cloud with 16-byte points at 10 Hz is 168 Mbit/s per subscriber; three unicast subscribers need 503 Mbit/s and six exceed 1 Gbit/s; downsampling to 10 % of the points cuts it to 17 Mbit/s per subscriber. A 2.1 MB message costs about 0.42 ms per copy at an illustrative 5 GB/s. The numbers are illustrative.

8. Real examples

• Mapping robot: crop and downsample in the driver container; the SLAM and obstacle nodes share the reduced cloud in one process.

9. Failure scenarios

• Six nodes subscribing to the raw cloud over Ethernet.
• Large messages dropped because of a small kernel receive buffer.
• Compression enabled on a CPU that is already saturated.

10. Troubleshooting

1. Measure the bandwidth with ros2 topic bw.
2. Check kernel buffer sizes and drops.
3. Count the copies between driver and consumer.

11. Validation

• The calculation prints the lines shown (executed by the site tests).
• Bandwidth and CPU meet the budget at full rate with all consumers running.

12. Engineering checklist

• Bandwidth per subscriber
• Downsample early
• Composition and unique_ptr
• Buffer tuning
• Re-measured

13. Further references

• ROS 2 Jazzy, Intra-process communication demo: docs.ros.org, read 2026-11-06.
• ROS 2 Jazzy, DDS tuning: docs.ros.org, read 2026-11-06.
• ROS 2 Jazzy, Configure zero copy loaned messages: docs.ros.org, read 2026-11-06.

Related on FixTheRobot