FixTheRobot Explore Robotics / Systems Engineering / Engineering Guides / Module 19

Engineering Guides · Module 19

ROS bag recording strategy

Record what you will need to explain a failure, within the disk and the CPU you have, and keep the last minutes in memory.

  • Intermediate
  • Module 19: Testing, Validation & Field Deployment
  • Written for ROS 2 Jazzy where ROS 2 is used
  • Sources checked 2026-11-06

Assumptions: Numbers in the examples are illustrative and the code is executed by the site’s tests. Facts taken from a cited source are marked; other statements are general engineering knowledge and must be confirmed on the datasheet of the part you use.

1. Engineering problem

Failures cannot be reproduced on demand, so the recording is the evidence. Recording everything can fill a disk in hours or overload the computer; recording too little leaves nothing to analyse.

2. When is this needed?

  • Setting up logging for field tests and deployments.
  • When a failure happened and there was no usable recording.
  • When bags are too big.

3. Selection and design decisions

The rosbag2 README (jazzy, read 2026-11-06) documents: compression is off by default; the only compression format available is zstd, enabled with --compression-mode (for example file) and --compression-format zstd; bags can be split by size or duration (--max-bag-size, --max-bag-duration); storage plugins are mcap and sqlite3; QoS can be overridden with --qos-profile-overrides-path (necessary to record best-effort topics reliably); and snapshot mode (--snapshot-mode) keeps an in-memory circular buffer and writes it only when the ~/snapshot service is called, “capturing the last N bytes of incoming data” around an error or point of interest.

ChoiceEffectNote
Topic list vs allControls rate and sizeAlways include /tf, /tf_static, camera_info and the diagnostics
Raw images and cloudsDominate the data volumeRecord compressed images or cropped clouds if the analysis allows
Compression (zstd)Smaller files, more CPURatio depends on the data
SplittingManageable files, safer on a crashChoose by size or duration
Snapshot modeRecords only around eventsNeeds a trigger and memory for the buffer
Storage pluginmcap or sqlite3Check tool support in your pipeline

4. Advantages and disadvantages

  • Record everything is simple and expensive; selected topics need foresight.
  • Compression saves disk and costs CPU on a loaded computer.
  • Snapshot mode saves disk and misses what happened before the buffer.

5. Requirements and constraints

  • Disk budget for the longest mission with margin.
  • Recording does not starve the robot’s own CPU or disk I/O.
  • Clock and metadata (software version, configuration) are recorded with the bag.

6. Architecture

topics ─▶ rosbag2 recorder (QoS overrides, compression, splitting) ─▶ disk
                 └▶ snapshot buffer in memory ─▶ dumped on an event

7. Implementation

  1. List the topics needed to analyse each failure type.
  2. Estimate the data rate and disk need (below).
  3. Choose compression, splitting and snapshot settings.
  4. Test recording under full robot load.
  5. Check a bag by replaying it (see the bag replay guide).
topics = {"/scan": (40e3, 10), "/points": (2.1e6, 10), "/camera/image_raw": (2.76e6, 15), "/imu/data": (300, 200), "/tf": (2e3, 50), "/odom": (700, 50)}   # (bytes per message, Hz)
raw = sum(b * hz for b, hz in topics.values())
print(f"uncompressed rate: {raw / 1e6:.1f} MB/s = {raw * 3600 / 1e9:.0f} GB per hour")
excl = {k: v for k, v in topics.items() if k != "/camera/image_raw"}
print(f"without the raw camera: {sum(b * hz for b, hz in excl.values()) / 1e6:.1f} MB/s")
ratio = 0.5                                            # assumed compression ratio (illustrative; depends on the data)
print(f"with compression at an assumed ratio {ratio}: {raw * ratio * 3600 / 1e9:.0f} GB per hour")
disk_gb = 500
print(f"500 GB disk holds about {disk_gb / (raw * 3600 / 1e9):.1f} hours uncompressed")
snap_s = 30
print(f"snapshot buffer for the last {snap_s} s needs about {raw * snap_s / 1e6:.0f} MB of memory")

Printed (checked by the site’s tests):

uncompressed rate: 63.0 MB/s = 227 GB per hour
without the raw camera: 21.6 MB/s
with compression at an assumed ratio 0.5: 113 GB per hour
500 GB disk holds about 2.2 hours uncompressed
snapshot buffer for the last 30 s needs about 1890 MB of memory

Reading it: the example topic set is 63 MB/s (227 GB per hour), of which the raw camera is two thirds; without it 21.6 MB/s; with an assumed compression ratio of 0.5 about 113 GB per hour; a 500 GB disk holds 2.2 hours uncompressed; and a 30 s snapshot buffer needs about 1.9 GB of memory. Sizes and rates are illustrative.

8. Real examples

  • Delivery robot: snapshot mode with a 30 s buffer, dumped when the localisation monitor raises an error; full recording only on test days.

9. Failure scenarios

  • No /tf_static in the bag, so nothing transforms on replay.
  • The disk fills overnight.
  • A recorder that overloads the CPU and causes the failure it is meant to capture.

10. Troubleshooting

  1. Check message counts per topic against the expected rates.
  2. Check CPU and disk write speed while recording.
  3. Check that QoS overrides were used for best-effort topics.

11. Validation

  • The calculation prints the lines shown (executed by the site tests).
  • A test bag replays and contains every topic and the expected counts.

12. Engineering checklist

13. Further references

Related on FixTheRobot