Perception debugging guide

1. Engineering problem

When a robot misses or invents an object, the symptom appears at the end of a long pipeline. Debugging from the end is guesswork; checking each stage’s input and output narrows the failure to one stage.

2. When is this needed?

• A detection is missed, doubled, mislocated or invented.
• After any change to a driver, calibration or parameter.
• In field failures with recorded data.

3. Selection and design decisions

Debug in layers: data (is the message arriving, with sensible stamp and frame?), geometry (frames and calibration), processing (per-stage counts and parameters), then association and tracking. Record a bag of the failing situation so each stage can be replayed offline.

4. Advantages and disadvantages

• More logging helps debugging and costs CPU and storage.
• Replay from a bag is reproducible; live tests show real timing.
• Stage counters are cheap and catch most threshold mistakes.

5. Requirements and constraints

• Every stage publishes a count and a processing time.
• Failing cases are recorded as bags and kept.
• Expected ratios per stage are written down from healthy runs.

6. Architecture

bag ─▶ stage 1 ─▶ stage 2 ─▶ stage 3 ─▶ stage 4
          count    count    count    count     compare each ratio with its healthy range

7. Implementation

1. Record a bag containing the failure.
2. Replay it and count items at each stage.
3. Compare each ratio with the healthy range (below) and find the stage that is out of range.
4. Look at that stage’s parameters and inputs in a visualiser.
5. Fix, replay, and add the case to your regression set.

stages = [("raw points", 28000, None), ("after crop", 26000, (0.8, 1.0)), ("after ground removal", 3100, (0.05, 0.5)),
          ("points in clusters", 300, (0.5, 1.0)), ("points in tracked objects", 280, (0.8, 1.0))]
for (n0, c0, _), (n1, c1, rng) in zip(stages, stages[1:]):
    ratio = c1 / c0
    flag = "" if rng[0] <= ratio <= rng[1] else f"  <-- outside expected {rng[0]:.2f}-{rng[1]:.2f}"
    print(f"{n0} -> {n1}: {c0} -> {c1} ({ratio * 100:.1f} %){flag}")

Printed (checked by the site’s tests):

raw points -> after crop: 28000 -> 26000 (92.9 %)
after crop -> after ground removal: 26000 -> 3100 (11.9 %)
after ground removal -> points in clusters: 3100 -> 300 (9.7 %)  <-- outside expected 0.50-1.00
points in clusters -> points in tracked objects: 300 -> 280 (93.3 %)

Reading it: ground removal keeping 11.9 % of points and tracking keeping 93.3 % are in range; clustering keeps only 9.7 % of the points, below the expected 50 to 100 %, so the failing stage is the clustering (for example a minimum-size threshold that is too high). The counts and ranges are illustrative: derive yours from healthy recordings.

8. Real examples

• Phantom obstacle: a counter at each stage shows the points appear after the crop, pointing to a frame error in the crop box.

9. Failure scenarios

• Changing several parameters at once and not knowing which fixed it.
• Debugging live without a recording.
• Trusting the final visualisation without checking frames and stamps.

10. Troubleshooting

1. Start at the first stage and move forward until the data is wrong.
2. Check frames and time before parameters.
3. Keep a regression bag for every fixed bug.

11. Validation

• The example prints the lines shown (executed by the site tests).
• The failing bag is processed correctly after the fix, and the regression set still passes.

12. Engineering checklist

• Failure recorded
• Per-stage counts
• Healthy ranges known
• Frames and stamps checked
• One change at a time
• Regression bag added

13. Further references

• ROS 2 tools: ros2 bag, ros2 topic and tf2 tools; see the tf2 guide.

Related on FixTheRobot