M7-T4 — Perception Failures: Glass, Dust and Dropouts

Learning objectives

After this lesson, you should be able to:

• List the main sensing failures (transparent and mirror surfaces, absorbing surfaces, airborne particles, sunlight, dropouts and stale data) and their symptoms.
• Explain how a lidar and a depth camera can disagree, and what the disagreement means.
• Describe cheap checks: range statistics, data age, cross-sensor agreement and scan-to-map match.
• Say what the robot should do when a check fails, and why the navigation lidar is not a safety sensor.

The intuition

A lidar measures how long a pulse of light takes to come back. It has no idea what the surface is. Most walls send enough light back to the sensor. Glass sends most of it straight on, so the pulse continues to whatever lies behind and the sensor reports a distant wall, or nothing. A mirror sends it somewhere else, and the sensor reports a wall that is really a reflection. A matt black surface soaks it up and returns too little. Steam, dust and fog scatter it and return a false obstacle close by.

The common feature is that the failure is consistent with the physics and invisible in the data type. A laser scan with a hole in it looks like a laser scan. The costmap layer will mark and clear exactly as designed, and the planner will follow the result. The system is behaving correctly with wrong inputs, so you cannot find it by looking for errors in the software.

What exposes it is a second opinion. A depth camera, a sonar, a bumper, the map, the odometry or simple statistics on the scan can each disagree with the lidar when the lidar is wrong. The figure shows the most common case in a hospital: a glass partition that Scout’s lidar sees straight through.

The mathematics

A catalogue of failures

Statistics of a scan

The standard beam model of a range sensor (Thrun et al.) mixes four causes of a reading: a correct hit with small noise, an unexpected short return (a person, dust), a maximum-range reading (no return), and a random reading. A scan whose mix departs from the usual is evidence of trouble. Useful numbers to monitor include:

• The fraction of no-return beams in the forward sector. A sudden rise suggests black or transparent surfaces or a dirty window.
• The fraction of very short returns that do not repeat between scans. Dust and steam.
• The match score between the scan and the map at the estimated pose. A drop means the pose or the sensor is wrong (lesson M7-T7).
• The valid-pixel fraction of a depth image, which falls in sunlight.

Data age

If a message arrives with timestamp t_m and is used at time t, its age is t − t_m. A robot moving at v has moved v · age since the measurement, which reduces the sight distance by that amount, or equivalently adds to the latency of lesson M7-T1:

Disagreement as a test

Let two sensors measure the clearance ahead as d₁ and d₂. A conservative rule uses min(d₁, d₂), and a disagreement larger than a threshold |d₁ − d₂| > ε is itself a warning to slow down, because one of them is wrong and the robot does not know which.

A worked example

What the glass takes. Scout’s lidar has 720 beams at 0.5° steps. A glass partition spans 40° of the field of view. Without the glass, those 40° would give 80 beams that stop at the partition. With it, the 80 beams read a wall 6 m behind. For the costmap, the partition does not exist: the beams clear every cell up to 6 m, and there are no marks at the partition at all. In such a scan the glass is invisible, even though 11% of the beams cross it.

A stale scan. The lidar driver stalls for 0.5 s. The costmap still holds the last scan, so the robot trusts a picture that is 0.5 s old. Scout, at 1.2 m/s, has moved 0.6 m without new information. With 2.0 m of sight and a = 0.8 m/s², the speed limit with the normal latency of 0.3 s is

The normal limit of 1.56 m/s is above Scout’s own 1.5 m/s maximum, so the maximum applies; the stale scan lowers the limit to 1.26 m/s. A robot that keeps its normal limit while data is old is driving blind for the age of the data. A data-age watchdog with a threshold of 0.3 s would have cut the limit as soon as the stall began and stopped the robot if it went on.

Two sensors disagree. The lidar reports 4.0 m of free corridor; a forward depth camera reports an obstacle at 1.1 m. The difference is 2.9 m, far above a 0.3 m threshold. With the conservative rule Scout uses 1.1 m, slows to the speed that stops within it (0.8 · (−0.3 + sqrt(0.09 + 2.75)) = 1.1 m/s, before margin), and flags the disagreement. In a corridor with a glass door, the camera is right.

Engineering implications and limitations

Tell the planner what it cannot see

Known troublemakers belong in the map: a glass partition can be drawn as an obstacle in the static layer (lesson M2-T4) or as a keep-out zone, so that it is avoided even though no sensor sees it. Mirrors and polished lift doors can be given a masked region. The map is cheaper than hoping the sensors work.

Several sensors, different physics

Pair a lidar with a sensor that fails differently: sonar or radar for glass, a depth camera for objects below the scan plane, bumpers and cliff sensors as the last line. Use the cautious combination (the smaller clearance) for safety decisions, as the layered costmap does with the maximum cost (lesson M2-T4).

Watchdogs on the data itself

Check, at every cycle, the age of each message, the rate, whether the sequence advances, and the fraction of valid readings. A failed check must change behaviour: slow down, stop, switch to the cautious sensor, or ask for help. A check that only writes to the log is not a defence.

The navigation lidar is not a safety sensor

A lidar used for mapping and avoidance is a general-purpose sensor. Safety-rated scanners that protect people are built and certified for that job, under standards for electro-sensitive protective equipment such as IEC 61496, with defined detection capability and failure behaviour. Where a hospital robot could hurt someone, the protective function should rely on such a device and its safety case, not on the navigation costmap. This lesson teaches the engineering of the navigation side only.

Common misconceptions

Engineering takeaways

1. Glass, mirrors, black surfaces, airborne particles, sunlight and dropouts each fool a sensor in a predictable way.
2. The failure usually looks like valid data, so you find it by second opinions: other sensors, the map, statistics and data age.
3. Stale data costs sight: the effective latency grows by the age and the effective sight shrinks by v · age.
4. A failed check must change behaviour, by slowing, stopping, falling back to a cautious sensor or asking for help.
5. The navigation lidar is not a safety-rated sensor, and this lesson does not replace a safety case.

References and further reading

Probabilistic Robotics gives the beam model; the Koch et al. paper is a study of transparent and reflective surfaces in laser scans.

• S. Thrun, W. Burgard and D. Fox, “Probabilistic Robotics”, MIT Press, 2005 (the beam-based range sensor model, chapter 6).
• R. Koch, S. May, P. Murmann and A. Nüchter, “Identification of transparent and specular reflective material in laser scans to discriminate affected measurements for faultless robotic SLAM”, Robotics and Autonomous Systems, 87, 2017.
• Nav2 documentation, “Obstacle Layer Parameters” and “Collision Monitor”, docs.nav2.org.
• IEC 61496, “Safety of machinery — Electro-sensitive protective equipment” (named for orientation).

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

What’s next

Next: how to organise “try this, else that” into a behaviour tree that wraps planning and recovery.

Related content

Nothing below is required to finish this lesson.

• Before
  M7-T3: Navigation Under Pose Uncertainty

  Use pose covariance to inflate obstacles or tighten speed, and explain why a confident plan on a wrong pose is dangerous.