This lab works on phones, but writing code is much easier on a laptop.
Perception · Sensor Fusion and Scene Understanding · P8-L2 · Intermediate
A robot has a camera and a depth sensor. At 3 m the depth sensor is spot on. At 14 m it is off by more than two metres, and the camera, which only knows how big a person looks, is closer to the truth.
You will find where the two cross over, compute the size-based estimate, and measure both by range.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 finds the range at which the two estimates are equally accurate. Stage 2 writes the size-based distance. Stage 3 compares the sources by range bin. All measurements are simulated with stated error models. Your code is graded on hidden data.
Educational simplification: the depth error is modelled as 0.012 z² metres, the box height is rounded to whole pixels, and the class height varies ±12% around its average; real depth sensors have systematic errors and real detectors have box errors that depend on pose and occlusion. All data in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.