This lab works on phones, but writing code is much easier on a laptop.
Perception · Geometry and Camera Perception · P3-L3 · Intermediate
A robot with a depth camera has to tell how far a panel is and whether it will fit through a gap. The depth image is full of numbers, some of them zero, and the panel is a few dozen pixels across.
You will learn how accurate depth can be, turn the depth image into points, and measure the panel without being fooled by missing pixels.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 is a calculation you can play with: the accuracy of stereo depth against distance and baseline. Stage 2 writes the back-projection and handles invalid pixels. Stage 3 measures an object from a box in the depth image. Your code is graded on hidden scenes; the simulator keeps the true layout private.
Educational simplification: a simulated 120 by 90 depth camera with an ideal pinhole model, a flat floor and a flat panel, 0.3% multiplicative depth noise and 4% random dropouts reported as zero. Real depth sensors also have edge artefacts, multipath and surface-dependent errors. All data in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.