This lab works on phones, but writing code is much easier on a laptop.
Perception · Sensor Fusion and Scene Understanding · P8-L3 · Advanced
The camera and the lidar both work. Each one, on its own, finds the cones. But when the robot fuses them, the cone from the camera is half a metre to the left of the same cone from the lidar, and the gap gets bigger the farther away the cone is.
You will read that pattern, compute the transform that explains it, and then find a different fault, a time delay, that looks like a calibration error only when things move.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 reads the residuals between two sensors and corrects them by hand. Stage 2 writes the closed-form estimate of a rotation and a shift from matched points. Stage 3 finds an integer time offset. All points are simulated. Your code is graded on hidden faults that the grader keeps.
Educational simplification: 2D ground-plane points, one rotation and one translation, integer-frame time offsets, independent Gaussian noise and known correspondences. Real calibration also involves 3D rotations, scale and intrinsics, motion distortion within a scan, and unknown correspondences. All data in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.