This lab works on phones, but writing code is much easier on a laptop.
Localization & SLAM · Visual and Visual-Inertial SLAM · M6-L1 · Intermediate
The visual odometry front end keeps losing its features. In the logs the surviving ones slide along edges, and on the fast turns nearly all of them vanish.
You repair the tracker in the order it fails: which pixels to pick, how to follow them, and what to do when the image moves or blurs too much.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math and, for matrices, slamkit (matmul, transpose, solve, inverse, identity). The data arrive as the dictionary D. There is no NumPy, random, json or copy in the browser runtime. Your code runs locally and never leaves this page.
Stage 1 turns the structure-tensor argument of M6-T1 into a detector, graded against the true corners of hidden images. Stage 2 implements the Lucas–Kanade tracker of M6-T2 with iteration and a validity flag, graded on shifted images and on decoy points in empty regions. Stage 3 is an experiment on the three things that most often break tracking: motion size, blur and the pyramid.
Every visual SLAM system starts here. A tracker that does not know when it has failed feeds wrong measurements to the estimator, and the damage appears many modules later as drift.
Educational simplification: synthetic grey-scale images of soft-edged rectangles (48 × 48 and 96 × 96), pure translation between frames, brightness constancy, a 7 × 7 window and nearest-integer start points. Real trackers use pyramids by default, sub-pixel detection, affine or photometric models and a grid for coverage.
Get notified when new robotics labs are released. One email per new lab, nothing else.