This lab works on phones, but writing code is much easier on a laptop.
Perception · Geometry and Camera Perception · P3-L1 · Intermediate
A robot knows where a pallet is in the room and wants to know where it will appear in the camera image, to look there first. The numbers it computes put the pallet in a corner, or on the wrong side of the picture, or in an image it is not even in front of.
You will see how size depends on distance and focal length, write the projection, and then add the camera pose that turns world coordinates into camera coordinates.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 is an experiment with the two numbers that set image size. Stage 2 writes the projection and its visibility test. Stage 3 adds the camera pose and the direction of the rotation. Your code is graded on hidden point sets and camera poses; the simulator keeps the expected pixel positions private.
Educational simplification: an ideal pinhole camera with a 320 by 240 image, square pixels and no lens distortion; points are exact. Real cameras have distortion, rolling shutters and calibration error; Lab P3-L2 treats errors in the camera constants. All data in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.