This lab works on phones, but writing code is much easier on a laptop.
Perception · Perception Foundations · P1-L3 · Beginner
A small robot has to pick up the red block. Its camera sees four blocks on the floor. Between the image and the grip there are four steps: find the block, say where it is, choose the nearest one, decide what to do.
Each step has a bug. You will repair them one at a time and then run the whole chain.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 finds the red block with a colour test that survives a change of light. Stage 2 moves the answer from image coordinates to the robot frame. Stage 3 connects the pieces to a decision and handles the cases where perception finds nothing, finds two things, or finds something at the edge of the grasp zone. Your code is graded on hidden images; the simulator keeps the true block positions private.
Educational simplification: a simulated 40 by 28 pixel camera looking straight down at a flat floor with a known scale and a robot at a known pixel; blocks are flat rectangles with pure colours and Gaussian pixel noise. Real images have perspective, shadows, reflections and lens distortion. All images in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.