This lab works on phones, but writing code is much easier on a laptop.
Perception · Object Detection and Semantic Understanding · P6-L1 · Intermediate
A detector has been bolted onto a robot and reports a list of numbers. Some objects appear twice, some boxes sit on empty floor, and the scores do not mean what the engineer thought.
You will read the raw output, decode it into boxes with classes and probabilities, suppress duplicates, and see what a class-per-pixel map leaves out.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 is a reading exercise on scores and thresholds. Stage 2 decodes raw detector arrays. Stage 3 writes non-maximum suppression. Stage 4 turns a class map into regions and shows why semantic labels cannot separate touching objects. The detector in this lab is a simulation with a stated error model, not a trained neural network. Your code is graded on hidden scenes; the simulator keeps the true boxes and classes private.
Educational simplification and honesty note: all detections and class maps are produced by a simulator from known ground truth using an explicit error model (missed objects, jittered boxes, wrong classes, duplicates and false alarms, controlled by object size and occlusion). No neural network is run, and nothing here shows how a real network performs. The raw output arrays follow the format that typical detectors use so that decoding them is realistic. All data in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.