This lab works on phones, but writing code is much easier on a laptop.
Perception · 3D Perception and Point Clouds · P5-L1 · Intermediate
A robot has a cloud of 3D points and no labels. Before it can find ground, walls or poles, it has to read the local shape around every point: is this part of a sheet, a thin rod or a bush?
You will see how the neighbourhood size matters, compute surface normals, and classify points by the shape of their neighbourhood.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 is an experiment on the neighbourhood size. Stage 2 writes the normal estimation from the covariance and its eigenvectors. Stage 3 classifies points by eigenvalue ratios. Your code is graded on hidden clouds; the simulator keeps the true surface kinds and normals private. It builds on, and does not repeat, the Sensors point-cloud labs.
Educational simplification: a simulated cloud of about 205 points on a ground patch, a wall, a thin pole and a bush with 2 cm Gaussian noise and uneven sampling; neighbourhoods are the k nearest points by brute force. Real LiDAR clouds have rings, range-dependent density, and thousands of points to handle. All data in this lab are simulated.
Get notified when new robotics labs are released. One email per new lab, nothing else.