This lab works on phones, but writing code is much easier on a laptop.
LLM · Building LLM Applications · LM11-L2 · Intermediate
A model’s output is text. When an application turns that text into an action, the application, not the model, is responsible for what happens. The model may produce malformed JSON, call a tool that does not exist, pass a number that is out of range, or follow instructions it found in a document.
You will decide which checks a guard needs, write the validator that applies the schema, and execute calls against a safety policy that remembers the robot’s state.
You have passed every stage. Now connect it back to the theory and move on:
Plain Python with math (and slamkit for small matrix helpers: matmul, transpose, solve, inverse, identity). Images, point clouds and detections arrive as lists inside the dictionary D; an image is a list of rows, each row a list of pixels. There is no NumPy or OpenCV: you write the algorithm yourself, which is the point of the lab. Print with print().
Stage 1 explores the policy choices of a guard on twelve fixed model outputs. Stage 2 writes a schema validator. Stage 3 executes tool calls against a safety policy with state. No language model is run and no robot exists: the outputs are fixed strings and the robot is a point on a grid. Your code is graded on hidden outputs and sequences.
Educational simplification: three tools with numeric or text arguments, a rectangular workspace, a point robot. Real systems use JSON Schema libraries, typed tool definitions and many more cases; the structure of the checks is the same.
Get notified when new robotics labs are released. One email per new lab, nothing else.