FixTheRobot Explore Robotics / Localization & SLAM / M8 / M8-L2

This lab works on phones, but writing code is much easier on a laptop.

Localization & SLAM · Advanced SLAM and Research · M8-L2 · Advanced

Compare SLAM Trajectories Using Evaluation Metrics

Three students claim to have the best SLAM on the same path. One reports 8 cm, one 2 m, one 0.3 m, and each used a different script. You are asked to settle it.

You write the evaluation yourself, then find out which questions the metrics answer and which ranking is fair.

Your task

    Simulation
    solution.pyLoading Python…
    Ctrl+Enter runs
    What Python can I use?

    Plain Python with math and, for matrices, slamkit (matmul, transpose, solve, inverse, identity). The data arrive as the dictionary D. There is no NumPy, random, json or copy in the browser runtime. Your code runs locally and never leaves this page.

    Hints

    What this lab teaches

    Stage 1 implements the metrics of M8-T8: ATE after a closed-form alignment (with an optional scale) and the translational RPE over an interval, graded against reference values for three runs generated in different frames and scales. Stage 2 is the discussion of fairness as an experiment: metric, interval and alignment.

    Why it matters

    Most disagreements between reported SLAM results come from the evaluation protocol rather than the algorithm. Being able to reproduce the numbers is the first step to trusting them.

    Theory for this lab

    Advanced SLAM and Research labs

    1. M8-L1: Handle Dynamic Objects in Mapping
    2. M8-L2: Compare SLAM Trajectories Using Evaluation Metrics: this lab.
    3. M8-L3: Diagnose Localization in a Feature-Poor Environment
    4. M8-L4: Evaluate a SLAM System Under Controlled Failures

    Educational simplification: planar poses; index-aligned samples with no time association; the alignment uses positions only; one RPE interval at a time. Real evaluation tools handle 3D, time stamps, sub-trajectory alignment and several intervals.

    Get the next lab

    Get notified when new robotics labs are released. One email per new lab, nothing else.