UAV Systems Engineering · Capstone 1 of 4

Vision-Aided Autonomous UAV

Close the loop on vision: a UAV that estimates its own motion from a stereo camera and an IMU and flies a waypoint mission on that estimate.

Builds on modules 2, 3, 5
About 25–40 hours
Research-level

This project connects modules 2, 3 and 5. You will run a visual-inertial estimator on sensor data, wrap it with monitoring, and use its output to fly waypoints with a trajectory tracker, first on a dataset, then in simulation.

Mission and scope

A UAV must fly a rectangular waypoint mission 20 m by 10 m at 2.5 m/s in a textured indoor environment with no GNSS. Position comes only from a stereo camera and an IMU. The vehicle must return to within 0.5 m of its start after one lap and must stop safely if visual tracking fails.

Work in three stages so that each result is checked before the next depends on it: (A) estimator on a recorded dataset with ground truth, (B) closed loop in simulation, (C) failure injection.

What you will implement

Stereo camera and IMU data pipeline with timestamps preserved from acquisition (lesson M7-T8).
A visual-inertial odometry estimator: run VINS-Fusion or OpenVINS (lessons M4-T4, M4-T5) with calibration and IMU noise parameters you derived (lesson M3-T7).
State estimation wrapper that publishes pose, velocity and covariance and an estimator-health record (feature count, inlier ratio, NIS).
Waypoint navigation with a rate-limited trajectory generator (lesson M5-T10).
Trajectory tracking with PID and with MPC, compared on the same lap (lessons M5-T6, M5-T8).
Evaluation: ATE, RPE and closure error with evo, and tracking error statistics.

System architecture

Capstone 1 architecture. The monitor’s health record is part of the interface, not an afterthought.

Recommended starting points: EuRoC MAV or TUM-VI datasets for stage A (stereo camera, IMU and ground truth); PX4 SITL with a Gazebo stereo-camera model for stages $B$ and $C$ , using the external-odometry bridge of lesson M6-T10 once the estimator works.

Requirements

Capstone requirements and how each is verified
ID	Requirement	Verification method
R1	Estimated position error after one 60 m lap shall be below 1.0% of distance on the dataset.	Test: evo ATE/RPE on the dataset with ground truth
R2	In simulation, the vehicle shall return within 0.5 m of its start after one lap at 2.5 m/s.	Test: simulated closed-loop run, 10 seeds
R3	The estimator shall publish pose, velocity, covariance and health at the camera rate with end-to-end latency below 80 ms (p99).	Test: latency measurement (lesson M7-T6)
R4	If the number of tracked features falls below the threshold for 1 s the vehicle shall hold position, and after 5 s land.	Test: camera dropout injection in simulation
R5	Tracking RMS error for the MPC shall be reported against the PID baseline on the same trajectory.	Analysis: comparison table and plot
R6	Estimator consistency: mean NIS of velocity updates within the 95% interval over the run.	Analysis: NIS plot (lesson M3-T14)

Stage A: estimator on a dataset

Pick a dataset sequence with stereo images, IMU and ground truth and verify its timestamps and calibration files.
Set IMU noise parameters from an Allan analysis or the dataset’s documentation (lesson M3-T7).
Run the estimator, record the trajectory, and evaluate with evo: absolute trajectory error and relative pose error per distance.
Plot feature count and inlier ratio against error: find the frames where error grows.
Write down the estimator’s drift as a percentage of distance and compare with requirement R1.

Stage B: closed loop in simulation

Replace the dataset with a simulated stereo camera and IMU on PX4 SITL (lesson M6-T14).
Feed the estimator output to PX4 as external odometry (lesson M6-T10) or fly in offboard mode using the estimator state directly.
Implement the waypoint generator and fly the lap with a PID cascade, then with an MPC tracker.
Measure return error, tracking error and latency.

Stage C: failures

Inject a camera dropout, a low-texture segment and a 20 ms camera-IMU timing offset. For each, record the monitor’s response and the vehicle’s behaviour, and update the FMEA row for visual odometry failure (lesson M8-T7). A capstone is not complete if the vehicle’s response to a visual failure was not observed.

Deliverables

A short design note: architecture, interfaces and frames (with a table of every topic, frame and rate).
Dataset evaluation report: ATE/RPE plots and table, drift percentage, discussion of the worst segments.
Closed-loop results: tracking error for PID and MPC, return error over 10 runs.
Failure-injection report: what each failure did and how the monitor responded.
Code repository with a README that reproduces every figure from raw data.

Acceptance criteria and grading rubric

Acceptance criteria
Criterion	Acceptable	Strong
Estimator evaluation	ATE and RPE reported on a standard dataset with correct alignment	Also correlates error with feature count and explains the worst segments
Closed loop	Lap flown in simulation with return error reported	Compares controllers with statistics over multiple seeds
Monitoring	A feature-count rule triggers hold and landing	Uses NIS and covariance as well, with a measured false-alarm rate
Timing	Latency reported	Latency budget per stage and a test of jitter effect
Honesty	States what was not tested	Quantifies uncertainty and repeats runs

Common failure points

Starting with the closed loop before the estimator is validated: it is impossible to tell estimator error from controller error.
Using the dataset’s ground truth without the correct alignment (SE(3) or yaw-only): errors look huge or tiny for the wrong reason.
Wrong IMU noise parameters copied from another sensor.
Treating a perfect simulation camera as proof of real-world robustness: repeat with injected noise and blur.
Frames: ENU/FLU in ROS against NED/FRD in PX4 (lesson M1-T4).

References

Record your progress

Capstones are assessed by you against the rubric above, so this records your own sign-off.