M8-T5: Flight-test planning

Learning objectives

After this lesson, you should be able to:

• Write a test card with objective, configuration, entry criteria, steps, abort criteria and data to record.
• Plan a staged build-up of test points for a navigation capability.
• Define safety measures: pilot, observer, geofence, kill switch, site risk assessment.
• Handle data and debriefs so that every flight produces usable, versioned evidence.

Conceptual explanation

Established The staged build-up with test cards, abort criteria and a safety pilot is standard in flight-test practice and uncrewed-aircraft testing. Regulatory requirements for operating a UAV (permissions, airspace, insurance, operator qualifications) depend on the jurisdiction and are outside this lesson: comply with the rules that apply where you fly.

Elements of a test plan:

• Test points: the specific conditions to be flown (a hover, a 10 m translation, a 30 m route in the lit corridor, GNSS disabled for 60 s, a deliberate VO dropout), ordered from low to high risk.
• Test card per flight: objective, hypothesis, configuration (baseline ID), entry criteria, step list, abort criteria, expected result, data to record and who is responsible for each action.
• Safety measures: a safety pilot able to take manual control at any time, a flight termination or kill switch tested on the ground, a geofence and return-to-launch configuration, a clear area, an observer, weather limits, a site risk assessment, an emergency plan.
• Build-up: change one thing per step (environment, speed, duration, autonomy level), and proceed only after reviewing the log of the previous step.
• Debrief and data: logs, video and notes archived with the baseline; surprises become defects or new test points.

Engineering intuition

The pilot’s authority is the key safety element. Autonomy testing without a safety pilot who knows the abort criteria and has practised the takeover is just hoping. The abort criteria must be written before the flight, in measurable terms (“abort if horizontal position innovation test ratio exceeds 1.0 for more than 1 s”, “if position error from the reference exceeds 1.5 m”, “if battery below 30 %”), so that no decision has to be made under stress.

Build-up rules: each flight changes one variable; if it passes, the next step extends one variable. When something unexpected happens, step back to the last known good condition and understand it before going forward. Schedule pressure is the usual reason this discipline is dropped; the test plan is where you resist it.

Mathematical formulation

Risk of a test point can be reasoned about as likelihood × severity, with the severity bounded by the safety measures. For a planned excursion, the maximum position excursion from a failure that is detected after $T_{d}$ and acted on after $T_{a}$ at speed $v$ is

For example at 3 m/s with a detection time of 0.5 s, pilot reaction of 1.0 s and braking at $3\,\mathrm{m}/\mathrm{s}^{2}$, the excursion is 3 × 1.5 + 9/6 = 6 m, so a safety distance of at least 6 m plus margin must exist in every direction of travel; at 6 m/s it becomes 6 × 1.5 + 36/6 = 15 m. This is why speed is the variable to build up slowly. The number of flights needed for a statistical claim follows the requirement’s statistic (see the requirements lesson): the test plan must contain enough repetitions at the condition that the requirement’s claim is made.

$v$

vehicle speed (m/s).

$T_{d}$, $T_{a}$

detection time and action (pilot reaction) time (s).

$a_{b}$

braking deceleration $(m/s^{2})$.

$d_{\text{safe}}$

available clear distance.

Architecture and data flow

Test plan (test points, mapping to requirements) → per-flight test card (baseline ID, configuration manifest) → pre-flight checklist and go/no-go → flight with live monitoring (ground station, safety pilot, observer calling aborts) → logs, video, notes → debrief within a day → analysis scripts produce metrics and pass/fail → results attached to the traceability matrix → decision on the next test card. Anomalies go to the defect tracker with the log excerpts.

Real-world UAV applications

• Staged testing of a new VIO configuration outdoors.
• First flights of GNSS-denied navigation with a safety pilot.
• Fault-injection flights with bounded consequences.
• Preparing acceptance flights against measurable criteria.

Implementation example

This runs as shown. A test-card checker: it verifies that every card has the required fields, measurable abort criteria, a safety pilot, an excursion distance that fits the available clear distance, and an entry criterion that refers to the previous card’s result. The cards are illustrative.

import re
REQUIRED = ['id', 'objective', 'baseline', 'entry', 'steps', 'abort', 'record', 'pilot', 'speed_mps', 'clear_m']
cards = [
 dict(id='FT-01', objective='hover 60 s in the lit corridor with VIO fused; confirm estimator health', baseline='v1.4', entry=['ground checks passed', 'HIL pass'],
      steps=['arm', 'take off to 2 m', 'hover 60 s', 'land'], abort=['pilot takeover if position error > 0.5 m', 'abort if EKF test ratio > 1.0 for 1 s', 'battery < 30 %'],
      record=['ulog', 'video', 'motion capture'], pilot='A. Pilot', speed_mps=0.0, clear_m=6.0),
 dict(id='FT-02', objective='translate 10 m at 1 m/s with VIO; compare with motion capture', baseline='v1.4', entry=['FT-01 pass'],
      steps=['take off', 'fly 10 m', 'hold', 'return', 'land'], abort=['pilot takeover if position error > 1.0 m', 'abort if EKF test ratio > 1.0 for 1 s'],
      record=['ulog', 'video', 'motion capture'], pilot='A. Pilot', speed_mps=1.0, clear_m=6.0),
 dict(id='FT-03', objective='GNSS-denied 5-minute route at 4 m/s', baseline='v1.4', entry=['all good'],
      steps=['take off', 'fly route'], abort=['abort if it looks bad'], record=['ulog'], pilot='', speed_mps=4.0, clear_m=6.0)]
def excursion(v, Td=0.5, Ta=1.0, ab=3.0): return v * (Td + Ta) + v * v / (2 * ab)
def check(cards):
    problems = []; prev = None
    for c in cards:
        for k in REQUIRED:
            if not c.get(k) and not (k == 'speed_mps'): problems.append(f"{c['id']}: missing {k}")
        for a in c.get('abort', []):
            if not re.search(r'\d', a): problems.append(f"{c['id']}: abort criterion not measurable: '{a}'")
        if excursion(c['speed_mps']) + 1.0 > c['clear_m']: problems.append(f"{c['id']}: excursion {excursion(c['speed_mps']):.1f} m + 1 m margin exceeds clear distance {c['clear_m']} m")
        if prev and not any(prev['id'] in e for e in c['entry']): problems.append(f"{c['id']}: entry criteria do not require {prev['id']}")
        prev = c
    return problems
for p in check(cards): print("PROBLEM:", p)
assert abs(excursion(3.0) - 6.0) < 1e-9 and abs(excursion(6.0) - 15.0) < 1e-9
assert not any(p.startswith('FT-01') or p.startswith('FT-02') for p in check(cards)) and sum(p.startswith('FT-03') for p in check(cards)) >= 4

Common engineering mistakes

Failure modes

Debugging approach

1. Review the abort thresholds against logged margins from earlier flights.
2. Measure pilot reaction times and the vehicle’s braking distance in a safe test.
3. Check that the baseline ID and manifest are on every card.
4. Review the data returned against the planned recordings.

Practical exercises

1. Write the build-up of test points for a 5-minute GNSS-denied inspection route (at least six steps).
2. Compute the excursion distance for 5 m/s with $T_{d}=0.3\,\mathrm{s}$, $T_{a}=1.2\,\mathrm{s}$ and $a_{b}=4\,\mathrm{m}/\mathrm{s}^{2}$.
3. Write measurable abort criteria for a VIO dropout test.
4. Design the debrief checklist for one flight.

Interview questions

Write an answer in your own words, then reveal the model answer. Nothing is sent anywhere.

References and further reading

Primary papers, official documentation and standards. Each claim above that is specific to a tool or version points here.

• NASA Systems Engineering Handbook (NASA/SP-2007-6105), test and verification
• EASA: Operating a drone (EU rules; check the rules that apply where you fly)
• FAA: Unmanned aircraft systems (US rules; check the rules that apply where you fly)

What’s next