M8-T16: Technical risk assessment

Learning objectives

After this lesson, you should be able to:

• Define consistent severity and likelihood scales for a UAV programme.
• Build a risk register with ranking that does not hide rare severe risks.
• Apply the mitigation hierarchy and estimate residual risk.
• Document acceptance with the authority who accepts it.

Conceptual explanation

Established Risk matrices and mitigation hierarchies are standard (for example in MIL-STD-882 and ISO 31000 style processes). The scale values below are examples for a research UAV and must be adapted to your programme, operating category and the regulations that apply.

Severity describes the worst credible consequence; likelihood describes how often it is expected, over a stated exposure (per flight, per flight hour). A common five-by-five scale:

The mitigation hierarchy in order of effectiveness: eliminate the hazard (design it out: a prop guard, a safer site), reduce the likelihood (redundancy, monitors, procedures), reduce the severity (lower speed, frangible parts, a geofence, distance from people), then warnings and training. Procedures and training are the weakest because they depend on people not erring.

Engineering intuition

A matrix can mislead. Multiplying severity by likelihood treats a frequent nuisance and a rare catastrophe as equal if the products match. Rank by severity first, then likelihood: a catastrophic risk with an “improbable” rating still needs a justification for that rating and independent barriers. Also remember that likelihood estimates for new systems are judgement, with large uncertainty; test evidence and field data improve them, and the register should record the basis.

Risk acceptance belongs to someone with authority over the consequence, not to the engineer who is also late. A signed, dated acceptance with the conditions (“accepted for flights below 10 m/s over the test field only”) is what turns an engineering finding into an operational limit.

Mathematical formulation

Quantified risk for a hazard with per-flight probability $p$ and a consequence measure $c$ (expected loss) over $n$ flights is

For example a per-flight event probability of $10^{-3}$ is $1-(0.999)^{200}\approx 18$ % over a 200-flight campaign: “improbable per flight” is not improbable over a campaign. The evidence behind $p$ improves with data: the upper confidence bound after $k$ events in $n$ flights comes from the Clopper-Pearson interval, and for $k=0$ the rule of three gives $p_{\text{up}}\approx 3/n$ (95 %), so 300 incident-free flights only support a claim of $p$ below $10^{-2}$, not $10^{-4}$. Likelihood classes for a hazard that depends on several independent barrier failures multiply: $P=P_{\text{initiating}}\cdot \Pi (1-e_{j})$ where $e_{j}$ is the effectiveness of barrier $j$, which is why the barrier effectiveness must itself be verified (and why common cause between barriers erodes the product).

$p$

event probability per flight.

$n$

number of flights.

$c$

consequence measure.

$e_{j}$

effectiveness (probability of success) of barrier $j$.

Architecture and data flow

FMEA and fault-tree outputs, incident reports and test findings → risk register (ID, hazard, cause, consequence, severity, likelihood with basis, mitigations, owner, residual severity and likelihood, status, acceptance record) in version control → ranking script → review board → decisions (accept, mitigate further, stop) recorded and linked to requirements and tests. Residual risks flow into the operational limitations and the briefing of every flight.

Real-world UAV applications

• Prioritising fixes before the first outdoor GNSS-denied flight.
• Deciding which risks to accept for a research campaign.
• Setting operational limits from residual risk.
• Reporting risk status to a review board.

Implementation example

This runs as shown. A risk register with severity-first ranking, residual risk after mitigations, campaign-level probabilities from the per-flight value, and the evidence check that flags likelihood claims that the number of flights cannot support. The entries are illustrative.

import math
LIK = {'A': 1e-1, 'B': 1e-2, 'C': 1e-3, 'D': 1e-4, 'E': 1e-5}                              # representative per-flight values for each class (example scale)
register = [
 dict(id='RK-1', hazard='VIO lost, undetected drift into a wall', sev=3, lik='C', mit='VO health monitor + fallback + pilot takeover', res_lik='D', basis='SIL: 100 % detection in 45 runs', flights_evidence=45),
 dict(id='RK-2', hazard='companion crash during offboard flight', sev=3, lik='B', mit='offboard loss failsafe: hold, tested', res_lik='D', basis='HIL + 10 flight tests', flights_evidence=10),
 dict(id='RK-3', hazard='vehicle strikes a person during a GNSS-denied flight', sev=5, lik='D', mit='site closed to persons; geofence; low speed', res_lik='E', basis='procedure and geofence test', flights_evidence=12),
 dict(id='RK-4', hazard='battery depletion before return', sev=3, lik='C', mit='energy-based return trigger', res_lik='D', basis='bench model + 6 flights', flights_evidence=6),
 dict(id='RK-5', hazard='mission delayed by calibration error', sev=1, lik='A', mit='pre-flight calibration check', res_lik='C', basis='field experience', flights_evidence=30)]
rank = sorted(register, key=lambda r: (-r['sev'], LIK[r['lik']] * -1))                    # severity first, then likelihood
print(f"{'id':>5} sev lik -> res   per-flight p (residual)   P(event in 200 flights)   evidence check")
for r in rank:
    p = LIK[r['res_lik']]; camp = 1 - (1 - p) ** 200; supported = 3 / r['flights_evidence'] <= p          # rule of three: can n flights support p?
    r['supported'] = supported
    print(f"{r['id']:>5} {r['sev']:>3} {r['lik']:>3} -> {r['res_lik']}      {p:8.0e}                  {camp:6.1%}                  {'OK' if supported else 'NOT SUPPORTED by ' + str(r['flights_evidence']) + ' flights (needs ' + str(math.ceil(3 / p)) + ')'}")
assert rank[0]['id'] == 'RK-3' and rank[-1]['id'] == 'RK-5'
unsupported = [r['id'] for r in rank if not r['supported']]; print("residual likelihood claims not supported by the evidence:", unsupported)
assert 'RK-3' in unsupported and abs((1 - (1 - 1e-3) ** 200) - 0.1814) < 1e-3

Common engineering mistakes

Failure modes

Debugging approach

1. Check each likelihood against the evidence in flights, hours and tests.
2. Check that each mitigation has a verification test in the traceability matrix.
3. Review the highest-severity rows with an independent reviewer.
4. Recompute the campaign-level probabilities for the planned number of flights.

Practical exercises

1. Build a register of eight risks for your GNSS-denied capstone and rank them.
2. Compute the number of incident-free flights needed to support a per-flight probability of $10^{-3}$ at 95 % confidence.
3. Choose three mitigations for “vehicle strikes a person” and place them on the hierarchy.
4. Write the acceptance statement for a residual risk with conditions.

Interview questions

Write an answer in your own words, then reveal the model answer. Nothing is sent anywhere.

References and further reading

Primary papers, official documentation and standards. Each claim above that is specific to a tool or version points here.

• ISO 31000: Risk management
• MIL-STD-882E: System Safety
• NASA Systems Engineering Handbook (NASA/SP-2007-6105), technical risk management

What’s next