M7-T18: Requirements traceability

Learning objectives

After this lesson, you should be able to:

• Build a bidirectional traceability matrix from needs and hazards to results.
• Compute coverage and identify gaps, orphans and stale results.
• Use traceability to scope the impact of a change.
• Automate the checks in the repository.

Conceptual explanation

Established Traceability is standard systems and safety-engineering practice and is required by many certification and assurance standards. Which links are mandatory depends on the standard that applies; this lesson describes the common core.

Four relations, each in both directions:

• Need or hazard → requirement: every requirement exists for a reason (a stakeholder need, a mission need, a mitigation of an identified hazard).
• Requirement → design element: every requirement is allocated to something that implements it (a module, a parameter, a procedure).
• Requirement → verification: every requirement has at least one test, analysis, inspection or demonstration with a pass criterion.
• Verification → result: every verification has a recorded result tied to a software and hardware version.

Defects to look for: gaps (a requirement with no design or no verification), orphans (a design element or a test case tracing to no requirement: either a missing requirement or unnecessary work), stale results (the result is for an older version than the one being flown), and weak links (a test whose criterion does not actually demonstrate the requirement).

Engineering intuition

The matrix is useful because it can be queried both ways. Forward: “for this hazard, which requirements mitigate it, and have they all been verified on this release?” Backward: “this test failed; which requirement is at risk, and which hazard does that touch?”. During a design change the forward query is the impact analysis: change the odometry timeout, and the matrix lists the requirements, designs and tests it touches.

The discipline that pays most is writing requirements so that each has a single, testable criterion, with an ID. A requirement like “navigate accurately” cannot trace to a test; “horizontal position error shall not exceed 2.0 m (95th percentile) over a 5-minute GNSS-denied segment” can.

Mathematical formulation

Simple coverage metrics, each between 0 and 1 and required to be 1 (or explained) at a review:

Orphan sets are $O_{\text{test}}$ = {$t$ : $\operatorname{trace} (t)=\varnothing$} and $O_{\text{design}}$ = {$e$ : $\operatorname{trace} (e)=\varnothing$}. Change impact for a changed requirement $r$ is the transitive closure of its links: $\mathrm{impact}(r)=\mathrm{design}(r)\cup \mathrm{verif}(r)$ ∪ {requirements sharing the design elements}. For a verification result to count, its version stamp $V_{\text{res}}$ must equal the current baseline $V_{\text{now}}$ (or be covered by an impact analysis that argues the change does not affect it), otherwise the result is stale.

$R$

set of requirements.

design(r), verif(r)

design elements and verifications linked to $r$.

$V_{\text{res}}$, $V_{\text{now}}$

version of the result and of the current baseline.

Architecture and data flow

Requirements, hazards, design elements and tests live as structured records (YAML, a requirements tool, or issues) with stable IDs and explicit link fields. Test code carries the requirement IDs it verifies (for example in a marker or docstring) and test runs write results with the baseline version. A script in continuous integration builds the matrix, computes coverage, and fails the build on gaps, orphans and stale results beyond an allowed list.

Real-world UAV applications

• Showing which requirements a flight-test campaign verified.
• Impact analysis after changing a failsafe timeout.
• Evidence for a safety review or an airworthiness assessment.
• Planning regression tests after a release.

Implementation example

This runs as shown. A traceability matrix as data and a checker computing coverage, gaps, orphans and stale results, including a change-impact query.

needs = {'H-01': 'loss of GNSS leads to unbounded drift', 'N-02': 'mission completes in GNSS-denied area'}
reqs = {'R-10': dict(text='position error <= 2 m (p95) for 5 min without GNSS', traces_to=['N-02', 'H-01'], design=['vio', 'ekf2-ev'], verif=['T-100', 'T-101']),
        'R-11': dict(text='on VIO loss, land or hold within 2 s', traces_to=['H-01'], design=['failsafe-vio'], verif=['T-110']),
        'R-12': dict(text='odometry latency <= 100 ms (p99)', traces_to=['N-02'], design=['vio', 'timestamps'], verif=[]),                 # gap: no verification
        'R-13': dict(text='log all fusion decisions', traces_to=[], design=['logger'], verif=['T-130'])}                                    # orphan requirement: no need
tests = {'T-100': dict(result='pass', version='v1.4'), 'T-101': dict(result='pass', version='v1.3'), 'T-110': dict(result='fail', version='v1.4'),
         'T-130': dict(result='pass', version='v1.4'), 'T-999': dict(result='pass', version='v1.4')}                                            # T-999 verifies nothing
now = 'v1.4'
traced = {t for r in reqs.values() for t in r['verif']}
gaps = [r for r, v in reqs.items() if not v['verif'] or not v['design']]
orphan_reqs = [r for r, v in reqs.items() if not v['traces_to']]
orphan_tests = [t for t in tests if t not in traced]
stale = [t for t, v in tests.items() if v['version'] != now]
passing = [r for r, v in reqs.items() if v['verif'] and all(tests[t]['result'] == 'pass' and tests[t]['version'] == now for t in v['verif'])]
print(f"coverage: verification {sum(1 for v in reqs.values() if v['verif']) / len(reqs):.0%}, passing on {now}: {len(passing) / len(reqs):.0%}")
print("gaps:", gaps, " orphan requirements:", orphan_reqs, " orphan tests:", orphan_tests, " stale results:", stale)
impact = lambda d: sorted(r for r, v in reqs.items() if d in v['design'])
print("change to 'vio' impacts requirements:", impact('vio'), "-> tests:", sorted({t for r in impact('vio') for t in reqs[r]['verif']}))
assert gaps == ['R-12'] and orphan_reqs == ['R-13'] and orphan_tests == ['T-999'] and stale == ['T-101'] and passing == ['R-13']
assert impact('vio') == ['R-10', 'R-12']

Common engineering mistakes

Failure modes

Debugging approach

1. Run the checker in CI and look at the list of gaps before every review.
2. Query from a hazard to see the complete chain.
3. Check version stamps of the evidence for each requirement.
4. Review a sample of tests to confirm their criteria match the requirement text.

Practical exercises

1. Build the matrix for the five GNSS-denied requirements of your capstone and run the checker.
2. Add a check that every hazard has at least one requirement and every requirement of a hazard has a passing test.
3. Compute the impact of changing the VIO timeout parameter.
4. Rewrite two vague requirements as measurable ones with a test.

Interview questions

Write an answer in your own words, then reveal the model answer. Nothing is sent anywhere.

References and further reading

Primary papers, official documentation and standards. Each claim above that is specific to a tool or version points here.

• NASA Systems Engineering Handbook (NASA/SP-2007-6105)
• INCOSE Systems Engineering Handbook
• SAE ARP4754A: Guidelines for development of civil aircraft and systems

What’s next