Skip to the lesson
FixTheRobot Explore Robotics / Navigation / M10 Capstone: The Shift / M10-T2

Navigation · Theory · M10

M10-T2 — Engineering Trade-offs and Safety Cases

You will never maximise everything. A design is a choice, and an argument is how you defend it, including the parts you cannot defend.

  • Lesson 2 of 2
  • Module 10: Capstone: The Shift
  • Theory
  • About 10 minutes to read

Speed, safety, battery and cost pull against each other. Scout is faster with no zone limits and safer with all of them, and neither extreme is the answer. This lesson is about choosing deliberately, writing down why, and arguing that the choice is acceptable with evidence a reviewer can check. It also teaches how to say plainly what remains unsafe or unknown.

Learning objectives

After this lesson, you should be able to:

  • Compare designs on several criteria and identify the ones on the Pareto front.
  • Structure a safety argument as claims, arguments and evidence, with context and assumptions.
  • Build a hazard table with cause, severity, mitigation, residual risk and the evidence for each.
  • State residual risks and the limits of the evidence, including common-cause failures.

The intuition

Every time you pass a pharmacy counter you rely on a safety case you have never seen: someone argued that the pills in the packet are the pills on the label, and showed the evidence. A safety case is that kind of argument for a system. It is not a certificate and not a pile of test results. It is a chain that starts at a claim (“this robot is acceptably safe here”), breaks it into smaller claims by an explained strategy, and ends at evidence that a stranger can inspect.

A trade-off is the other half. Before you can argue that a design is acceptable you have to know what you gave up for it. If you can say “we cap the atrium at 0.5 m/s, costing 50 s per delivery, because the sonar sees glass only at 0.8 m”, the argument is half written.

Scope of this track. The lesson teaches the concepts. A real robot in a real hospital needs a safety process that follows the applicable standards, with qualified people, independent review and certification where required. Nothing here is a substitute.

A safety argument as a tree of claimsA top claim that Scout is acceptably safe in a stated context. Below it, an argument by hazard splits it into three sub-claims about glass, people and the door, each resting on evidence such as a stop test, simulated shifts and a scripted door cycle. A side box lists the assumptions.G1 Scout is acceptably safe onwards 3 and 4, day shift (context C1) S1 argue over each identified hazard G2 No glasscontactG3 No contactwith peopleG4 Door cannottrap or strand Sn1evidenceStop testat 0.5 m/s: 0.31 mSn2evidenceHidden shifts,bench latency dataSn3evidenceScripted doorcycles, 30 s timeout A1 sonar reaches 0.8 mA2 people walk, not runA3 map matches building Every claim has evidence ora sub-claim; every assumptionis written down.
A safety argument as a tree: a top claim under a stated context, a strategy that splits it by hazard, sub-claims, evidence, and the assumptions that everything rests on.

The mathematics

Trade-offs and the Pareto front

Score each design on every criterion that matters. Design X dominates design Y if X is at least as good on every criterion and better on one. The designs that no other design dominates form the Pareto front; choosing among them is a decision about values, not arithmetic.

DesignPolicyCycle timeEnergy per shiftWorst contact speed
A1.2 m/s everywhere239 s92 Wh0.86 m/s (glass)
BZone limits: 0.6 junction, 0.5 atrium290 s115 Wh0 (stops short)
C0.5 m/s everywhere410 s168 Wh0 (stops short)

A is not acceptable (it hits the glass). C is dominated by B (same safety, slower and more energy) and, with only 160 Wh usable, it also fails the energy requirement. B is the only design left, and it is on the front because it buys safety where the hazard is.

Claims, arguments and evidence

  • Claim (goal): a statement that can be true or false, in a stated context: “Scout does not contact glass partitions on wards 3 and 4 at speeds up to the zone limit.”
  • Argument (strategy): why the sub-claims add up to the claim: “argue over every hazard in the hazard table”, or “argue over each layer of protection”.
  • Evidence (solution): a test, an analysis or a log that supports a claim, with its conditions: “stop tests at 0.5 m/s, 30 runs, glass at 0°, 15° and 30° incidence”.
  • Assumptions and justifications: what must hold for the argument to be valid, written so that somebody can check them.

The graphical form used in industry is the Goal Structuring Notation. The same idea can be written as a table or as prose; what matters is that each claim points to something that supports it.

Hazard analysis

List what can go wrong, rank it, say what reduces it, and say what is left. A simple scale is severity S from 1 (nuisance) to 4 (serious injury) and likelihood L from 1 (remote) to 5 (frequent) before and after controls.

HazardCauseS × L beforeControlS × L afterEvidence
Contact with glassLaser sees through it3 × 4 = 12Zone limit 0.5 m/s, sonar, map flag3 × 1 = 3Stop test, simulated shifts
Contact with a person at a nicheOccluded, short warning4 × 3 = 12Junction limit, safe-speed rule4 × 1 = 4Bench latency, hidden shifts
Door closes on robotTimer cycle2 × 3 = 6Door sensor holds, route timeout2 × 1 = 2Scripted door cycles
Stuck in a corridorWaiting for the unclearable2 × 4 = 8Cost-based recovery, help in 40 s2 × 2 = 4Recovery table review
Late urgent deliveryPoor dispatch, long waits3 × 3 = 9Deadline-aware ordering3 × 2 = 6Hidden-shift deadlines
Battery flat in the corridorLong detours and waits2 × 3 = 6Energy budget, return rule2 × 1 = 2Shift energy logs

Layers and common causes

If hazards need several failures at once, the chance is the product of their chances, if they are independent. A shared cause (one computer, one power supply, one wrong map) breaks the independence. Add the common-cause term to the argument rather than hoping.

Layers of protection with a common cause
rate=f×P(layer 1 fails)×P(layer 2 fails)+f×P(common cause)\text{rate}=f\times P(\text{layer}\,1\,\text{fails})\times P(\text{layer}\,2\,\text{fails})+f\times P\text{(common cause)}
f
how often the hazardous situation arises, per delivery.
P(layer k fails)
probability, on demand, that protection k does not work; assumed independent.
P(common cause)
probability that one fault defeats all layers at once.

A worked example

Choosing among designs. At the glass the sonar vouches for 0.8 m. Design A goes at 1.2 m/s with tr = 0.3 s and a = 0.8 m/s²:

Glass contact
distance left when braking starts=0.8−1.2×0.3=0.44 m\text{distance left when braking starts}=0.8-1.2\times 0.3=0.44\,\mathrm{m}
speed at the pane=1.22−2×0.8×0.44=0.86 m/s  contact\text{speed at the pane}=\sqrt{1.2^{2}-2\times 0.8\times 0.44}=0.86\,\mathrm{m}/\mathrm{s}\;\mathrm{contact}
B at 0.5 m/s: stops in 0.15+0.156=0.31 m, 0.49 m before the paneB\,\text{at}\,0.5\,\mathrm{m}/\mathrm{s}:\,\text{stops in}\,0.15+0.156=0.31\,\mathrm{m},\,0.49\,\mathrm{m}\,\text{before the pane}

Across 40 deliveries a day, design A takes 34 minutes less than B (2.66 h against 3.22 h), but its worst case is a contact at 0.86 m/s. The saved time is not worth that, and the argument is written in one sentence with numbers.

How strong is the layered argument? A visitor steps in front of the robot on 0.2 of deliveries. If the laser misses the person with probability 10-2 and the brakes fail with probability 10-3, independently:

Rates with and without a common cause
rate=0.2×10−2×10−3=2×10−6 per delivery(1 in 500 000)\text{rate}=0.2\times 10^{-2}\times 10^{-3}=2\times 10^{-6}\,\text{per delivery}(1\,\text{in}\,500\,000)
with P(common cause)=10−4(one software hang stops both):\text{with}\,P\text{(common cause)}=10^{-4}\text{(one software hang stops both)}:
rate=2×10−6+0.2×10−4=2.2×10−5(1 in 45 000)\text{rate}=2\times 10^{-6}+0.2\times 10^{-4}=2.2\times 10^{-5}(1\,\text{in}\,45\,000)

The common cause dominates by a factor of ten. That is why a safety layer must be independent of the planner, as the architecture of lesson M10-T1 requires.

What the evidence supports. Suppose 10 000 simulated shifts show no contact. Under the simulator’s assumptions the 95 per cent upper bound is 3 / 10 000 = 3 × 10-4 per shift. The field rate also depends on what the simulator leaves out (glass at an angle, a person running, a map that is out of date), so the claim made from simulation is “no contact in the modelled world”, with the unmodelled part argued separately or stated as residual risk.

Engineering implications and limitations

Write the claim narrowly

“Safe” is not a claim; “does not contact glass partitions of the types in the survey, at speeds up to the zone limit, in daylight, on wards 3 and 4” is. Narrow claims can be supported and checked, and what falls outside them is visible.

Say what each piece of evidence cannot show

Each item of evidence needs its conditions and its limits. A stop test at 0° incidence says nothing about glass at 45°. A hidden-shift simulation says nothing about a sensor model it does not contain.

State residual risk plainly

After the controls, some risk remains. List it, say who accepts it and under what operating limits, and say what would trigger a review: a new building wing, a firmware update, a near miss.

Keep the argument alive

The case is a living document tied to a configuration and a map version. Change a parameter, the sensor or the building, and the evidence may no longer apply. Re-check before you re-deploy.

Independence and review

The people who built the robot are the worst reviewers of its safety argument. Plan an independent review, and for products used around the public a certification path under the relevant standards (for example ISO 3691-4 for driverless industrial vehicles and ISO 13482 for personal care robots, by title; which applies depends on the product and the jurisdiction).

Common misconceptions

“A safety case is a test report.”

A report is evidence. A case adds the claim, the reasoning that links the evidence to the claim, the context, and the assumptions that make it valid.

“If every hazard has a control, the risk is zero.”

Controls reduce risk, never to zero. The honest output is a residual risk and the limits of use that go with it.

“More evidence of the same kind is always better.”

Ten thousand runs in one simulator add confidence only about what that simulator models. Different kinds of evidence cover different gaps.

“The track’s labs show Scout is safe.”

They show the functions behave as intended on hidden simulated shifts. They teach concepts and are not a safety case or a certification, and a real robot needs its own process.

Engineering takeaways

  1. A design is a choice among trade-offs; the Pareto front shows which choices are not dominated.
  2. A safety case links claims to evidence through an argument, within a stated context and assumptions.
  3. A hazard table says what can go wrong, how it is controlled and what risk remains.
  4. Common-cause failures can dominate an otherwise strong layered argument.
  5. State residual risk and limits plainly; this track does not replace a certified safety process.

References and further reading

Safety-case literature and standards, cited by title; check the current editions before using them.

  • T. P. Kelly, “Arguing Safety: A Systematic Approach to Managing Safety Cases”, DPhil thesis, University of York, 1998.
  • T. Kelly and R. Weaver, “The Goal Structuring Notation: A Safety Argument Notation”, Proceedings of the Dependable Systems and Networks Workshop on Assurance Cases, 2004.
  • ISO 12100, “Safety of machinery: General principles for design: Risk assessment and risk reduction”.
  • ISO 3691-4, “Industrial trucks: Safety requirements and verification: Part 4: Driverless industrial trucks and their systems”.
  • ISO 13482, “Robots and robotic devices: Safety requirements for personal care robots”.

Knowledge check

Three conceptual questions. Write an answer, then reveal the explanation. Your answers stay in your browser.

Question 1

Design X takes 300 s per delivery and uses 100 Wh; design Y takes 280 s and uses 110 Wh; design Z takes 320 s and uses 120 Wh. Which are on the Pareto front for time and energy?

Explanation

X and Y. Neither is at least as good as the other on both criteria. Z is dominated by X (X is faster and uses less energy).

Question 2

What are the three kinds of element in a basic safety argument, and what does an assumption do?

Explanation

Claims, arguments (strategies) and evidence. An assumption states a condition that must hold for the argument to be valid, such as “the map matches the building”, so that it can be checked and monitored.

Question 3

Two independent layers fail with probabilities 10⁻² and 10⁻³ on a hazard that arises 0.1 times per delivery. A common cause defeats both with probability 10⁻⁴. What is the rate, and which term dominates?

Explanation

Layers: 0.1 × 10⁻² × 10⁻³ = 10⁻⁶. Common cause: 0.1 × 10⁻⁴ = 10⁻⁵. The total is 1.1 × 10⁻⁵ per delivery, dominated by the common cause by a factor of ten.

What’s next

This is the end of the Navigation track. Go back to any lab with a failing shift and read the report again: you now have the vocabulary to say what went wrong and what you would change.

Nothing below is required to finish this lesson.