FixTheRobot Explore Robotics / Systems Engineering / Design & Selection / Module 01

Design & Selection · Module 01

Engineering trade-off analysis

A trade study is only as good as its weights. The sensitivity check shows whether the answer depends on an argument you have not settled.

  • Intermediate
  • Module 01: Robotics System Architecture & Requirements
  • Written for ROS 2 Jazzy where ROS 2 is used
  • Sources checked 2026-11-05

Assumptions: A practical template based on the NASA Systems Engineering Handbook; not a standard. All numbers in the example are illustrative and the code is executed by the site’s tests.

1. Engineering problem

Choosing a sensor, computer or architecture involves competing criteria (cost, range, compute, robustness). A weighted matrix makes the reasoning explicit and repeatable. Its weakness is that the result depends on the weights, which are a judgement. The sensitivity analysis shows how robust the choice is.

2. When is this needed?

  • Choosing between two or more credible options.
  • Justifying a choice to a customer or a review.
  • When a team disagrees: the matrix shows which disagreement matters.

3. Selection and design decisions

The NASA handbook covers trade studies and decision analysis (Section 6.8) and recommends documenting criteria, weights, rationale and the result.

StepWhat to do
CriteriaPick criteria from the requirements; keep them independent
WeightsSum to 1; agree them before scoring the options
ScoresScore each option per criterion on one scale (here 1–5) from data (datasheets, tests)
TotalWeighted sum per option
SensitivityVary each weight; find where the winner changes

4. Advantages and disadvantages

  • Simple scores are quick and hide detail; measured values with utility curves are more accurate and take longer.
  • Hard requirements are pass/fail gates, applied before the matrix, not weights inside it.
  • Cost is often better shown separately from a score to avoid hiding it.

5. Requirements and constraints

  • Scores come from datasheets or tests, with a source, never from preference alone.
  • Weights are agreed before options are scored.
  • The analysis is saved with the decision and its date.

6. Architecture

gates (must-have) ─▶ remaining options ─▶ weighted matrix ─▶ winner
                                              └▶ sensitivity: does the winner change when a weight moves?

7. Implementation

  1. Apply the must-have requirements as gates and drop options that fail them.
  2. Choose criteria and weights; record who agreed them.
  3. Score each option from sources.
  4. Compute the totals.
  5. Vary the most disputed weight, rescaling the others so the sum stays 1, and record where the winner changes.
criteria = {"cost": 0.3, "range": 0.3, "compute": 0.2, "robustness": 0.2}
options = {"A": {"cost": 4, "range": 2, "compute": 5, "robustness": 3},
           "B": {"cost": 2, "range": 5, "compute": 3, "robustness": 4},
           "C": {"cost": 3, "range": 4, "compute": 4, "robustness": 2}}
def scores(w):
    return {o: round(sum(w[c] * v[c] for c in w), 3) for o, v in options.items()}
def with_weight(c, x):                                  # change one weight, rescale the others so the total stays 1
    rest = 1 - criteria[c]
    return {k: (x if k == c else criteria[k] * (1 - x) / rest) for k in criteria}
base = scores(criteria)
print(base, "->", max(base, key=base.get))
spans, start, cur = [], 0.0, None
for step in range(0, 101):
    x = step / 100
    s = scores(with_weight("range", x)); w = max(s, key=s.get)
    if w != cur:
        if cur is not None:
            spans.append((start, round(x - 0.01, 2), cur))
        start, cur = x, w
spans.append((start, 1.0, cur))
for a, b, w in spans:
    print(f"range weight {a:.2f} to {b:.2f}: {w} wins")

Printed (checked by the site’s tests):

{'A': 3.4, 'B': 3.5, 'C': 3.3} -> B
range weight 0.00 to 0.27: A wins
range weight 0.28 to 1.00: B wins

Reading it: with the base weights B wins narrowly (3.5 vs 3.4). If range is weighted below about 0.27, A wins instead. So the decision hinges on how much range matters: settle that with the customer rather than trusting the 0.1 gap. All scores here are made-up illustration values.

8. Real examples

  • LiDAR choice: gate on IP rating and interface, then weigh range, vertical field of view, price and driver maturity; see the LiDAR guide for product facts.

9. Failure scenarios

  • Weights chosen after seeing which option they favour.
  • Scores that mix scales (1–5 for one criterion, 1–10 for another).
  • A narrow margin treated as decisive.
  • Hard requirements entered as weights, so a failing option still wins.

10. Troubleshooting

  1. If the matrix disagrees with intuition, find the criterion that drives the difference and check its score and weight.
  2. If the sensitivity shows a flip, resolve the weight with data or the customer.
  3. If two options tie, add a criterion that reflects a real requirement or choose by risk.

11. Validation

  • The example prints the totals and spans above (checked by the site tests).
  • Each score has a source.
  • The decision record lists options, criteria, weights, scores, sensitivity and the decision.

12. Engineering checklist

13. Further references

Related on FixTheRobot