Preparing a field test

1. Engineering problem

Field time is expensive and conditions are not repeatable. A written plan makes sure the test answers a question, stays safe and produces data that can be analysed.

2. When is this needed?

• Before taking the robot out of the lab.
• Before a customer demonstration.
• Before a long run.

3. Selection and design decisions

A plan needs at least: objective; route or area; expected conditions; success criteria (measurable); abort criteria; a named safety officer; the data to record (see the bag strategy guide); and a rollback plan (what happens to the software and the robot afterwards). Add an energy budget: usable runtime with a reserve against the planned run time.

4. Advantages and disadvantages

• Longer tests give more data and more risk and battery use.
• A rehearsal in the lab costs hours and saves field time.
• Fewer variables per test make the results easier to interpret.

5. Requirements and constraints

• Success and abort criteria are measurable.
• The robot has a reserve of energy beyond the plan.
• The software version and configuration are recorded and frozen for the test.

6. Architecture

plan ─▶ rehearsal ─▶ field test (record) ─▶ review ─▶ decisions (fix, repeat, release)

7. Implementation

1. Write the plan with every item (below).
2. Check the energy budget.
3. Rehearse in the lab.
4. Run the test with the safety officer present.
5. Review the data and decide next steps.

REQUIRED = ["objective", "route", "conditions", "success_criteria", "abort_criteria", "safety_officer", "data_to_record", "rollback_plan"]
plan = {"objective": "20 runs of the loading-dock route", "route": "dock A to dock C", "conditions": "daylight, dry", "success_criteria": "goal within 0.25 m, no interventions",
        "abort_criteria": "", "safety_officer": "A. Example", "data_to_record": "bag with /tf, /scan, /odom"}
missing = [k for k in REQUIRED if not plan.get(k)]
print("missing or empty:", missing or "none")
energy_wh, power_w, reserve = 150.0, 60.0, 0.3
runtime_h = energy_wh * (1 - reserve) / power_w
print(f"usable runtime with a {int(reserve * 100)} % reserve: {runtime_h * 60:.0f} min; plan {20 * 6} min of runs ->", "fits" if runtime_h * 60 >= 120 else "needs a battery swap")

Printed (checked by the site’s tests):

missing or empty: ['abort_criteria', 'rollback_plan']
usable runtime with a 30 % reserve: 105 min; plan 120 min of runs -> needs a battery swap

Reading it: the example plan has no abort criteria and no rollback plan, which the checker reports; with 150 Wh of battery, 60 W of average power and a 30 % reserve the usable runtime is 105 minutes against 120 minutes of planned runs, so a battery swap is needed. The values are illustrative.

8. Real examples

• Loading dock trial: plan reviewed by a second engineer; abort when a forklift enters the aisle.

9. Failure scenarios

• No abort criteria, so a failing test continues.
• Changing the software during the test.
• Running out of battery mid-route.

10. Troubleshooting

1. If results are unclear, the criteria were not measurable.
2. If the test was aborted often, review the conditions.
3. If data is missing, review the recording plan.

11. Validation

• The checker prints the lines shown (executed by the site tests).
• Every field test has a written plan with all items before it starts.

12. Engineering checklist

• Objective
• Success criteria
• Abort criteria
• Safety officer
• Data plan
• Energy budget
• Rollback plan

13. Further references

• Practice-based guidance; the plan items are common engineering practice and not drawn from one cited source.

Related on FixTheRobot