FixTheRobot Explore Robotics / Systems Engineering / Engineering Guides / Module 16

Engineering Guides · Module 16

Camera-to-robot (hand–eye) calibration

Hand-eye calibration finds the fixed transform between the camera and the robot. Collect varied poses, then validate on points you did not use.

  • Intermediate
  • Module 16: Manipulation & Robotic Arm Integration
  • Written for ROS 2 Jazzy where ROS 2 is used
  • Sources checked 2026-11-06

Assumptions: Numbers in the examples are illustrative and the code is executed by the site’s tests. Facts taken from a cited source are marked; other statements are general engineering knowledge and must be confirmed on the datasheet of the part you use.

1. Engineering problem

For a robot to pick what a camera sees, the camera’s measurements must be expressed in the robot frame. The transform between them is found by hand-eye calibration: moving the arm to several poses while the camera views a fixed target.

2. When is this needed?

  • After mounting a camera on or near an arm.
  • When picks are consistently offset.
  • After a camera or mount change.

3. Selection and design decisions

The MoveIt Calibration package (tutorial read 2026-11-06) provides plugins and a graphical interface for hand-eye calibration for cameras rigidly mounted in the robot base frame (eye-to-hand) and for cameras mounted on the end effector (eye-in-hand); it needs a camera publishing images and a sensor_msgs/CameraInfo topic with good intrinsic calibration and an accurate frame, an ArUco-style target (the tutorial’s defaults use DICT_5X5_250 markers) that stays stationary in the robot base frame, and a sequence of 5 or more poses. The target must be flat. Calibrate the intrinsics first (the tutorial points to the camera_calibration package).

SetupCamera mountedFixed targetResult
Eye-in-handOn the end effectorIn the work spaceCamera pose relative to the end effector
Eye-to-handFixed near the robotOn the end effectorCamera pose relative to the robot base

4. Advantages and disadvantages

  • More poses improve accuracy and cost time; vary rotation as well as position.
  • A larger target is detected better and needs space.
  • Calibrating at the working distance reduces extrapolation error.

5. Requirements and constraints

  • Intrinsics calibrated first.
  • The target is flat, rigid and stationary during collection.
  • Poses cover rotation about every axis.

6. Architecture

arm poses (robot kinematics) + target detections (camera) ─▶ hand-eye solver ─▶ camera↔flange or camera↔base transform ─▶ validate on touch points

7. Implementation

  1. Calibrate intrinsics.
  2. Print the target with known parameters and mount it flat.
  3. Collect at least five (better ten or more) poses with varied rotation.
  4. Compute the calibration and publish the transform.
  5. Validate by touching or placing at held-out checkpoints (below).
import numpy as np
def diversity(rotations_deg):
    axes = np.array(rotations_deg, dtype=float)
    spans = axes.max(axis=0) - axes.min(axis=0)
    return spans
def rms(errors_mm):
    return float(np.sqrt(np.mean(np.square(errors_mm))))
poses_rpy = [(0, 0, 0), (10, -5, 20), (-15, 10, -25), (20, 15, 5), (-5, -20, 30), (12, 8, -10)]       # end-effector roll, pitch, yaw in degrees for each calibration pose
span = diversity(poses_rpy)
print("number of poses:", len(poses_rpy), "(the MoveIt tutorial asks for 5 or more)")
print("rotation span per axis (deg):", span.tolist(), "-> " + ("diverse enough" if (span >= 30).all() else "add rotation about the weak axis"))
errs = [1.2, 0.8, 2.1, 1.5, 0.9, 1.7]                                                                 # position error in mm at touched checkpoints not used for the fit
print(f"validation: RMS position error {rms(errs):.2f} mm over {len(errs)} held-out checkpoints, worst {max(errs):.1f} mm")
print("requirement 2 mm:", "pass" if rms(errs) <= 2.0 and max(errs) <= 3.0 else "fail")

Printed (checked by the site’s tests):

number of poses: 6 (the MoveIt tutorial asks for 5 or more)
rotation span per axis (deg): [35.0, 35.0, 55.0] -> diverse enough
validation: RMS position error 1.44 mm over 6 held-out checkpoints, worst 2.1 mm
requirement 2 mm: pass

Reading it: the six example poses span 35°, 35° and 55° about the three axes (the check requires at least 30° on each), and the validation on six held-out checkpoints gives an RMS position error of 1.44 mm and a worst case of 2.1 mm, passing an example requirement of 2 mm. The numbers are illustrative.

8. Real examples

  • Bin picking cell: eye-to-hand calibration with a target on the gripper; validation by touching a marked plate.

9. Failure scenarios

  • Intrinsics skipped, so errors grow toward the image edges.
  • Poses that differ only in position, so rotation is poorly constrained.
  • Validating on the same poses used for the fit.

10. Troubleshooting

  1. If error is a constant offset, check the target size and the frames.
  2. If error grows with distance from the calibration poses, add poses there.
  3. Check that the target did not move during the collection.

11. Validation

  • The checker prints the lines shown (executed by the site tests).
  • Held-out checkpoints are reached within the requirement at several places in the work space.

12. Engineering checklist

13. Further references

  • MoveIt hand-eye calibration tutorial: moveit.picknik.ai, read 2026-11-06 (the tutorial text still contains ROS 1-era build commands; its calibration procedure is what is used here).

Related on FixTheRobot