Robot hand-eye calibration: how nine-point alignment works

A vision-guided robot is only as good as the transform that connects image pixels to robot coordinates. Robot hand-eye calibration builds that transform, and the most common version is the nine-point method: move the robot (or the part) through a grid of known positions while the camera records where a reference feature lands in the image. This article explains what the solver actually does, why pose spread and rotation matter, where the accuracy goes, and what to check in the camera, lens and lighting so the calibration stays valid for months. It is for anyone speccing a vision-guided pick, place or inspection cell.

What the calibration solves for

The camera lives in its own world: pixel columns and rows. The robot lives in millimetres on its base frame. Between them sit several rigid transforms: camera to its bracket, bracket to the robot flange or the cell frame, and (for a tilted camera) a perspective map from the part plane to the sensor.

For a camera that looks straight down at a fixed plane, the 2D-to-3D transform reduces to a scale, a rotation, and a translation:

where p_pixel is the feature position in pixels, R a 2D rotation, s the scale in mm per pixel, and t the offset. Nine point pairs give eighteen equations for six unknowns, so the solver has redundancy to detect bad poses and report a residual.

If the camera is tilted, perspective enters. Points closer to the camera compress differently than points farther away, and a pure scale-and-rotate fit stops working. Either square the camera to the part plane (the simplest fix, and usually the right one) or add a perspective/homography step to the calibration. The planar transform also assumes the part always sits in the same plane; if part height varies, height error scales directly into position error, so calibrate at the actual working height.

How a nine-point session runs

1. Fix a calibration target at the working plane: a printed dot grid or checkerboard on a machined flat, or measured holes in a fixture.
2. Set the final production lens, focus, aperture and lighting before you capture anything. Changing focus afterwards moves the scale.
3. Move the robot so the gripper or a pointer touches a reference feature at nine or more positions spread across the field of view, recording the robot pose and the pixel position of the same feature each time.
4. Include rotation. If the tool rotates between picks, capture at least a few poses with the part or target rotated 45 to 90 degrees so the solver sees the rotation the cell will actually use.
5. Let the software solve the transform and read the residual. Re-project the robot points back into the image and check the error at the corners, not only in the centre.
6. Verify with a physical test: command the robot to a pixel position and measure the miss with a gauge or a marked part.

Poses clustered in the middle of the image are the most common mistake. The transform interpolates well where you sampled and extrapolates badly where you did not, and the corners are exactly where parts drift during production.

Camera and lens choices that keep the transform stable

The calibration inherits every optical flaw in the imaging chain. Two properties matter most: geometric distortion and mechanical stability.

Distortion bends straight lines, so a linear pixel-to-mm map cannot be exact away from the image centre. A 35 mm lens designed for machine vision keeps this manageable over a 2/3" sensor. The LS-FFL-0679 covers 2/3" sensors with a C-mount and is rated for 6-10 MP, which gives the feature detector clean, sharp dots to centroid.

For the camera, a global shutter avoids the smear that rolling shutter adds when anything moves during capture. The MV-CU020-90GC is a 2.5 MP GigE camera with a global shutter, a 1/1.7" sensor and a C-mount, which suits a fixed station where a few hundred thousand pixels per part are plenty. When you need finer pixel scale over the same field, a 12 MP sensor halves the mm-per-pixel figure and halves the position error contributed by feature detection. The LS-CMR-0166 delivers 12 MP at 30 fps over USB3 with a 4000 × 3000 resolution on a 1/1.7" sensor.

Worked example: pixel scale and expected accuracy

Suppose the field of view must cover a 120 mm wide pick area, and the camera is the 12 MP model above with 4000 pixels across. The scale estimate is:

If the feature detector and the robot pose each contribute about one pixel of uncertainty, and the calibration residual comes in at two pixels, the expected repeatability at the part is roughly:

That is the accuracy the cell can promise before mechanical stack-ups in the gripper. If the application needs 0.05 mm, shrink the field of view, raise the resolution, or improve the optical geometry — the calibration cannot create precision the imaging chain does not have. The lens focal length calculator turns a required field of view and working distance into a focal length, and the camera resolution calculator checks the pixel scale you will get.

Lighting and triggering for repeatable captures

Calibration images and production images must come from the same optical conditions. Drift in light output shifts feature centroids; a centroid shift of a tenth of a pixel across a 120 mm field is 0.003 mm, which is fine, but a change of lighting geometry can shift features by several pixels and destroy a 0.1 mm calibration.

Drive the lights from a light controller with fixed power settings and, for moving parts, a strobe timed to the exposure. Fixed exposure and fixed gain also matter: auto exposure that varies between poses changes the apparent centre of a dot with soft edges. If your feature centroids wander between frames, the contour position jitter under automatic exposure article walks through why and how to lock the settings down.

Fixed camera versus camera on the robot

Two geometries dominate, and they calibrate differently.

For a fixed camera, the nine-point grid over the part plane is usually all you need. When the camera rides on the arm, the robot moves the camera over a fixed target through many flange poses, and the solver separates the camera-to-flange transform from the target pose — this is the full hand-eye problem. Choosing robot poses that actually constrain that solve is its own skill; the article on eye in hand calibration pose selection covers which positions work and which silently weaken the result. For deeper controls-side detail, this pixel-to-robot coordinate transform walkthrough for a Fanuc CRX hand-eye calibration shows the same solve on a specific controller.

Keeping the calibration valid

A calibration is a physical measurement, and it expires the moment its assumptions break. Recalibrate or verify after:

• any bump to the camera, bracket or lens (focus ring included),
• a lens or camera swap — the board level camera replacement guide covers what carries over and what does not,
• robot mastering or collision recovery, which shifts the base frame the transform points into,
• lighting changes beyond a small power trim.

Build a daily or per-shift verification: place a known artefact, command the robot, log the miss, and alarm when it exceeds a threshold. That turns silent drift into a logged event. When something has already gone wrong, the vision calibration recovery article gives a sequence for rebuilding without starting from scratch.

When nine points is not the right tool

The planar nine-point method assumes the part always presents in one plane and one orientation family. It is the wrong tool when:

• Part height varies. Every millimetre of height error moves the pick point by that amount times the camera tilt. Square the camera up or measure height separately.
• Parts rotate out of plane. A 2D transform cannot represent tilt; you need a 3D sensor or a second view.
• The field is larger than the robot's dexterous zone. Sample the grid where the robot actually works, and consider two stations with two calibrations rather than one stretched transform.
• You need sub-pixel metrology, not guidance. For measurement, a full camera calibration with distortion model and traceable targets beats a fit-to-nine-points every time; the article on calibration target field of view and lens coverage margins explains how much margin a target needs.

Checklist before you run the session

• Lens, focus, aperture, lighting and exposure locked to production values.
• Target flat and rigid at the working plane, features spanning at least 80 percent of the field.
• Nine or more poses, spread to the corners, including the rotations production will use.
• Residual recorded, plus a corner reprojection check.
• Physical verification against a gauge or marked part.
• Drift check scheduled per shift, with an alarm threshold.

If you are speccing the imaging side from scratch, the area scan cameras finder filters by resolution, shutter and mount, and the lenses finder matches focal length and image circle to the sensor; the camera configurator then steps through the pair with the mount and cable. For placement of the camera relative to a moving conveyor, see conveyor tracking camera placement. To size the field of view and pixel scale before you order, start from the worked example above and confirm with the calculators — and if the geometry is unusual, contact us with the working distance and tolerance and we will check the numbers with you.