Hand-eye calibration for an upward-facing camera on a robot

An upward-facing camera breaks the usual hand-eye routine. With the camera on the arm looking down at a fixed board, the robot pokes nine points and the job is done. With the camera bolted below the work plane looking up, there is nothing for it to touch. The fix is to swap the roles: the robot grips a mark plate and carries it over the camera. What follows: the grip and plate targets, the nine-pose pattern, the solver inputs, the accuracy to plan for, and the five-point check that signs the calibration off.

How do you calibrate a hand-eye pair when the camera points up?

You invert the rig: fix the camera looking up, let the robot grip a matte mark plate, and move that plate through at least nine poses over the camera — the same AX=XB solver then returns the fixed camera-to-base transform.

That is the only structural change. Plate detection, the solver and the verification run exactly as they do with the camera on the arm, because AX=XB never asks which device moves; it only needs matched relative motions from both sides. The classic version, where the camera rides the flange and pokes a fixed nine-point board, is laid out in robot hand-eye calibration: how nine-point alignment works.

The inverted rig buys you two things. The robot calibrates with its production gripper attached, so the tooling you pick with is the tooling you calibrate. And the camera never sees a production part during calibration: the mark plate stands in for every part, so opaque, black or mirror-finish products are irrelevant to the solve.

What do you need before you start?

Four things: solved camera intrinsics, a matte nine-dot mark plate, a grip that returns the plate to within ±0.05 mm and ±0.1°, and a robot whose ISO 9283 pose repeatability is ±0.05 mm or better — most industrial arms sit between ±0.02 and ±0.05 mm.

Camera and lens, focused and locked. Any of the area scan cameras in the camera finder works once two conditions are met. Intrinsics must be solved for the exact lens, aperture and focus position — hand-eye results inherit intrinsic errors one-for-one, as camera intrinsics for hand-eye calibration: fix the pose first explains — and the focus and iris rings must be mechanically locked so nothing moves between calibration and production.

A matte mark plate. Design target: 60 × 60 mm, nine dots of Ø 6 mm on a 20 mm grid, one corner dot enlarged to Ø 8 mm so plate yaw is never ambiguous. Matte black on matte white; no gloss anywhere.

A repeatable grip. Vacuum cup on the plate centroid, plate on a stiff carrier, one dedicated calibration plate that always enters the gripper the same way round.

A rigid mount. A column or bracket bolted to the cell frame — see camera stands or the accessories finder. A mount that moves 0.1 mm between morning and afternoon shows up directly in your picks.

Light. A matte plate needs little; if the dots wash out or vanish, the fixes are the same as for chessboards — see chessboard not detected in calibration: lighting and focus fixes.

Resolution is the one camera spec worth spending on, because it pushes the detection term far under the grip and robot terms. The MV-CS200-10UM is a 20 MP mono USB3 camera with a 1" sensor and C-mount, and fits this rig directly.

For a smaller field a 2.5 MP GigE camera such as the MV-CU020-90GC does the same job. A stand like the LS-MNT-0040 — 700 mm height, 400 × 450 mm base — carries the camera rigidly under the work plane, and a tilt stage such as the LS-MNT-0001 squares the camera to the plate plane in minutes.

How do you pick the working distance and lens?

Size the field to the plate plus your positioning window, then apply m = sensor width / FOV and f = WD × m / (1 + m); at 600 mm, a 40 mm lens on a 2448 × 2048 px, 3.45 µm sensor covers 118 × 99 mm.

Worked example, arithmetic included. The illustrative sensor is 2448 × 2048 px at 3.45 µm, so 2448 × 0.00345 mm = 8.45 mm wide. The plate is 60 mm and you want ±30 mm of positioning margin, so FOV = 120 mm. Then m = 8.45 / 120 = 0.0704 and f = 600 × 0.0704 / 1.0704 = 39.5 mm — round to the stock 40 mm. Check forward: m = 40 / (600 − 40) = 0.0714, FOV = 8.45 / 0.0714 = 118 mm wide and 7.07 / 0.0714 = 99 mm high. Ground sample distance is 3.45 µm / 0.0714 = 0.048 mm/px, so a Ø 6 mm dot spans 6 / 0.048 = 125 px — plenty for a centroid to 0.1 px.

Focus depth is generous at this magnification:

With f/5.6 and a circle of confusion of 0.04 mm (about one pixel): DOF ≈ 2 × 5.6 × 0.04 × 1.071 / 0.0051 ≈ 94 mm, about ±47 mm around the focused plane. Depth of field is never the constraint here. The constraint is keeping the plate at the same height pose after pose, because the solver assumes one rigid plate-to-flange transform. Set the capture plane at the pick height so the calibration is exercised where production will run.

How do you hold the mark plate so the calibration stays valid?

The plate-to-flange offset is an unknown the solver absorbs — but only if it is the same unknown at every pose. Target ±0.05 mm and ±0.1° re-grip repeatability; a ±0.2 mm sloppy grip alone sets the calibration error at ±0.2 mm.

Vacuum cups are the usual choice because they self-center a flat plate; mechanical fingers work too, as long as they locate the plate rather than only clamping it. Quantify the grip before calibrating: move to a pose over the camera, capture, release, re-grip, and repeat ten times. The spread of the corner-dot centroid across those ten captures is your re-grip repeatability; hold it inside the targets above in X, Y, Z and yaw. If the spread is wider, fix the tooling first — stiffen the carrier, move the cup to the true centroid, add a stop so the plate cannot sit tilted. No solver averages a sloppy grip away, and no amount of extra poses compensates it.

What are the steps for the nine captures?

You capture nine image-plus-pose pairs: grip the plate, jog the TCP over a 3 × 3 grid at pick height with three yaw settings, settle 200 ms per pose, detect the dots, and save each flange pose from the controller with its frame.

1. Mount the camera. Bolt it under the work plane on a rigid column; square it to the plate plane to about 1°. The solver tolerates tilt, but a square camera keeps every dot near the same height, which makes focus and lighting uniform.
2. Set the optics. Working distance 600 mm, aperture f/5.6, focus on the plate at pick height, then lock focus and iris mechanically. Temperature moves focus during the day — see camera focus drift with temperature: diagnose daytime blur.
3. Solve intrinsics for that exact lens and focus position, before any robot motion.
4. Pass the grip test above before calibrating anything.
5. Jog the nine poses. A 3 × 3 grid over the central 80% of the field of view, all at pick height, yaw at 0° for three, +45° for three, −45° for three. If the arm has a wrist, add ±10° tilt on three of them; the pose-diversity rules are the same as when the camera rides the arm — see eye in hand calibration pose selection: robot positions that work.
6. Capture. At each pose wait 200 ms for vibration to settle, grab one frame, detect the nine dots with sub-pixel centroids, then read the flange pose from the controller after the settle window.
7. Solve. Feed the nine pairs to the solver in eye-to-hand mode. A library such as OpenCV takes its cv.calibrateHandEye inputs in eye-in-hand form, so you pass the robot poses inverted — base-to-gripper instead of gripper-to-base — and get the camera-to-base result.
8. Publish X once. Save the transform as a static camera-to-base value that every client reads at start-up — see publish eye-to-hand calibration as a static camera transform.
9. Verify with the five-point check below before anyone picks a part.

How does the solver turn nine captures into one transform?

It solves A·X = X·B: A is each relative flange motion reported by the controller, B the same motion seen by the camera as plate movement, and X the constant camera-to-base transform; Tsai-Lenz or Park-Martin return X from the eight relative motions nine poses provide.

Nine poses yield eight relative motions, and solvers use all pairs, not only neighbors, so the system is heavily over-determined and the least-squares solution averages the noise down. The classical methods — Tsai-Lenz, Park-Martin and relatives — differ in how they split rotation from translation. With clean data they land well inside your repeatability budget, so the solver choice is the least of your worries. Note what the solver does not need: the plate-to-flange offset is never measured, only kept constant, because the rigid-body algebra absorbs it into X.

How accurate will the result be?

Add the error sources root-sum-square: dot detection to 0.1 px is 0.005 mm at 0.048 mm/px, robot repeatability ±0.03 mm, re-grip ±0.05 mm, so √(0.005² + 0.03² + 0.05²) = 0.06 mm; plan on a residual of 0.05–0.08 mm and picks near 0.2 mm worst case.

The ordering matters more than the total. The grip term dominates: halve the re-grip spread and the whole budget shrinks with it, while more pixels barely move the number once dots span dozens of pixels. Detection noise also averages down over nine dots at nine poses, so its effective contribution is smaller than the single-dot figure. Robot repeatability is whatever the data sheet says, quoted per ISO 9283, and mostly lands between ±0.02 and ±0.05 mm.

Absolute accuracy is the term people forget. Repeatability returns the flange to the same spot; absolute accuracy is how truthful that spot is across the envelope, and on a long-reach arm it can add tenths of a millimetre at the field edges. That is why the verification below runs at production poses: it measures the whole chain, and that number — not the solver's residual — is what you quote to production.

How do you verify the calibration before production?

Command the TCP to five fresh points over the field corners and center at pick height, read each one back from the image through the solved transform, and accept the calibration only when the mean error is ≤0.1 mm and the maximum is ≤0.3 mm.

For each point, read the plate's corner-dot centroid from the image, map it to base coordinates through the intrinsics and X, and diff against the commanded TCP position; the mean and maximum over the five points are the acceptance numbers above. If the maximum fails, the suspect order is fixed: re-grip first (re-run the ten-grip test), pose spread second (add three or four poses), focus third — the solver is almost never the culprit. The same acceptance logic, for both camera positions, is in hand-eye calibration accuracy verification: test before you trust it, and the fixed-camera case specifically in eye-to-hand calibration: verify the camera-to-base result.

Mapping a dot centroid to a base pick point is then a short, fixed recipe — intrinsics to the camera frame, X to the base frame. A pixel-to-robot-base walkthrough for a six-axis arm follows it line by line, and use eye-to-hand calibration results to compute robot-base picks works the pick-side arithmetic.

When does an upward camera stop working?

It fails for three reasons: grip poses leaving the field of view, part heights wandering off the calibrated plane, or a camera bumped since the last solve — all three predictable from the numbers above, and all three with fixes.

Coverage. Grip poses must stay inside the field. At 600 mm the worked example covers 118 × 99 mm; keep the 40 mm lens and raise the camera to 1,500 mm: m = 40 / 1460 = 0.0274, FOV = 8.45 / 0.0274 = 308 mm, GSD = 3.45 µm / 0.0274 = 0.13 mm/px — dot detection stays near 0.013 mm, still far under the grip term. Coverage costs little; the grip sets the accuracy.

Height wander. Focus is not the limit (±47 mm of depth of field); mapping is. Parallax shifts a point by height error × off-center distance ÷ working distance, so parts must sit at pick height, or the pick code must use the full camera model together with the part's measured height.

A bumped camera. Any knock, lens swap or focus turn voids the solve — re-run the nine captures, and re-verify after fixture work near the camera. What slow drift of a fixed rig does to picks is covered in static camera calibration accuracy: why moved parts drift off target.

The rig earns its keep when the feature you must find faces down — parts on a transparent belt, an underside reference — or when the tool would shadow a downward camera. If the feature faces up, a camera on the arm stays the simpler build. If you set the cell up in an offline robot programming tool, a hand-eye configuration walkthrough for an offline programming environment mirrors the same nine-capture flow.

To size the camera and lens for your own field, start at the camera configurator; more calibration material sits on the applications topic page, and the contact page is open if you want the numbers checked against your robot's data sheet before ordering.