ArUco marker hand-eye calibration: why the marker pose stays unknown

Camera on the wrist, one printed ArUco marker clamped somewhere on the bench, a tracker watching that marker, and a solver that combines the two streams: that is the entire eye-in-hand set-up. The question that stalls people is deceptively simple — where do you measure the marker relative to the robot base? The answer is that you never do. The solver carries the base-to-marker pose as one constant unknown and resolves it alongside the flange-to-camera transform. This page explains why that works, the four capture conditions the method depends on, how large the marker must sit in the image, and the checks that tell you the result is good — with a worked sizing example along the way.

The one assumption that replaces the measurement

Every calibration sample hands you two transforms. The robot controller reports the flange pose for sample i, written T_BF(i): base to flange. The ArUco tracker reads the same image and reports T_CM(i): camera to marker. Walk the loop around the rig — base to flange, flange to camera, camera to marker — and you arrive back at the base, because the marker is fixed to the same world the base stands in:

The camera never moves on its bracket, so T_FC, flange to camera, is one constant. The marker never moves on its plate, so T_BM, base to marker, is another. Two unknown constants, a dozen or more samples, and the rotation and translation components together hand the solver far more equations than it needs.

Why you never survey the marker

Solve the over-determined system and both constants fall out together. Substitute the solved T_FC into any single sample and T_BM appears — no tape measure, no reference target, no survey. That is the entire trick, and it answers the question that stops most first attempts: how can you feed camera-to-marker pairs to the solver when you do not know where the camera is? You do not need to know. The inputs are the per-sample robot pose and the per-sample tracker pose; the constants are outputs, not inputs. The frame pairs an eye-in-hand calibration consumes are exactly these two streams.

The same structure holds with the camera fixed in the cell and the marker on the tool: the unknowns swap roles, nothing else changes. The solver does insist on one word — constant. The derivation holds only while neither the camera mount nor the marker shifts.

Four capture conditions that keep the marker constant

Fixed and flat. Glue the print to an aluminum or acrylic plate. A corner sitting 1 mm out of plane shifts its projection sideways by roughly 1 mm × tan 30° ≈ 0.6 mm at a 30° viewing tilt — an estimate, but the same order as the accuracy you are chasing. Loose paper curls with humidity and drifts mid-session.

Even, matte lighting. Light the marker diffusely from the front. Glossy laminate or a bare overhead lamp lays a specular streak across the black border, and corner detection degrades exactly where the accuracy is made.

Sample only when stopped. Save a pair when the arm is fully still and the tracker has held the marker steady for several consecutive frames. A frame grabbed while the arm settles pairs a blurred marker with a flange pose the robot had not yet reached.

Never head-on. Keep 15-30° of tilt between the camera axis and the marker normal in every pose, and spread the tilt direction across the set. A camera that stares straight down the marker normal leaves the rotation components about that axis unobservable, and the solve fails quietly instead of complaining.

Measure the print, then enter the meters

The generator draws the marker at a nominal size; the printer decides what lands on the paper. Measure the black square across its sides with calipers and enter that value in meters. One real set-up measured its print at 0.12558 m and passed exactly that value, together with its marker ID 566, to the tracker. The ID itself is arbitrary — 566, 42, anything — as long as tracker and calibrator agree on it. The entered side length converts the tracker's pixel solution into millimeters, so it sets the scale of everything downstream. The print-and-measure procedure is shown step by step in measured marker size in hand-eye calibration.

Worked example: how much of the frame the marker needs

Size the optics backwards from one rule: the marker should span 60-70% of the image width, leaving 15-20% margin per side for tilt and for the detector's white border.

Take a 12 MP camera as the wrist unit. The LS-CMR-0166 resolves 4000 × 3000 at 30 fps over USB3 on a 1/1.7" sensor.

A 1/1.7" sensor is about 7.2 mm wide, so at 4000 pixels across, each pixel covers about 1.8 µm. With a 16 mm lens at a 400 mm working distance:

The 125.58 mm marker fills 125.58 / 180 ≈ 70% of the width, leaves (180 - 125.58) / 2 ≈ 27 mm of margin per side, and spans 4000 × 125.58 / 180 ≈ 2790 pixels. That is a scale of 45 µm per pixel at the marker. Corner detection typically lands within 0.1-0.3 px, so corner positions hold to roughly 10 µm, near 0.01% of the marker side.

Swap in a 25 mm lens at the same 400 mm and the field drops to 7.2 × 400 / 25 ≈ 115 mm — narrower than the marker, which no longer fits at all. With a 25 mm lens you would back out to roughly 660 mm, where the field is 190 mm and the marker sits at a comfortable 66%.

For a 125.6 mm marker, a 16 mm-class lens at 400 mm working distance is the geometry to copy. The lens focal length calculator runs the same field-of-view arithmetic for your own sensor and working distance; if the target is a whole board rather than one marker, hand-eye calibration target size works out that larger geometry.

The frames you supply, and the one you leave empty

The tracker publishes its solution between two frames you name: camera_frame at the optical center and marker_frame at the marker's origin — the aruco node puts camera_frame → marker_frame on tf. The robot side supplies base to flange. The calibrator consumes the two streams sample by sample; an AX=XB solver such as visp_hand2eye_calibrator takes exactly these frame pairs. The third name, ref_frame, exists only for calibrating against a different root, for instance a depth camera whose tf root sits away from its RGB optical center. With a plain wrist camera, leave ref_frame empty or set it equal to camera_frame. List the frames that actually exist in the tf display of rviz or in rqt_tf_tree before starting; a misnamed frame is the most common silent failure at start-up.

Intrinsics come first: calibrate the lens, lock the focus, and only then record the pose set, because every tracker pose inherits any error in the camera model. The order is laid out in camera intrinsics for hand-eye calibration, and the tracker-side transform plumbing in publishing an ArUco marker pose as a TF frame.

A compact global-shutter GigE camera is the usual wrist choice. The MV-CU020-90GC pairs a 1/1.7" sensor with a 2.5 MP global shutter.

How many poses, and how to spread them

Ten to twenty stopped poses is the working range. Shape the set with two rules. First, consecutive samples should differ by a rotation: pairs related only by translation add nothing, because only relative rotations constrain the fixed transforms. Second, hold every pose inside the marker-fraction band from the worked example, keep the tilt from the capture conditions, and spread the tilt direction in azimuth — paint the viewing hemisphere rather than revisiting one sweet spot. Eye-in-hand calibration pose selection plans these positions in detail, and hand-eye calibration dataset acquisition shows how to record each robot pose beside its image so a bad sample can be dropped later.

The marker needs a home that will not move. The LS-MNT-0040 slide-table stand is 700 mm tall on a 400 × 450 mm base; clamped to the bench, it holds a marker plate motionless through an afternoon of arm passes.

When the constant assumption breaks

Bump the marker mid-session and T_BM has changed: every sample before the bump belongs to a different problem, so recapture the whole set after any move. Nudge the camera on its wrist bracket and T_FC has changed, with the same consequence — a bracket that rotates under cable load produces exactly this failure, and the fix is described in wrist camera mounting orientation. Enter the wrong size and the rotations survive while every translation comes out scaled by the same proportion. Print on loose paper and the corners drift as the sheet curls over a long capture; mount it on plate.

Checks before you trust the result

Two quick checks. First, use the solved flange-to-camera transform to compute the base-to-marker pose from every sample individually: the values must agree. On a rigid bench, with the marker tracked at thousands of pixels across, pose-to-pose spread of a few tenths of a degree and a few tenths of a millimeter is normal; one sample far outside the spread had bad tracking or a bad robot pose, so drop it and re-solve. Second, close the loop: drive the arm to a fresh pose, read the tracker, and compare the marker position the calibration predicts in the base frame against the tracker's direct observation. The runtime use of that result is covered in the object pose after hand-eye calibration, and the same verification logic for a fixed camera is in eye-to-hand calibration verification.

To build the set: choose the sensor in the area scan cameras range or the camera finder, match the lens to your working distance in the camera configurator, and put the marker plate on one of the camera stands. If you want the marker size and working-distance plan checked before ordering, the contact page is the way in, and the rest of the 3D material lives on the 3D vision topic page.