Hand-eye calibration wrong robot frame: match tool0 to the sample

A hand-eye calibration that returns rotation within a degree or two and translation off by tens of centimetres is not a solver problem — it is an input problem. You park the end effector right beside the camera, re-run the solver, and RViz still shows the camera hanging far from the arm. The diagnosis takes two minutes: stop the arm, read the tool0-to-base pose the way you always do, and compare it with the hand-to-world sample your calibration window recorded for that same pose. This guide gives the thresholds to hold a result to, the two causes behind the mismatch, and the fix for each.

Symptom: rotation fits, translation lands tens of centimetres away

The signature is easy to recognise. The rotation part of the result looks plausible, often within about a degree of where you would place the camera by eye, while the position part misses by 10, 30 or 90 cm. Hand-eye calibration estimates one fixed transform between the end effector and the sensor, so an error this lopsided points at a single input stream, not at the optimiser.

A sound eye-to-hand result lands within about 1–2 cm of the truth — see what a normal hand-eye translation error looks like. That sets the decision line at 5 cm, five times the typical residual: below it, suspect pose diversity and sampling hygiene; above it, stop re-running the solver and hunt the input fault, because no amount of extra poses closes a gap of tens of centimetres.

The two-minute check: sample against tool0 in RViz

Bring the arm to one of your sample poses and let it settle completely. Read the tool0-to-base pose in RViz, or echo the transform from the command line, and write down position and orientation. Then open the calibration window's record for that same sample and read its hand-to-world entry.

At a stopped pose both numbers come from the same edge of the TF tree, so they must agree to the millimetre. A larger gap means the calibration is sampling a frame other than the one you are watching, or two nodes are fighting over that edge. If RViz shows nothing at all for the frame you named, that is a simpler fault with its own fix — see RViz fixed frame world does not exist: pick a real TF frame.

The example values show the fingerprint: 62 mm of pure offset along one axis with rotation untouched is a frame sitting one tool-length away from the one you think you sampled, not noise. Judge the pattern, not the digits.

Why the sampled frame is not the frame you named

Three causes cover nearly every case.

A second publisher on the same edge. If the robot driver and another node — a static publisher or the planning stack — both publish tool0 or base, TF flips between them and the window can record a pose the display never showed. The test and fix are in duplicate hand-eye calibration publishers: find and fix the conflict.

You watch one frame and sample another. The robot_effector_frame argument names tool0, the flange, while your eye goes to a TCP frame that carries a tool offset. Both views are internally correct, but the finished result is then relative to a frame your downstream code never consumes. Decide once: calibrate and publish against the flange frame and apply tool offsets after, the same discipline the target frame missing: what to check guide enforces.

Sampling on the move. With freehand_robot_movement true and robot_velocity_scaling at 0.5, the arm still swings briskly between poses. The marker image and the TF read can then straddle a motion, producing sample pairs that disagree internally. Sample only when the arm has settled, and lower the scaling until you trust the cell.

The camera-side fault: a scale error, not a frame error

If the sample matches tool0 at every stopped pose, the robot stream is clean and the camera side is next. A planar marker's distance is inferred from its declared size, so a wrong size inflates every distance by the same share while rotation barely moves.

Worked example: the marker's black square truly measures 0.100 m, but marker_size declares 0.120 m, and the standoff at a sample is 0.800 m. The tracker infers d_est = 0.800 × 1.20 = 0.960 m, so that reading is 0.160 m too far. Solved over the whole pose set, the camera position shifts by the same order — tens of centimetres for a marker at arm's length — while rotation stays credible. The 0.1 m default in a launch file is a starting value, not a measurement: measure the printed black square, not the white border, and set it to the millimetre.

A focal length error does the same thing. Depth scales roughly linearly with focal length, so intrinsics 10% high push every point 10% too far, 80 mm at that 0.800 m standoff. This fault has ended real calibrations: a tracking camera's factory intrinsics were off enough to skew every range reading, and the identical procedure passed once the tracking stream changed. Before swapping hardware, calibrate the intrinsics of the exact image stream the tracker consumes — camera intrinsics for hand-eye calibration: fix the pose first gives the order that works.

Typical tracking sources on a cell of this kind are projected structured-light and ToF 3D sensors:

On any unit, calibrate or confirm the intrinsics of the exact image stream the tracker consumes before sampling; that check closes the fault described above.

Numbers to hold the result to

• Sampled hand-to-world against tool0-to-base, arm stopped: identical to the millimetre. Any larger gap: fix the frame or publisher before calibrating.
• marker_size: measured black square, in metres, set within ±1 mm; a 20% size error becomes a 20% range error.
• Translation residual after a clean run: about 1–2 cm. Above 5 cm: input fault, not pose count.
• Rotation residual: within about 1° for marker-based work at these standoffs.

Fix, resample and re-test

1. Repair the robot stream: one publisher per TF edge, and the same frame names in robot_base_frame and robot_effector_frame that you display and consume.
2. Measure the marker's black square and set marker_size to the millimetre.
3. Calibrate or confirm the intrinsics of the tracking stream.
4. Discard every sample taken under the fault, because old samples carry the bias forward — the rule in discard a bad calibration image or retake the pose applies to hand poses too.
5. Take 10–15 new poses with tilt spread in at least three directions, following hand-eye calibration dataset acquisition with saved robot poses.
6. Re-run the solver and publish the result as a static transform under the calibrated frame names, as in publish eye-to-hand calibration as a static camera transform.
7. Close with a displacement test before trusting picks: command a known motion and check the measured error, per hand-eye calibration accuracy verification: test before you trust it.

When more samples make it worse

The solver averages. A consistent error in one stream — the wrong frame, an oversize marker, skewed intrinsics — biases every sample in the same direction, so extra poses converge harder on the wrong answer, with tighter residuals that look reassuring. Grade the inputs first, then the count: with a clean sample-versus-tool0 comparison and a measured marker, ten poses beat thirty sloppy ones. And do not judge a result by rotation alone — the rotation term hides exactly the faults described above.

Pre-run checklist

Robot side:

• One publisher per TF edge; no leftover static publishers on tool0 or base.
• robot_effector_frame matches the frame you watch in RViz and consume downstream.
• Arm fully stopped at each sample; velocity scaling lowered.

Camera side:

• Marker's black square measured; marker_size set to the millimetre.
• Tracking stream intrinsics calibrated or confirmed on this unit.
• tracking_base_frame resolves to the optical frame the tracker actually uses.

Judging the result:

• Sample matches tool0 to the millimetre at a stopped pose.
• Translation residual at or under about 2 cm; rotation near 1°.

If the robot stream passes and the camera stream is the weak link, the 3D sensors category lists the profilers, structured-light and ToF cameras used as tracking and measurement sources, and the camera finder filters them by type. For a second opinion on a stubborn cell, contact us with your residual table, or browse the rest of the hand-eye guides under the 3D vision topic.