Hand-eye calibration unit mismatch: meters in, millimeters out

You rewrote the SDK sample into your own ROS node, drove it from a TF listener, and the eye-to-hand result came back wrong in a telling way: the orientation quaternion sits within a fraction of a degree of your previous, known-good pose, while the translation reads in the hundreds or thousands where your cell works in meters. TF publishes meters. Many 3D camera SDKs take pose translations in millimeters. Fed into one solver, the two scales damage exactly one output: the solved translation, by almost exactly 1000×. Here you get the three-number test that confirms the unit mismatch in a minute, the single boundary conversion that fixes it, the neighboring faults it mimics, and the checks to pass before the robot moves on the new pose.

Symptom: clean rotation, translation off by about 1000×

A failing result has a shape. The quaternion matches the old calibration, but position comes back as (-288.0, -896.0, 1691.0) where the known-good pose is (-0.288, -0.896, 1.691) m. Runtime picks then command targets far outside reach, or the robot controller rejects them as unreachable. A weak dataset does not look like this: poor pose collection degrades the rotation too, and a frame mix-up or quaternion-order error makes the orientation visibly wrong. A clean rotation with a scaled translation is the unit-mismatch signature. Run the test in the next section before you re-collect anything — recollecting takes an hour and fixes nothing.

Why only the translation breaks

The solve is the classic AX = XB hand-eye problem: robot flange poses from TF on one side, camera-observed target poses from the SDK on the other — the hand-eye calibration problem overview is a good refresher.

Rotation inputs are quaternions and matrices. They are dimensionless, so no unit can enter them, and the rotation survives any scale error intact. Translation inputs carry units, and here your two sources disagree by construction: TF is metric, while SDKs that document pose input in millimeters expect translations in millimeters. In the solver's translation equations the millimeter observations outweigh the meter values 1000:1, so the solution comes out at the millimeter scale of the true pose. Read as meters, that is 1000× too large. Feed millimeters where meters were expected and the ratio flips to 0.001. The exact blend depends on the pose set, so read 1000× as the signature, not a guaranteed exact factor.

The 1000× test: three numbers settle it

Measure one distance by tape: camera reference point to robot base origin. Compute the solved translation's magnitude and the ratio.

Expect r = 1.00 within about 1%: 1-2 cm of solve error on a stand-off near 2 m is roughly 1% of the distance. If r lands near 1000, your robot poses went in 1000× too small, and the fix below applies directly.

Fix: one conversion, at the SDK boundary

Keep meters as the pipeline unit: TF is metric and your messages are typed in meters. Convert in exactly one place — the function that hands a pose to the camera SDK — by scaling the x, y, z triplet by 1000 and passing the quaternion through untouched. Publish the solved pose back through the same boundary. One function owns the factor, so there is one place to audit.

# Illustrative helper names; the SDK call site is yours.
def robot_pose_m_to_mm(pose):
    # pose: (x, y, z, qw, qx, qy, qz), flange pose in meters
    x, y, z, qw, qx, qy, qz = pose
    return (1000.0 * x, 1000.0 * y, 1000.0 * z, qw, qx, qy, qz)

pose_m = (-0.288, -0.896, 1.691, -0.00163, 0.99943, 0.00191, 0.03370)
pose_mm = robot_pose_m_to_mm(pose_m)
# (-288.0, -896.0, 1691.0, -0.00163, 0.99943, 0.00191, 0.03370)

Test that proves it: the magnitude before the call is sqrt(0.288² + 0.896² + 1.691²) = sqrt(3.745241) ≈ 1.9353 m; after the call it is 1935.3 mm, exactly 1000×, and the quaternion norm stays 1.0000 because no component was scaled.

Four neighbors that mimic a unit mismatch

• Millimeter result pasted into a meter field. Same 1000× signature, but the solve was fine. The tape test tells the two apart: r near 1 means the printed numbers are already meters; r near 1000 means they are millimeters.
• Marker side length typed in the wrong unit. In an OpenCV solvePnP pipeline the returned tvec inherits the unit you typed for the marker, so a millimeter marker length scales every camera-side translation. See marker pose units and tvec.
• Wrong robot frame or mislabeled pose pairs. Unlike a pure unit error, the rotation degrades too. Check the frame each pose was recorded against: wrong robot frame.
• Quaternion component order. A wxyz-versus-xyzw mix makes the rotation visibly wrong and the translation follows it: quaternion mismatch.
• Point cloud scaled 1000× while the pose passes the tape check. That is the depth pixel scale, not the calibration. Convert it where the cloud is built: depth scale to meters.

Make the unit impossible to get wrong again

• Record the unit with every dataset: a unit field in the pose JSON or a _m / _mm suffix on the capture folder name.
• Own the factor in one function, called from every capture site. No inline 1000.0 anywhere else in the code.
• After every solve, run the tape check before anything moves; r must land within about 1% of 1.
• Publish the final pose in meters, so nothing downstream needs to know what the SDK takes.

Where the bridge runs

The TF listener, the SDK bridge and the solver usually share one vision controller next to the cell. Count ports before you size it: one LAN for the 3D camera, one for the robot controller, one for the PLC, plus PoE if the camera takes power over its network cable. The MV-VC5525-256G70-NN covers a multi-camera cell with PoE camera ports, 5+ LAN ports and 8 built-in light channels.

The MV-VC3301H-128G66 is the leaner single-camera bridge, with 5+ LAN ports, 4 built-in light channels and no PoE.

Verify before the robot moves

Then work up in this order:

1. Tape check: r within about 1% of 1, as in the test above.
2. Static target: transform a detected target into base coordinates at three positions spread across the workspace and compare against tape or calipers; expect agreement within the 1-2 cm a normal solve delivers — is 1-2 cm normal?.
3. Full procedure: the eye-to-hand verification checks cover the remaining failure modes.
4. Picks: feed the verified pose to your robot-base pick computation before any live cycle.

If you would rather run collect-and-solve inside your robot programming software, the same pose-pair discipline applies; this hand-eye calibration walk-through for a 6-axis cobot shows the flow end to end.

If the hardware side is still open, start from the 3D sensor finder or the 3D sensors category; more calibration and point-cloud troubleshooting sits on the 3D vision topic page. If the tape check still fails after a clean unit boundary, the pose set is the suspect, not the units — bring the dataset to our team.