The marker positions come back within a millimetre, the camera's xyz against the robot base looks sensible, and then the orientation quaternion lands tens of degrees away from anything the cell should report. That combination, good translation with bad xyzw, is one of the most repeatable patterns in hand-eye calibration, and it almost never means the solver failed. It means the rotation is written in one convention and read in another: a sign the algebra allows, a frame direction nobody declared, or a multiply order flipped inside the check itself. This guide gives you the checks in the order that clears them fastest, with the arithmetic for each, and a residual threshold that tells you when the calibration is actually done.
Why does the quaternion deviate while the translation is right?
Nine times out of ten the solve is healthy and the comparison is not: quaternions double-cover rotations, so q and -q are one identical orientation, and the camera SDK, the robot controller and your test script rarely agree on which frame points which way. Translation carries no such doubling, which is why xyz survives while xyzw looks broken. After the sign, the two mechanical mismatches are a transform read against its own inverse and a composition in the flipped order; behind them trail Euler-order swaps, axis permutations between libraries, and marker measurements skewed by poor intrinsics. The sections below test each in turn, cheapest first.
If your numbers run the other way, orientation plausible and translation off by centimetres, read hand-eye calibration translation error: is 1-2 cm normal? instead; it is the mirror symptom.
What is the true angle between two quaternions?
Compute the rotation angle between the two results as θ = 2 × acos(|q1·q2|), built on the absolute dot product of the unit quaternions: the absolute value removes the sign freedom, and this θ, not the component-wise gap, is the only deviation worth judging. Normalise both inputs first; an unnormalised quaternion skews acos.
The absolute value exists because a quaternion and its negation encode the same rotation; without it, roughly half of all comparisons would report a false 180°.
Did you compare the transform with its own inverse?
Invert-check second, because a flange-to-camera result read as camera-to-flange reports twice the rotation angle of X: a 30° twist compared against its own inverse shows 60°, on every pose pair, while the translation still looks fine. The inverse of a unit quaternion is its conjugate, (qw, -qx, -qy, -qz).
Worked example: take X as a 30° rotation about Z. Then qw = cos(15°) = 0.9659 and qz = sin(15°) = 0.2588, so qX = (0.9659, 0, 0, 0.2588). Compared against its own inverse, (0.9659, 0, 0, -0.2588), the dot product is 0.9659 × 0.9659 - 0.2588 × 0.2588 = 0.933 - 0.067 = 0.866, so θ = 2 × acos(0.866) = 60°. A 30° twist reports as 60°, and it does so identically for every pose pair, which is the tell. An error proportional to twice the rotation of X is the fingerprint of an inversion; scattered noise is not.
The same flip further downstream turns a good solve into a rotated point cloud: point cloud rotated 90 degrees after hand-eye calibration is this exact failure one step later in the pipeline.
Is the multiply order flipped in your check?
Multiplication order is suspect three: the hand-eye equation A·X = X·B ties one relative robot motion A to the same motion B seen by the camera, and a verification composed in the flipped order can certify a wrong X, so test the prediction, never the identity.
A is the robot's motion between two poses, in the flange frame; B is that same motion measured by the camera. Declare X's direction, predict B for each pair, and score it against the measured B with the θ formula. Residuals under about 1° on every pair mean direction and order are consistent. Residuals that grow with the robot's rotation between poses mean the composition is backwards. Eye-to-hand cells solve the same equation for base → camera; the numbers look interchangeable and are not, which is why the written declaration outranks the value.
Robot-side, the same class of mismatch hides in the work object: controllers report tool orientation in the active work-object frame, and a stale or wrongly picked one appears as a constant orientation offset. The work-object orientation quaternion mismatch fix walks that specific case end to end.
Do you need to calibrate intrinsics on a depth camera at all?
No: a factory-calibrated depth camera needs no checkerboard session of your own, because its depth and colour intrinsics are stored on the device and the SDK's enumeration or calibration call prints them, so feed those numbers to the marker detector unchanged. Re-deriving intrinsics the factory already characterised adds error; it does not remove it.
What you read off is the usual set: fx, fy, cx, cy and the distortion coefficients. Project the marker corners with them during detection and let the depth stream supply range only. A bare board-level 2D camera is the exception that does need your own calibration: 15-20 checkerboard views spread over the frame and tilt, judged by reprojection error, with under about 0.3 px a pass and above about 1 px a reshoot rather than a solver tune.
Two depth-specific traps remain. A marker detected in the RGB image must be projected with the RGB parameters, not the depth camera's. And if you align or rectify streams before detection, the intrinsics change at each step; aligned depth streams: data layout and which intrinsics apply sorts out which set feeds which pixel. For the way intrinsics errors reach the hand-eye solve itself, read camera intrinsics for hand-eye calibration: fix the pose first.
How much orientation error is acceptable after the fix?
Judge the solve on the per-pair residual angle: under about 1° is a healthy marker-based run, 1-5° means add poses or improve the marker captures, and 5° or more on every pair is a convention error that no number of extra poses will fix.
| What θ shows per pose pair | What it means | What to do |
|---|---|---|
| Components 180° apart, θ ≈ 0° | q vs -q sign only | Nothing; compare through the absolute-value formula |
| θ = 2 × rotation angle of X, every pair | Transform read against its inverse | Publish one direction, name it, compare like with like |
| θ grows with the robot's rotation between poses | Multiply order flipped in the check | Recompute B_pred = inv(X)·A·X and rescore |
| θ near 90° or 180° exactly | Axis permutation or Euler-order swap | Declare frames, reconvert; do not patch components |
| 1-3°, scattered | Marker noise: single tags, distance, blur | Board instead of single tags, static capture, shorter exposure |
| 5° or more, every pair | Convention still wrong | Stop adding poses; fix the frames first |
The gate has a physical basis: with a printed board at 300-600 mm working distance and a static capture, marker orientation noise runs a few tenths of a degree, and a typical six-axis arm repeats to hundredths of a millimetre, so a healthy solve lands well under 1° of systematic error. Prove it once end-to-end with hand-eye calibration accuracy verification: test before you trust it, which turns this gate into a physical test.
How do you keep the frames straight for good?
Name the transform for what it is, camera_to_flange or base_to_camera with the direction in the name, publish it once, and make the absolute-value angle formula the only way anyone on the team compares two orientations.
- Convert once: matrix to quaternion with one routine, and write the component order next to the values. Textbook order is (w, x, y, z); many controllers list (x, y, z, w).
- Roundtrip-test the conversion: back to a matrix and through the θ formula, which must return 0°.
- Publish exactly one transform, with the direction in its name.
- Route every comparison through the absolute-value θ formula.
- Keep one publisher: a second node broadcasting an older solve silently fights the new one, and duplicate hand-eye calibration publishers: find and fix the conflict shows how to find it.
When the pick logic consumes the result, object pose in camera frame at runtime after hand-eye calibration defines what the published transform must mean downstream. If a correction has to go into the robot program itself, the quaternion orientation adjustment walkthrough for robot controller programs works the arithmetic on real controller values. The rest of the systems guides build on this same frame discipline.
Which camera should feed the next calibration run?
Hardware is rarely the culprit here, but two camera choices make orientation measurements steadier: a global shutter that never tears a marker image while the robot settles, and a depth camera whose factory intrinsics feed the detector known optics from the first pose. Both are purchase-time decisions, not calibration-time ones.
For a marker-based eye-in-hand build, the MV-CU020-90GC pairs a 2.5 MP global-shutter sensor with a C-mount and a GigE link, the combination that keeps tag corners crisp while the arm holds each pose.
- Sensor format
- 1/1.7"
- Lens mount
- C-mount
- Interface
- GigE
- Resolution
- Up to 3 MP
Where the cell measures depth as well as markers, the MV-DPS200P-02 is a projected structured-light 3D camera in the stereo / structured-light family, brought into the cell under the same frame-declaration discipline as any camera input.
- Type
- Stereo / structured light
Laser profilers and stereo 3D cameras
170 models · from $699.00Browse the area scan cameras and 3D sensors categories, or start at the camera configurator to spec the rest of the cell. If you send the pose list and residual table through the quote request, we will check the conventions with you before the next robot run.
Questions
What does AX = XB actually say, in plain language?
A is one robot motion measured between two poses, expressed in the flange frame; B is the same motion seen by the camera, expressed in the camera frame; X is the constant flange-to-camera transform that makes both descriptions agree. Each pose pair contributes one such equation and the solver combines all of them into a single X. When a library swaps the roles, eye-to-hand instead of eye-in-hand, the same equation solves the base-to-camera transform instead, which is why two correct-looking results can name different frames.
How many robot poses should a hand-eye run use?
Plan on 15 to 25 static poses. Spread the rotations so the marker board tilts toward at least three different regions of the camera view, and keep translations modest: rotation diversity drives the solve, translation diversity adds little. Poses with the board near the image edge are worth including because they stress the distortion model, but discard any capture with blurred or partially visible markers rather than feeding it to the solver.
Can I average the quaternions from several calibration runs?
Not by component-wise averaging, which is exactly what the q and -q sign trap punishes first. Align every run to a reference by flipping signs so each dot product is positive, then either convert to rotation matrices, sum and re-orthonormalise, or use a dedicated rotation-averaging routine. Run-to-run spread of a few tenths of a degree is normal; a spread above one degree points to a capture problem, not a rounding problem.
The solver prints a matrix and the controller wants quaternion components. How do I convert safely?
Convert once, with one documented routine, then verify the round trip by converting back and checking the angle formula returns 0°. Write the component order next to the stored values, because many controllers list (x, y, z, w) while most textbooks list (w, x, y, z). A reordering of the same numbers is a perfectly valid quaternion of a different rotation, and no residual check will warn you about it.
Do I have to redo hand-eye calibration after swapping the lens?
Redo the camera calibration, yes; the hand-eye transform, usually not. A different focal length changes the intrinsics that turn pixels into marker poses, so re-run the intrinsics first and then the hand-eye solve to confirm the result still holds. If the new lens moved the optical centre, or the camera body or mount was touched, redo the full procedure. If only the optics changed, the residual-angle check tells you within minutes whether the old transform survives.