Hand-eye calibration on a 5-axis robot: the missing rotation

Hand-eye calibration tools are built for 6-axis arms, and a 5-axis robot is where they start to refuse: the sample list stays empty, the stage after detection reports it cannot recognize a valid position, or the calibration converges and every pick still lands millimetres off. The usual cause is not the marker, the camera or the frame tree — it is a missing rotation. A classic solver needs the camera rotated about at least two non-parallel axes, and many 5-axis cells only ever rotate about one. You will find here which degree of freedom the solver cannot see, how to tell a data problem from a broken frame tree, and the three fixes that work: tilt the wrist, clock the camera square, or change the formulation.

What AX=XB asks of the arm

Every classic hand-eye solver consumes the same data: pairs of recorded poses. For each pair it forms the relative flange motion A and the relative marker motion B measured by the camera, then searches for the constant camera-to-flange transform X that satisfies A X = X B. Split X into its rotation R_X and offset t_X and the constraint separates into two equations.

The rotation equation says the marker's rotation is the flange's rotation viewed through the camera mount. That pins R_X down only in the directions you actually exercise. The classic 1989 Tsai-Lenz result states the requirement plainly: the pose set must contain rotations about at least two non-parallel axes. Give the solver one axis only, and entire families of different X reproduce exactly the same data.

Two prerequisites sit upstream and sink more calibrations than missing DoF ever does: the camera intrinsics must be fixed first, and the marker pose must reach the solver as a proper frame. The intrinsics-first checklist and the marker-pose-to-frame workflow cover both.

What a 5-axis arm can and cannot give the solver

Split 5-axis arms into two cases, because they fail differently.

A J1-J5 articulated arm with a wrist you can program in pitch and roll can point the camera several ways. Yaw the base for rotation about the vertical, pitch the wrist for rotation about a horizontal axis, and the two-axis requirement is met. When such an arm fails, the cause is usually the pose set — everything recorded tool-down — rather than the arm itself. The pose-variety rules for a 6-axis calibration apply here with one tilt group added.

The second case is the degenerate one: gantry-style axes, and any 5-axis cell that keeps the tool pointing at the floor because the gripper never needs a wrist roll. Every relative motion then contains rotation about one axis, the vertical. The camera may translate freely and it may yaw, but it never tips. That is the geometry the rest of this page works through.

The two directions the solver cannot see

The first blind direction is the camera's clocking about the vertical. Yaw the flange 30 degrees and the marker turns 30 degrees in the image no matter how the camera is clocked; the turn angle and the turn axis both stay silent about a rotation around that same axis. Yaw data alone can never reveal the clocking.

Pure translations can. Command three straight moves of 100 mm, one along each base axis, without yawing, and the marker drifts in three different image directions. Those directions rotate exactly with the clocking, so three non-coplanar moves pin the whole rotation matrix — the only route by which a tool-down cell ever sees its clocking, and it costs four photographs.

When you audit the solved matrix by hand, calculating the hand-eye angles from a rotation matrix is a five-minute arithmetic check worth running once.

The second blind direction is the camera offset along the vertical. In the translation equation the term (R_A - I) multiplies t_X, and for every rotation about the vertical that matrix has a null direction along the axis. The component of the offset along the axis never enters the arithmetic — not through yaws, not through z-strokes, not through any motion a tool-down cell can make. The height offset has to come from mechanics, which is why the touch-off in Fix 2 is not optional. How much standoff drift the focus tolerates before the marker softens is a depth-of-field question; check your sensor, f-number and working distance in the calculator below.

The standoff itself — how close the camera may sit and which target survives there — has its own target-and-standoff guide.

Worked example: a 2 degree clocking error at a 200 mm pick radius

The cell is a 5-axis vacuum picker that never tips the tool, so the pose set rotates about the vertical only. The camera adapter was machined and shimmed to look square, and the clocking was never solved; assume it is out by 2 degrees — a realistic outcome for a shimmed adapter, and the illustrative input for this example. The pick point sits 200 mm horizontally from the flange's yaw axis, typical for a vacuum cup hung off to one side.

The rule, in words: an unsolved clocking error of angle ψ about the vertical displaces every point at horizontal radius r from that axis by approximately r times ψ, because the point is carried around the axis instead of being seen straight.

A 2 degree assumption costs 7 mm at this radius — squarely the millimetres-to-centimetres band a mis-calibrated 5-axis cell produces. The sensitivity is 200 mm × π / 180 = 3.5 mm per degree, and it scales linearly: at a 400 mm radius the same 2 degrees puts the pick out by 14 mm. Working backwards from a ±0.5 mm pick tolerance at 200 mm gives a clocking budget of 0.5 / 3.5 = 0.14 degrees.

The decision follows. No shimmed adapter holds 0.14 degrees through mounting and temperature swing, so the clocking must be solved from translation data or measured with the drift test — never assumed from the bracket. The height blind spot is harsher still: every millimetre the standoff is off passes one-for-one into pick z.

Which layer is actually failing

A 5-axis failure sits in one of three layers, so test them in order. Layer one is detection: single images contain a clean marker and the detector returns a pose on its own. If that fails, fix focus, exposure or the printed target first, because no solver change will help.

Layer two is the frame tree. The marker frame must connect through the camera's optical frame to the base; when that optical frame flickers or never attaches, samples drop before the solver runs and the cause is frame publishing, not calibration. The missing-target-frame checklist walks that layer end to end.

Layer three is the data screen, and on a 5-axis it is the usual culprit. Before any math runs, the tool forms motion pairs from consecutive samples and rejects pairs whose relative rotations are near-parallel, or samples missing on one side of the pair. A tool-down pose set fails that screen wholesale. That is why detection can work perfectly on its own while the calibration stage insists it cannot recognize a position — the message is about the motions, not the images. The sample-once, replay-later workflow records a pose set once so you can re-run the screen without re-teaching the arm.

Fix 1: tilt the wrist so two axes appear

If the arm has a programmable wrist, the cheapest fix is pose variety. Record the usual tool-down ring of poses, then add two groups: one with the wrist pitched 20 to 30 degrees camera-forward, one pitched 20 to 30 degrees camera-sideways, marker still in view and in focus in every frame. Vertical yaws plus two tilt directions give rotations about non-parallel axes. The standard solver then converges unchanged — no new packages, no new mathematics.

Keep the tilts inside what the cell tolerates. The marker must stay in the field at the tipped pose and the dress pack must keep its bend radius. A board that fills the view tool-down may crop at a 30 degree tip, because the standoff grows along the optical axis; the target-sizing rule sizes the board for the worst standoff in the set.

Fix 2: clock it square, calibrate translation only

On a tool-down cell without a usable wrist, make the missing rotation a mechanical constant instead of a solver unknown. Dowel the camera adapter square to the flange so the rotation is zero or exactly 90 degrees by construction, then solve translation only. Three straight 100 mm moves along the base axes confirm the clocking from the drift directions. A yaw sweep about one known feature gives the in-plane offset. The height comes from a touch-off: a pointer or dial indicator on a machined datum lands within a few tenths of a millimetre. Done this way, in-plane pick accuracy sits in the arm's own repeatability band and z accuracy equals the touch-off.

The camera choice stays ordinary. A 2.5 MP GigE colour body with a global-shutter sensor, such as the MV-CU020-90GC, detects a printed marker at these standoffs. For small markers at longer range, a 12 MP USB3 body such as the MV-CH120-11UC buys marker pixels without changing any of the geometry above.

Fix 3: change the formulation or add the axis

The third route is mathematical. Limited-DoF hand-eye formulations in the literature treat the missing rotation as a known constant and solve only the observable remainder — the formal version of Fix 2. Some tools also accept a user-supplied or fixed rotation, which lets you hand the dowelled clocking to the solver and keep it off that degree of freedom.

The hardware route adds a calibrated rotary positioner as a seventh axis and folds its motion into the flange motion before the pairs are formed. That restores a second rotation axis, but the positioner's own repeatability joins the calibration chain: a unit repeating within ±0.05 mm puts that band into every result. Choose this route when the process needs the rotation anyway, not merely for calibration.

If the cell is early enough to choose the sensor, a projected structured-light camera such as the MV-DPS200P-02 swaps the printed board for a depth map and speeds detection. The observability problem is identical, though: no solver sees a rotation the arm never made.

Prove it before the first real pick

Verification is a test, not a feeling. Re-detect one datum feature with the arm parked at three widely separated poses; the residuals should agree within a few tenths of a millimetre. Run twenty full pick cycles and log the x-y spread — a spread that grows toward the image edges points at intrinsics or a warped printout, not at DoF. The verification sequence enumerates the tests, and the runtime eye-to-hand transform carries the result into the pick code. The same chain, worked end to end from a detected pixel to a robot coordinate, is in a pixel-to-robot coordinate transform walkthrough.

Re-run the drift test last. If the clocking has moved since calibration, the adapter is not rigid or the dowels are loose, and no amount of re-solving will hold it.

When the usual answers are wrong

Three cases get misdiagnosed as missing DoF. If the arm has a usable wrist, the fix is pose variety, and swapping solvers changes nothing. If residuals grow toward the image edges, the intrinsics or the target printout is at fault. And if the result comes out rotated by 90 degrees or mirrored, that is a frame-convention mismatch between camera, flange and base — the rotated-point-cloud diagnosis and the tool0 frame check cover both. None of the three is cured by more poses.

The decision, case by case

• Wrist-capable 5-axis arm: keep the standard solver and add two 20 to 30 degree tilt groups to the pose set. Expect datum residuals in the few-tenths band and picks at the arm's repeatability.
• Tool-down cell without a wrist: dowel the camera square, confirm the clocking with three 100 mm translation moves, sweep one yaw for the in-plane offset, touch off the height on a datum. Expect in-plane at arm repeatability and z at the touch-off.
• Process needs a second rotation anyway: add a calibrated rotary axis, fold it into the motion, and accept its repeatability into the chain.

Whichever branch you take, the checks above decide whether the calibration ships. If you are still choosing hardware, the area scan cameras and 3D sensors categories are the shortlists, the camera finder narrows by sensor, resolution and interface, and the camera configurator pairs camera, lens and light step by step. More calibration walkthroughs live on the 3D vision topic page and the guides hub, or contact us with your arm's joint list and we will help you size the pose set.