Hand-eye calibration for a humanoid robot: track the head as the base

A camera on a humanoid's head is the most useful sensor on the machine — and the hardest to calibrate, because the classic hand-eye procedure assumes the robot's base is bolted down. A humanoid's base floats: the torso leans, the balance controller never stops correcting, and the head's pose in the world changes even when every neck joint holds still. The fix is not a new solver. Treat the head as your base frame, make its pose known to the solver for every image, and the standard AX=XB machinery works unchanged. This guide gives you the two ways to track the head, the target-and-lens numbers to fix before collecting data, the step-by-step run, and the residual numbers that separate a good calibration from a failed one.

The answer: the head is the base, and you must track it

On a fixed arm, hand-eye calibration is straightforward because the base never moves: the cell is surveyed once — base frame calibration on a bolted-down production cell is that job — and the controller hands the solver one constant frame forever after. A humanoid gives you no such gift. So run the calibration exactly as in camera above the robot base: eye-to-hand setup, with one change: the frame playing the base is the head, and you supply its pose yourself, image by image. The camera is bolted to the head, the target rides on a moving end effector, and the constant you solve for is the head-to-camera transform.

Whatever the package's menus call it, every hand-eye solver fits the same equation:

The solver trusts A completely. On a fixed arm, A comes from encoders on a machine that cannot shift, so it is exact. On a humanoid, A is only as good as the chain you feed it: whatever that chain misses — torso lean, root drift, a flexing camera bracket — lands in X as error. That is the dynamic-base problem in one line, and measuring the head's motion is its only cure. If your camera sat on a wrist instead, this would be the classic eye-in-hand calibration input problem: same equation, and the same rule that both members of every pair must describe the same instant.

Two ways to make the head's pose known

The solver never sees the humanoid directly; it sees pose numbers. You have two honest ways to produce them.

Publish the root-to-head chain

Feed the solver the full transform from a fixed root frame up to the camera mount: root, torso, neck, head. The root end comes from your state estimation — planted feet, leg odometry, IMU fusion. That chain, not the neck encoders alone, is what makes a humanoid different from an arm: published reference-point methods that run both eye-in-hand and eye-on-base setups need little more than this chain and something the camera can follow. The chain is measurable, so measure it — the still test in step 2 tells you whether yours is good enough before you waste a dataset.

Track the head with a fixed external camera

Skip the internal chain: fix a 3D camera where it sees a rigid marker on the head, and feed the tracker's head poses to the solver as your base poses. This trades estimator drift for tracker noise, and at head distances tracker noise is the smaller number. The marker must be rigid on the head shell — a bracket beats a sticker on a curved cover — and the tracker itself goes on a stand, not a hand-moved tripod.

Two close-range options from the 3D camera range:

The MV-DPS200P-02 is a projected structured-light 3D camera; the MV-DLS300P-04 is a laser galvanometer stereo camera. Either class resolves a rigid marker at the standoff a head calibration uses. Before committing, read 3D camera robot integration: check compatibility — the interface and frame conventions decide how much work this route is.

And mount the tracker like optics, not furniture:

The LS-MNT-0040 stand carries the camera at a 700 mm height on a 400 × 450 mm base, with a slide table to trim the standoff without re-shimming. The rest of the camera stands range covers taller and shorter setups.

Fix the target, lens and standoff before you collect anything

Three numbers decide whether the run yields solvable images: standoff, board size, focal length. The board plus the head's pose-to-pose travel must sit inside the field of view with margin at the closest approach. How big a board your poses need works the general geometry; short-range camera target and standoff covers the close-in case most head cameras live in.

Worked example, with the arithmetic:

• Closest approach, head to board: 500 mm
• Board: 240 mm checkerboard, 20 mm squares
• Head travel to allow: 25 mm per axis
• Sensor: 7.2 mm wide, 1600 px across — 4.5 µm pixels

Field of view needed: 240 + 2 × 25 = 290 mm. Magnification: m = 7.2 ÷ 290 = 0.0248. Focal length: f = WD × m ÷ (1 + m) = 500 × 0.0248 ÷ 1.0248 = 12.1 mm — fit a 12 mm lens.

Check the real lens at the same 500 mm: m = 12 ÷ (500 − 12) = 12 ÷ 488 = 0.0246 FOV = 7.2 ÷ 0.0246 = 293 mm, so about 26 mm of margin each side of the board. Scale: 293 ÷ 1600 = 0.18 mm per pixel, so a 20 mm square spans about 110 px. Corner finding wants roughly ten pixels per square minimum; even 5 mm squares at 27 px hold up if the board must shrink.

One more number: motion. If the head sweeps 100 mm/s between poses, one pixel of blur at 0.18 mm/px caps the exposure at 0.18 ÷ 100 = 1.8 ms. Run 1 ms and light the board for it.

The run, step by step

1. Plant the root. Stand the humanoid at its mark, feet flat, balance controller active. If the still test below fails, anchor the machine on a rig or blocks.

2. Still test. Log 30 root-to-head transforms over one minute with everything commanded to hold. Pass: translation spread within 0.5 mm and rotation within 0.05° — 0.05° on a 400 mm lever is 0.35 mm, inside the budget. Fail: fix the anchoring or switch to the external tracker.
3. Freeze the optics. Tape focus and aperture; solve the camera intrinsics from a chessboard now and lock the file — camera intrinsics for hand-eye calibration is the reference for the order of operations.
4. Mount the target. On the end effector for the eye-on-base pattern, or fixed in the world for the eye-in-hand pattern with the head as the hand. Pick one arrangement for the whole run; the board stays flat, rigid and matte.
5. Collect 12 to 15 pose pairs. Change the head pose by at least 30° of rotation between poses on two different axes: neck pan, neck tilt, torso yaw. Log image and head pose at one timestamp — dataset acquisition with saved robot poses shows the mechanics.
6. Solve, then read the rotation content. Poses that all rotate about one axis leave AX=XB degenerate; when the solver fails or returns nonsense, AX=XB solver failure and motion degeneracy lists the fixes in order.
7. Verify with hardware, not residuals alone — next section.

What an untracked torso costs you

The 1–2 cm translation residual people report on humanoids has a specific cause, and it is arithmetic. Say the head sits 400 mm above the root frame your estimator reports, and the estimator's pitch is off by 2° after a lean:

Every pose pair now carries a 14 mm lie about where the head is, and the solver spreads that lie into X: you read centimetre-class translation error and conclude the method does not transfer to humanoids. It transfers; the input was wrong. The same lever with the head tracked to 0.2° gives 400 × 0.00349 = 1.4 mm — millimetre class, where a good run lands.

Decision rule: a translation residual of 2 mm or less, ship it. Ten millimetres or more, stop re-running the solver and fix the head tracking first. For the arm-side causes of the same symptom, read is a 1–2 cm translation error normal?.

Verify the answer before you trust it

Two physical tests, in this order:

Scatter test. Apply X, transform the fixed target into your root frame from ten different head poses, and check the spread. Agreement within 2 mm across poses says the head stream and X are consistent; scatter means the pose stream, not the transform.

Touch-off test. Point the camera at a marker at a measured spot, command the end effector to where the calibrated chain says it is, and measure the miss with a ruler. This predicts production accuracy. Eye-to-hand verification of the camera-to-base result and the accuracy verification run walk through both.

Check units once, early: a pose stream in metres against a solver expecting millimetres produces a thousandfold error that looks like nonsense — the metres-in, millimetres-out mismatch is a five-minute fix.

When the usual advice is wrong

• "Hold the base still" is impossible advice for a working humanoid — and unnecessary. You need a quiet base only while collecting pairs. The head-to-camera constant stays valid however the machine walks afterwards, as long as the mount holds and the optics stay untouched.
• The wrong unknown. You rarely need camera-to-hand directly from the head. Chain three transforms instead: head-to-camera from this run, root-to-head and root-to-hand from the kinematics. One head calibration then serves both arms.
• The wrong pose stream. Feeding wrist poses because the interface labels them robot pose calibrates a camera that is not on the wrist. Match the robot frame to the sample shows the identical failure on fixed arms.
• Walking mid-run. Every millimetre the root drifts goes straight into the pairs. Plant it or track it; there is no third option. And if you do not trust the reported root at all, the hand-eye machinery can be turned around to solve for the kinematic base frame itself from 3D position data — a heavier route, but it exists.

Checklist before you start

Before the run:

• Root planted or anchored; balance controller active
• Still test passed (step 2 thresholds)
• Focus and aperture taped; intrinsics solved and locked
• Field of view covers the board with margin at closest approach (worked example above)

During the run:

• 12–15 pose pairs, rotation change on two axes
• Image and head pose logged at one timestamp, one unit system end to end
• One arrangement only: target on the mover or target in the world, never both

After the run:

• Translation residual at or under 2 mm
• Scatter test inside the same budget; touch-off miss measured and written down

The hardware, in one place

The external-tracker route needs one 3D sensor and one rigid camera stand: browse the camera finder for the tracker and the accessories finder for the mount, or start the system configurator at the mount step. Send head geometry that falls outside the worked numbers — a long neck, a camera far off the last joint axis — through the contact page and we will run the lever arithmetic with you.