Point cloud not on robot base plane after hand-eye calibration

Your hand-eye calibration solved cleanly, the pose looks sensible in the tree, and still the transformed point cloud floats above the robot base plane — or sinks below it. That combination almost never means the solver failed. It means the transform chain you applied does not match the frame the cloud was built in. This guide is for anyone wiring an eye-to-hand 3D camera to a robot. You will run one flat-target test that localizes the fault, read its three numbers, and repair the chain — usually by adding one in-camera extrinsic — until the plane fit lands inside a millimetre of the truth.

What the pattern of the error tells you

A calibration that solves cleanly but drops the cloud off the plane is a frame-chain fault far more often than bad data. Name the fault from the shape of the error before you touch the solver. Capture once, transform into the base frame exactly as your production code does, look at any flat surface, and match what you see to a row here.

Rows are ordered by how often they explain the symptom. The first row covers most of the cases we see: the solver read the color image, and the cloud is built in the depth frame.

The flat-target test: three numbers

You need a depth camera that outputs a calibrated point cloud with its intrinsics — a projected structured-light camera such as the MV-DPS200P-02 qualifies — plus a flat part of known thickness and a plane-fit routine from your point-cloud library.

1. Lay the flat part on the surface the robot base is bolted to — the base plane itself, not a nearby bench. Note the fitting height: a 10.0 mm gauge plate puts the test surface at +10.0 mm.
2. Park the robot out of the field of view and capture one cloud.
3. Transform that cloud to the base frame through the exact chain your production code uses.
4. Fit a plane to the part's top surface and record three numbers: mean z, the tilt of the fitted normal against the base z-axis, and the in-plane angle of any machined edge.

At a 1,000–1,500 mm standoff with a structured-light camera on a rigid bench, work to this band: mean z within ±1 mm of the part's top height, tilt below 0.2°, a machined edge aligned within 0.5 mm and 0.5°. Laser profilers land well inside it. Miss the band, and the signature in the table names the section to read.

If your current camera cannot output a calibrated cloud, the 3D camera finder and the 3D sensors category are where to look.

Cause 1: the depth-to-color extrinsic is missing

Most solvers detect the target in the color image, so the calibration result is the color frame's pose, T_base←color. The SDK builds the cloud in the depth optical frame. Inside the camera head, a fixed, factory-measured lever arm separates those two frames — the depth-to-color extrinsic. Omit it, and every point carries that lever arm as error. Overhead, the arm lies along the optical axis, so nearly all of it lands on z.

Publish the composed product instead of the bare result. The multiplication order follows the frames left to right, the same discipline as moving point clouds between the optical and camera frames. A missing in-camera rotation hits harder — the cloud collapses in one axis pair — and that variant is the flat-in-XY optical-frame fix.

Whatever your camera's real lever arm, the diagnostic holds: a residual equal to the arm is a frame fault, not a calibration fault. Read the arm from the camera's extrinsic data and use that number, not a guess.

Cause 2: the transform is applied in the wrong direction

Displacement of about twice the standoff means inversion. With the rotation part at or near identity, the correct map adds the translation, p + t, and the inverse subtracts it, p − t. The gap between the two results is 2t, so a 1200 mm standoff produces a 2400 mm error.

Check one point by hand. Multiply the plate centre through your stored matrix in a script and compare with what the pipeline prints. If the hand result matches the pipeline and both are wrong, the stored matrix is the inverse of the intended one, or the multiplication order is: rotate-then-translate order shows that arithmetic, and a point cloud rotated 90° after hand-eye calibration is the rotation-only cousin of the same fault.

Cause 3: a fixed camera parented to a moving link

Eye-to-hand bolts the camera to the cell, so base←camera is one constant. A publisher left over from an eye-in-hand trial attaches the camera to the flange or tool frame instead, and every jog then drags the cloud with the arm.

The test takes two captures: shoot the plate, jog the robot 100 mm, shoot again. A plane fit that moves between captures has a moving parent. Publish base←camera as a static transform, the way publishing the eye-to-hand result as a static transform sets it out, then repeat the jog; the fit should hold to the noise floor. Cameras that genuinely ride the wrist follow the other pattern — the eye-to-hand set-up for a camera above the robot base and the eye-in-hand input check separate the two cases.

Cause 4: depth scale and unit slips

A unit slip is loud: metres parsed as millimetres put the plate 1000× away, and nothing fits anywhere. The quiet version is a scale constant applied to the raw depth twice, which stretches or shrinks z alone; the cloud then disagrees with a tape measure by a few percent while the image looks healthy. Measure the standoff, read the cloud's distance to the plate centre, and expect agreement within 2–3 mm at a 1200 mm standoff. The depth scale mismatch fix walks that constant, and a plane that bows instead of offsetting belongs to the bowed-plane diagnosis.

Fix the chain, then re-verify

1. Write the chain on paper: cloud frame → in-camera frames → base, naming the frame each transform actually connects.
2. Compose from the cloud's frame to the base in a single expression, ordered along those frames.
3. Publish the result as one static transform for an eye-to-hand cell.
4. Re-run the flat-target test against the pass band above.
5. Add the lateral check: seat a dowel pin at a known base position — (250.0, −180.0) mm works — and read the pin centre from the cloud. Agreement within ±0.5 mm clears x and y. A plane test cannot see lateral offsets; skip this and the first production pick will find the error for you.

Chaining is desk work; repeated capture sessions are not. A rig that runs the camera SDK, the static publisher and a viewer at once, with separate ports for the 3D camera, the robot controller and the PLC, is the brief of a vision controller.

New to the cell? The configurator walks the parts in order, starting at the camera: system configurator, camera step.

When it really is the calibration

Chain correct, and the plane still misses with a tilt? Then the pose set fed the solver. Re-run with 10–20 robot poses, tilt the target through 30–60°, fill all four corners of the field of view, and hold the target rigid between captures. A solver reporting in-plane residuals of a few tenths of a millimetre is delivering usable data; residuals in the millimetre range point at blurred or shifted captures.

Check the datum under the plate too. A plate bolted to the table, with the base frame taught to a fixture datum, inherits that difference into every reading. Re-teaching a base frame is its own procedure — how a body-in-white line re-teaches robot base frames shows it at production scale. If the matrix came out of an offline hand-eye tool, paste the full 4×4 rather than re-entering Euler angles; an offline hand-eye calibration walkthrough shows the result you should be copying.

A five-minute checklist before blaming the solver

• The cloud's own frame is named: depth, IR or color, exactly as the SDK builds it.
• One expression carries the cloud from its frame to the base, ordered along the frames.
• The in-camera extrinsic is in the chain whenever the solver's stream differs from the cloud's frame.
• base←camera is static and parented to the base, not the flange, for eye-to-hand.
• Cloud standoff agrees with a tape measure within 2–3 mm.
• Plane fit passes: ±1 mm mean, 0.2° tilt, edge within 0.5 mm and 0.5°.
• The dowel pin reads within ±0.5 mm in x and y at its known base position.

Pass the plane and the pin, and the transform is production-grade. The same chain then computes picks directly — using eye-to-hand results to compute robot-base picks shows how — and verifying the camera-to-base result adds an independent second check. More frame and calibration fixes sit on the 3D vision topic page, and a second opinion on a specific cell starts at the contact page.