ArUco Z axis distance error: fix the marker solve first

A few centimeters of error along the Z axis is the classic symptom of a monocular ArUco solve feeding a hand-eye calibration: X and Y look plausible, the extrinsic result lands short or long, and nothing in the solver output explains it. The solver is rarely guilty — the inputs are. The suspects are the marker side you typed in, the intrinsics file, the frame you sampled, and the optical center the pinhole model measures from. This guide gives you a five-minute robot test that isolates tracking, the scaling arithmetic that turns a ruler measurement into an expected error in millimeters, and the decision point where a printed marker stops being the right tool. The surrounding hand-eye workflow lives in the 3D vision guides; this page covers the marker-solve layer underneath it.

Why does ArUco pose show centimeters of error along the Z axis?

Treat the solver as innocent until the inputs are proven: with a marker 358 px wide at a 700 mm standoff, one pixel of corner error moves Z by only about 2 mm. A 30-80 mm error must come from a wrong input rather than from detection noise.

Ranked by how often each explains the symptom:

1. Marker side mismatch. The value entered in the detector differs from the printed black square. Distance scales one-to-one with that number.
2. Weak intrinsics. A calibrated focal length off by 2% shifts every reading by 14 mm at a 700 mm standoff; uncorrected distortion near the frame edges adds more.
3. Frame mismatch. The pose sampled during calibration is not the pose published at runtime.
4. Capture conditions. Motion blur, rolling-shutter skew or an oblique viewing angle bias the corner positions.
5. The optical-center offset. The pose Z is measured from the pinhole origin near the lens mount, not from the front ring — expected behavior, not a fault.

Work down the list with the tests below; the first three account for nearly every centimeter-scale case.

How do you test marker tracking with a 10 cm robot move?

Leave the marker fixed on the bench, keep it tracked, and move the robot 10 cm straight up with the pendant. The marker's pose in tf must shift by 10 cm along the camera axis, repeatable to about 2-3 mm, before any calibration run is worth recording.

1. Tape the marker to a rigid, flat backing on the bench, face-on to the camera, and confirm the pose is stable while nothing moves.
2. In manual mode at reduced speed, jog the robot straight up by that distance and read the marker translation from tf before and after.
3. Repeat the move toward and away from the camera, and left and right. Every commanded move must appear at the same magnitude in tf.
4. Finish with a rotation check: turn the flange 90° about the tool axis and confirm the marker pose rotates by about 90° about the viewing axis.

A pass clears the marker solve and moves suspicion to the frame handling and the robot side. A fail means the solve itself is off — the next three sections cover its inputs. Reading the pose at all depends on the marker being published correctly; the ArUco-to-tf publishing workflow shows that frame setup.

Does the configured marker size match the ruler measurement?

Measure the printed black square edge to edge and enter exactly that value in millimeters. Distance scales one-to-one with the configured side, so a marker printed at 54.0 mm but entered as 60.0 mm turns a true 700 mm standoff into a 778 mm reading.

The arithmetic, with illustrative numbers for a 12 MP, 1.1-inch-class sensor (3.45 µm pixels) and a 16 mm lens, where S_real is the measured side and S_set the configured side:

Three habits make this check conclusive: print at 100% scale with page-fitting off, measure after mounting with calipers or a steel ruler, and re-measure whenever the marker is reprinted, relaminated or remounted. Enter the side of the black square, not the white quiet zone around it. The same ratio corrupts every axis of the pose, but Z exposes it first because distance is the largest component. The print-and-measure procedure for calibration markers covers the measurement itself.

Are the camera intrinsics good enough for depth?

Recalibrate intrinsics until the reported RMS reprojection error sits near 0.3 px. Below about 0.5 px a monocular pose is trustworthy at bench distances, while above roughly 1 px expect percent-level translation bias of 7-14 mm at a 700 mm standoff.

Depth inherits every intrinsic error directly, because the distance is computed from the ratio of focal length to marker width in pixels. Uncorrected barrel distortion shrinks the apparent marker toward the frame edges, where distortion is strongest, so keep the marker inside the central two-thirds of the image during capture and calibration alike.

Rebuild the file with a flat chessboard or ChArUco target: 20-30 images, the target tilted through at least ±30°, at distances spanning the working range, camera and target both at rest for every frame. Then repeat the tracking test — if it passes with the new file, the old intrinsics were the cause. The intrinsics-first fix for hand-eye calibration covers the target set-up, and the known-distance check on Z translation catches a skewed principal point with a single tape-measured pose.

Why is the pose Z measured to the optical center, not the lens front?

The pinhole model measures from the effective optical center, which sits inside the lens tens of millimeters behind the front ring for a typical C-mount objective. Pose Z will never match a tape measure run to the lens front — and it does not have to.

What matters for the hand-eye result is that the same camera frame is used at calibration and at runtime. The solve estimates one transform from the robot flange to that frame, so the constant offset from optical center to housing is absorbed into the extrinsic and never appears as error.

Two things do turn the offset into error. Refocusing moves the principal plane: a lens re-adjusted between calibration and runtime shifts the effective center by millimeters, and millimeters here are exactly the symptom. Swapping the lens for one with a different pupil position invalidates the extrinsic outright. Lock focus with tape or a setscrew once focus is set, repeat the tracking test after any lens handling, and treat the calibrated lens as part of the calibration. If the distance reads wrong even on the bench, the fiducial distance checklist works the same ground from the pose side.

Are you sampling and publishing the same transform frame?

The extrinsic is only valid between the two frames you actually sampled. A solve that recorded flange-to-camera-A and a runtime that publishes camera-B-to-marker produce a robot that misses mostly along Z, by exactly the offset between the two camera frames.

Check three things with the calibration output in hand. First, the camera frame named in the solve must be the one the marker publisher writes into its transform messages: the camera convention used by tf browsers (X forward) differs from the optical convention (Z forward out of the lens), and reading the Z component from the wrong one hands you another axis's distance. Second, the robot frame sampled must be the frame your motion program commands; an end-effector frame that rviz shows differently from the physical gripper is a robot-kinematics problem, not a vision problem. Third, nothing may republish or re-parent the transform between capture and use.

A constant orientation offset at the gripper is harmless — the solve estimates the marker-to-flange transform anyway. What corrupts samples is a robot whose reported flange pose disagrees with reality. The runtime object-pose frame check and the robot-frame matching guide cover both sides, and the same frame rule holds in offline robot programming workflows, as in this camera-to-robot hand-eye calibration walkthrough.

Which capture settings keep marker corners within a pixel?

Shoot with the robot at rest, the marker face-on within 30°, and the exposure short enough that blur stays under a pixel. At a 700 mm standoff a 100 mm/s approach with a 16 mm lens smears corners by about 0.7 px per millisecond of exposure.

The blur arithmetic is direct — image-space speed equals focal length in pixels times robot speed divided by distance: 4,638 px × 100 mm/s ÷ 700 mm ≈ 660 px/s, so 1 ms costs 0.7 px and 10 ms destroys the solve. Capture between motion commands or budget the exposure accordingly. Viewing angle matters too: as the marker tilts off-normal, depth uncertainty grows roughly with the inverse cosine of the tilt, so treat 30° as the planning limit and 45° as the edge of usability.

Stop the iris down for depth of field, but not into diffraction. With a 2-pixel circle of confusion at 700 mm, F5.6 holds focus from roughly 630 to 780 mm — enough for a 150 mm pose spread. Past about F11 the diffraction blur alone exceeds 4 px on 3.45 µm pixels and softens every corner. The aperture and depth-of-field guide has the full model, and the diffraction limit explainer shows where the pixel-level floor sits. Light the print with even, diffuse illumination from the lighting finder and use a matte print — glare clips corners before blur does.

Pick the focal length and standoff together so the marker lands near 400 px wide. At 700 mm the 16 mm example covers a 618 × 453 mm field, leaving the marker inside the low-distortion center with margin for the pose spread.

A global-shutter, 12 MP camera on USB3 such as the MV-CH120-11UC fits this job.

The 16 mm example focal length matches the LS-FFL-0567, a C-mount lens covering 1.1" sensors and printing <0.3% distortion.

When does a printed marker stop being enough?

Move to a depth sensor when you need absolute centimeter accuracy on position, when nothing flat and matte can ride on the part, or when the tracking test keeps failing after size and intrinsics are fixed. Structured light measures depth directly, so marker scale leaves the error budget entirely.

A monocular marker solve is a repeatability instrument. Given a good intrinsics file it returns the same pose for the same scene to within a millimeter or two, which is what a hand-eye solve needs; what it cannot deliver is an absolute depth guarantee independent of the print. For bin picking, depalletizing and large workpieces, a stereo or projected structured-light camera returns depth per pixel and removes the printed-reference assumption. The 3D sensors category spans laser profilers, stereo cameras and logistics depth cameras, and the camera finder filters them by type.

A projected structured-light 3D camera such as the MV-DPS200P-02 measures depth directly, so the printed marker becomes an optional convenience instead of the measurement source.

Decision, case by case:

• Stay with the printed marker when poses are relative, the marker can sit flat, lighting is controlled, and the standoff is 0.3-1.5 m.
• Move to depth sensing when absolute accuracy on the part matters, surfaces are curved or reflective, or ambient light swings.

What do you re-check before trusting the extrinsic result?

Re-run the three input checks after every change — the axis test, the ruler against the configured size, and the reprojection error near 0.3 px — and record all three with the calibration file so the next drift has a baseline to compare against.

Prevention list:

• Lock lens focus mechanically and leave it untouched after intrinsics and extrinsics are captured.
• Re-measure the marker after any reprint, relamination or remount; the measured value goes into the detector configuration, not into memory.
• Gate acquisition on robot-at-rest, so no frame is accepted while the arm is still moving.
• Write the sampled frame names into the calibration notes and assert them at runtime.
• Repeat the tracking test after remounting the camera or touching the lens.
• Close with a verification move: command a measured 100 mm approach and compare the pose the chain returns; the accuracy verification procedure defines the pass criteria.

To spec the camera and lens pair for a marker cell, start at the camera configurator, compare area scan cameras and lenses for your standoff, or reach us through the contact page.