Publish an ArUco marker pose as a TF frame for hand-eye calibration

Your easy_handeye launch file asks for two frames the system never publishes yet: tracking_base_frame and tracking_marker_frame. The camera streams and the image topic connect, but the calibration still waits. Nothing in the pipeline has recognized a marker. This guide is for anyone wiring an ArUco tag into a ROS hand-eye calibration on an industrial camera. You will publish the marker pose yourself as a TF frame, size the printed tag against your standoff, set both launch parameters to live frames, and prove the result before the robot moves.

What does tracking_marker_frame mean in easy_handeye?

It is the TF frame your tracking system must publish for the target it sees: for an ArUco tag, the pose your detection node computes for the printed marker, and easy_handeye only subscribes to that frame, never detects the marker itself.

The comment above those two lines in the sample launch says it plainly: fill in the frames your tracking system publishes. easy_handeye consumes two transform chains. The robot side arrives over TF as well, from base to flange. The tracking side is tracking_base_frame to tracking_marker_frame, and something in your system has to supply it. The sample values /optical_origin and /optical_target are placeholders for exactly that pair: the camera's optical frame, and the frame attached to the marker the camera sees. Until a node broadcasts that second frame, the calibration has nothing to solve with.

What do you need before the marker pose is usable?

You need three things: a calibrated camera publishing its intrinsics, a printed marker measured with calipers, and a marker-detection node configured with the right dictionary and the measured size — miss any one and every pose it publishes inherits the error.

The pose estimate converts corner pixels into meters by comparing them with the tag's known geometry, so it inherits every upstream error. The camera's intrinsics — focal lengths, principal point, distortion coefficients — come from a calibration you run once per camera and lens. When the pose looks plausible but distances drift, the pose data is the usual culprit, so fix the intrinsics pose first. Then confirm the driver actually publishes it: validate and publish camera_info covers that check. The same pixel-to-metric reasoning appears in teach-pendant workflows too; a pixel-to-robot coordinate transform walkthrough shows the transform from the robot side.

The printed tag needs one measurement: the side of the black square, taken with calipers. The white border and any laminated margin never enter the size setting.

Any camera whose driver delivers frames and calibration fits the job; the worked example below uses the LS-CMR-0166, a 12 MP USB3 camera with a 1/1.7" rolling-shutter IMX226 that reads 4000 × 3000 at 30 fps.

The detection node, the driver and easy_handeye share one machine without strain; the MV-VC2140-128G40-1T vision controller brings two USB 3 ports and 8 GB of memory to that stack.

How do you publish the ArUco marker pose as a TF frame?

You run a marker-detection node beside the camera driver, feed it the calibrated image stream and the measured marker size, and broadcast the estimated pose as a child of the camera's optical frame, so the calibration finally has its tracking chain.

1. Print the tag in a standard dictionary on matte paper with a white border at least one module wide, measure the black square with calipers, and enter the measured value: a nominal 50 mm tag that prints at 49.8 mm goes in as 49.8.
2. Calibrate the camera for the exact lens and working distance you will use, and keep that lens on the camera for every capture afterwards.
3. Start the camera driver and confirm both the images and the calibration arrive.
4. Start the marker-detection node: easy_aruco exists for exactly this job, and a thin wrapper over the OpenCV aruco module does the same. Set the dictionary, the measured size and the camera optical frame as the parent. If the aruco module is missing from your build, settle that dependency first.
5. Park the tag at a known distance and read the published Z before any robot motion.
6. Set tracking_base_frame to the camera optical frame and tracking_marker_frame to the published marker frame, alongside your robot's base and flange frames.
7. Move the robot through varied stations and record one camera pose and one flange pose per station.
8. Run the solve, publish the result, and prove it with a test move.

Which values go into tracking_base_frame and tracking_marker_frame?

tracking_base_frame takes the camera's optical frame — the /optical_origin placeholder in the sample launch — and tracking_marker_frame takes the marker frame your node publishes, the /optical_target; both must already exist in TF before easy_handeye starts.

Eye-in-hand and eye-to-hand share that tracking pair; the robot side is what changes. Eye-in-hand pairs the camera frame with the flange frame and returns camera-to-flange. Eye-to-hand pairs the camera frame with the robot base and returns camera-to-base. Keep the parent chain your detector publishes rather than renaming frames to taste: most ArUco nodes put the origin at the tag center with Z pointing out of the paper toward the camera, and the solver assumes nothing more than a consistent tree. The same two-chain structure drives hand-eye tools well beyond this stack; a hand-eye calibration walkthrough for a collaborative robot shows the identical pairing on the robot side.

How large should the printed ArUco marker be?

Size the black square so it spans about 1,000 pixels at your farthest calibration pose; with a 25 mm lens on a 12 MP 1/1.7" camera that is roughly 30 mm of tag per 500 mm of standoff, so a 50 mm tag works to about 0.8 m.

The pose quality follows the tag's pixel span, because corner localization carries about one pixel of error and its effect shrinks in proportion as the span grows. At a 1,000-pixel span, a 1-pixel corner error is 0.1% of the geometry; that is the whole basis of the rule.

Worked-example inputs: f = 25 mm focal length, Z = 500 mm standoff, S = 50 mm printed black-square side, p = 1.55 µm pixel pitch — a 1/1.7" 12 MP imager divides its 6.2 mm width by 4,000 pixels.

m = 25 mm ÷ 500 mm = 0.05. The tag images at 50 mm × 0.05 = 2.5 mm, which is 2.5 mm ÷ 0.00155 mm/px ≈ 1,613 pixels, about 40% of the frame width. A 1-pixel corner error is then 1 ÷ 1,613 ≈ 0.062% of the tag geometry, or 0.00062 × 500 mm ≈ 0.31 mm at the standoff. Add print scaling, paper flatness and distortion residuals, and the practical pose error lands in the low millimeters — the range a hand-eye solve can live with.

Rearranged, the rule reads S ≈ 1,000 × p × Z ÷ f. At Z = 500 mm that is 1,000 × 0.00155 mm × 20 ≈ 31 mm of tag, which is where the 30 mm per 500 mm figure comes from, and where a 50 mm tag holds the 1,000-pixel span out to about 0.8 m.

One tag is not the ceiling. A 2023 IEEE study of marker-board layouts measured a 52.96% accuracy improvement going from one marker to nine, and a 563.59% precision improvement, paid for with processing time. When standoff or accuracy targets squeeze a single tag, a small board buys the margin back.

What happens if the marker size setting is wrong?

The reported distance scales by the ratio you typed: enter 60 mm for a true 50 mm tag and every translation stretches by 1.2×, so a real 500 mm standoff reads as 600 mm — a 100 mm error no later step can remove.

The perspective solve carries one free scale, and the size you enter fixes it. Distance in the result is angular size re-expressed in millimeters of assumed tag, so a 20% size error is a 20% distance error at every range. Lateral offsets scale identically: a tag 100 mm off-axis reads as 120 mm. Rotation survives, because angles do not depend on the scale — which is exactly why a wrong size can hide until the robot reaches 100 mm short or long.

Worked through with the formula above: S_set ÷ S_true = 60 ÷ 50 = 1.20, so Z_read = 500 mm × 1.20 = 600 mm. At a true 200 mm standoff the same mistake reports 240 mm — the absolute error grows with range while the percentage stays fixed.

Measure the black square once and type that number. A careful print lands within about 0.4 mm of nominal, 0.8% on a 50 mm tag, worth 4 mm at 500 mm — small but not free, and the caliper minute removes it. When published distances already disagree with a ruler, walk the fiducial pose distance checks before blaming the calibration.

How do you verify the frame before you run the calibration?

Check two things before any robot motion: park the marker at a measured 300 mm and confirm the published Z lands within a few millimeters, then tilt the tag 45° and confirm its Z axis still points back at the lens.

Three distance checks catch most faults. Measure 200 mm, 300 mm and 400 mm with a ruler and confirm the published Z tracks each within a few millimeters. A systematic offset that grows with range points at the intrinsics; a constant offset points at frame wiring, and the known-distance Z correction pins that one down.

Orientation and stability come next. Tilt the tag 45°: its Z axis should keep pointing at the lens. Then hold everything still for ten seconds and watch the numbers; expect wobble in the tenths of a millimeter, and half a millimeter or more usually means blur, so shorten the exposure and add light.

After the solve, verify end to end with a commanded test move, or run the full accuracy-verification procedure before trusting the result. When the final translation error lands near 1-2 cm, treat it as a hand-eye integration outcome rather than a marker problem — the tag contributes tenths of a millimeter, and the guide on whether 1-2 cm of translation error is normal walks that gap.

When is an ArUco marker the wrong target?

Swap the target when the standoff approaches the lens's minimum focus, when print flatness or glare bends corners, or when you need better than roughly 1 mm of marker-pose accuracy — boards with more corners beat a single tag on every count.

Close range is the first case, and geometry says when: field width is 6.2 mm × Z ÷ f, so the 50 mm tag stops fitting that 25 mm lens inside about 200 mm, and tight stands become a target-and-standoff design problem of their own.

Glossy paper, curved surfaces and unflat prints bend corners in ways the solve reads as pose error; matte stock and a rigid backing cost nothing. When the accuracy target sits near 1 mm or better from the tag alone, add corners per image: the nine-marker board from the study, or a checkerboard-marker hybrid, gives the solver more constraints in the same frame.

Pose capture on the move adds one more requirement: the whole frame must expose at once. The MV-CH120-11UC is a 12 MP USB3 camera with a global shutter and a high-speed readout, so fast robot motion does not skew its corners the way a rolling shutter does.

Which camera and controller run this stack?

A 12 MP USB3 camera and a fanless controller with spare USB 3 ports carry this stack, and both sit one category away: start from the area scan cameras and vision controllers ranges, then let the configurator check mount, cable and trigger choices.

Once the calibration is published, the same marker frame keeps earning its keep: it anchors ArUco pick-and-place grasps and reports object poses in the base frame at runtime, so the tag you calibrated with becomes the tag the cell works from.

To size the hardware against your standoff and cycle time, filter the camera finder by interface and resolution, step through the camera configurator, or browse the systems topic page for the rest of the calibration chain; an industrial mini PC collection helps if you build the host yourself. If the numbers refuse to settle, bring the setup to the contact page and we will work it through with you.