Hand-eye target detection topic: pick the raw camera feed

A hand-eye calibration that never finds its target is usually two mistakes stacked: the Target Pose Detection panel is subscribed to the wrong image topic, and the target itself is too small to hold a stable pose. This guide is for anyone wiring up eye-in-hand or eye-to-hand calibration with a board-level camera and a printed marker. You will fix the Image Topic and cameraInfo Topic pair, learn why the Object_frame list on the Context tab stays empty, size a ChArUco board to the pixels you actually have, and capture a pose set the solver can use. The arithmetic that decides it: a single ArUco marker offers four corners, a printed ChArUco board still offers sixteen with half of it hidden, and that gap is what turns noisy marker poses into a calibration fit for production.

Why is the hand-eye target not detected?

In most cases the Target Pose Detection panel is listening to its own annotated output instead of the camera's raw feed. The detector then receives a picture that already shows no target and never runs a fresh search on live camera data.

The symptoms look like a target problem: the plate is plainly in view, the panel runs, nothing is ever detected, and the Object_frame list on the Context tab stays empty. Rank the causes by how often they turn out to be the culprit:

1. Image Topic points at the annotated output — the detector is being fed its own result image, which contains no new scene data.
2. cameraInfo Topic belongs to a different stream or resolution, so the intrinsics do not match the pixels being searched.
3. Too few corners: a single ArUco marker, or a board sitting so small in frame that corner localization is unstable.
4. Glare, blur or drifting exposure: a glossy print, a rolling-shutter frame taken while the arm settles, or auto-exposure re-balancing between poses.
5. A print that is not flat, or a plate that flexes in the gripper.

Work down the list in order; the first two items take minutes and fix most runs. One naming note first: eye on hand and eye in hand describe the same set-up — a camera carried by the wrist — opposite to a fixed camera watching the wrist, and the wiring below is identical for both. Only the transform you solve for changes, as the eye-in-hand vs eye-to-hand comparison shows.

Which image topic should the target detection tool subscribe to?

Point the Image Topic field at the raw color stream your camera driver publishes — the topic that shares the /camera/color/ prefix with your /camera/color/camera_info stream — because /handeye_calibration/target_detection is the tool's own output image with detected corners drawn on it, never an input.

The annotated topic exists so you can see what the detector saw. Feeding it back creates a loop in which no detector ever receives camera data, which is exactly what an empty detection run looks like from the outside. To find the right name, list the topics your camera driver publishes, open the raw image topic in an image viewer, and confirm you see the live scene at full resolution; then enter that exact string in Image Topic. Keep the pair matched: cameraInfo Topic must describe the same stream you subscribed to, because the detector maps pixel coordinates onto the target through those intrinsics. If the image is cropped, scaled or binned anywhere in the pipeline, the intrinsics stop matching and detection fails even with a perfect board in view.

Why is the Object_frame dropdown empty on the Context tab?

The Object_frame list fills only after at least one detection succeeds and writes a target pose; an empty dropdown is the symptom of the failed detection upstream, not a setting you can pre-fill by hand.

So resist reconfiguring the drop-down. Fix the image input and the target, capture one confirmed detection, and the frame appears in the list ready to select. If the list is still empty after a detection you can see succeeding, the frame name was lost between the detector and the store, and the hand-eye target frame checklist walks the remaining causes in order.

How large should the calibration target be in the frame?

Size the board to span at least a quarter of the frame width — a third if the working distance allows — because corner error at the target equals pixel footprint times corner noise, and pixel footprint grows in direct proportion to working distance.

The footprint math is short. Board cameras such as the LS-BCM-0056 put 1.6 μm×1.6 μm pixels on a 12 MP, 1/2-inch sensor, so one relation covers the sizing question:

Take an 8 mm M12 lens, a 300 mm working distance and that 1.6 μm pitch. The footprint at the target is 0.0016 mm × 300 / 8 = 0.060 mm per pixel, so a 40 mm wide board spans 40 / 0.060 ≈ 670 pixels. Corner repeatability of 0.1–0.3 px — typical for sub-pixel refinement on a sharp matte print — puts the corner coordinates within 6–18 μm at the target plane, well under the millimeter tolerances a gripper cares about.

Push the same rig out to 800 mm and the footprint grows to 0.16 mm per pixel. The board drops to about 250 pixels and corner noise triples to 16–48 μm. Working close is the cheapest accuracy gain available, and the relation runs backwards too: from the field of view you want, the focal length calculator returns the lens.

Leave margin around the board in every pose. A board clipped by the frame edge at a tilt angle loses its corners exactly when tilt matters most; partial views of a ChArUco board are tolerable, full views are better. Board dimension errors propagate into distance errors in a diagnosable way, and print flatness is the other half of accuracy — the ChArUco dimensions guide and the flatness guide cover both.

Is a single ArUco marker accurate enough for hand-eye calibration?

No — four corners cannot hold a pose steady: a single ArUco marker hands the solver four points, every frame re-rolls a few tenths of a pixel of noise onto all four, and those errors flow into the hand-eye transform as millimeters of offset at the tool flange.

Corner count is the lever you control. Hide half of each target and the difference is stark:

The averaging improves as the square root of the point count: sixteen points against four halves the random part of the pose error, and thirty-six against sixteen cuts it by roughly a third more (√(16/4) = 2, √(36/16) = 1.5). The same printed area spent on a ChArUco layout therefore beats a lone marker at every distance. Print the board matte, glue it to a rigid plate, and measure the finished square size before entering it in the tool.

How do you capture calibration poses the solver can use?

Capture 15 to 25 poses that vary wrist tilt by at least ±30° and camera-to-target distance by at least ±20%, lock exposure and gain before the first shot, and let the arm settle completely at every station, because a moving or auto-exposing camera writes skew and flicker into the corner coordinates.

Then run the capture as a fixed routine:

1. Fix the input: Image Topic on the raw camera stream, cameraInfo Topic matched, live view confirmed at full resolution.
2. Mount the board rigidly, bolted to a stand or fixture and never held by hand inside the cell; enter the measured square pitch in the tool.
3. Lock exposure and gain. Auto modes re-balance between stations and the corners shift with them.
4. Park station one: board centered, spanning about a third of the frame width, arm at rest for a full second before capturing.
5. Sweep the wrist through tilts of ±30–45° about two axes and vary distance ±20%, keeping the board fully inside the frame; the pose-selection guide lists robot positions that produce usable detections.
6. Inspect every detection before accepting it, discard blurred or glared frames, and re-shoot that pose rather than letting a bad image into the solver — the bad-image decision guide gives the criteria.
7. Run the solver on the accepted set; nine well-spread positions can already solve the transform, and nine-point alignment explains why placement matters more than quantity.
8. Read the residuals before saving anything — the acceptance numbers are in the next section.

Which camera and stand hardware makes hand-eye capture repeatable?

Two pieces of hardware decide whether captures repeat: a global-shutter board camera on the wrist, because a rolling shutter smears any frame taken before the arm is truly still, and a rigid stand that parks the target plate at a standoff you can re-create within a millimeter.

On the wrist side, the board-level cameras category is where to look. The LS-BCM-0003 is a 3.1 MP USB3 board camera with an IMX252 sensor in a 1/1.8-inch format, global shutter and 120 fps. Global shutter is the property that matters here — every pose snapshot lands in one instant — and the frame rate lets you grab several shots per station and keep the sharpest.

Tighter wrists call for smaller optics. The MV-CB016-10GM-M-W is a 1.6 MP GigE board camera with an IMX296 sensor, mono, global shutter and an M12 lens mount. The M12 mount keeps weight down on a moving flange; format, focal length and IR-cut choices for M12 optics are covered in the M12 lens guide.

On the target side, bolt the plate to something that never moves. The camera stands category splits into slide tables, tilt stages and rotary stages for exactly this job, and the accessories finder filters them by type. The LS-MNT-0040 is a professional slide-table stand with a 700 mm column and a 400 x 450 mm base, enough height for a typical cell standoff and a slide that re-positions the target without re-clamping.

To pre-check corner detection across tilt without powering the robot, the LS-MNT-0001 slope-angle slide tilts the plate on the bench, and the LS-MNT-0220 adds a 60 mm XY-plus-rotation stage for lateral offsets.

How do you verify a hand-eye calibration before production?

Accept the calibration when the average reprojection residual lands sub-pixel and a physical pick repeats within a couple of millimeters — the tool's numbers plus one real test with a part in the gripper, never the numbers alone.

Run the acceptance list every time:

• Residuals against targets: compare average and maximum reprojection values with the bar set in the residual values guide; an average above it means the capture set, not the solver, needs work.
• Reprojection overlay: re-draw the detected corners on the raw images and look for a systematic drift across poses — a trend means a bent target or a loose mount, not noise.
• Physical pick test: command a pick at three positions across the workspace and measure the offset each time; a repeating offset points at the transform, a scattered one at detection quality, and the pick-offset diagnostic separates the two.
• Eye-to-hand variant: with a fixed camera, run the same acceptance on the camera-to-base result — the verification steps differ only in which transform you test.
• Production logic: locate the part, then locate it again relative to the gripper — the two-locate pattern that camera pick-and-place guidance argues for in production cells.

If the hardware side still needs parts, the camera finder filters board cameras by shutter, mount and interface, and the system configurator walks the camera-to-lens pairings; a second opinion on stand or lens choice is one message away on the contact page. The rest of the hand-eye failure modes, from frame losses to solver crashes, sit on the systems topic page.