Use eye-to-hand calibration results to compute robot-base picks

You have a finished eye-to-hand calibration: a translation and a quaternion that describe where the camera sits relative to the robot base. The remaining work is to publish that result as a static transform, apply it to every detection your vision software produces, and prove the result before the first real pick. This article covers all three steps for a fixed camera looking at the robot's workspace, and points out the failures that cost the most time. If your calibration is not done yet, start with robot hand-eye calibration: how nine-point alignment works.

What the calibration result gives you

The solver returns a homogeneous transform: a rotation, often as a quaternion, plus a translation vector. Confirm its documented direction before using it. The equations here require the pose of the camera's optical frame expressed in the robot base frame. If your solver returns the inverse relationship, invert that transform first. This is a fixed relationship, unlike eye-in-hand where the camera moves with the flange.

Here, point coordinates are homogeneous column vectors. A depth camera commonly reports points in an optical frame with Z pointing forward out of the lens, but confirm your software's convention. The calibration matrix converts those coordinates into a frame the robot controller understands. If your point cloud looks wrong after transforming, check whether you used the optical frame or the camera housing frame; they can differ by a fixed rotation. The article on how to transform depth point clouds between optical frame and camera frame covers that distinction in detail.

Step 1: put the result into your frame tree

Store the calibration where your transform broadcaster reads it. In a ROS-style stack, this is typically a parameter file specifying the fixed-camera configuration, the camera optical frame, the robot base frame, and the transform values. Parameter names depend on the calibration package. Two things to get right:

1. Select the package's documented eye-to-hand configuration and use a namespace or prefix that distinguishes it from an eye-in-hand result.
2. Point the launch file explicitly at your saved result rather than relying on the default output path, or state the values directly in the file the publisher loads.

Then publish the camera optical frame as a static child of the robot base frame, using the camera pose expressed in base coordinates. Do not also publish the same child frame from another broadcaster. If lookups fail after startup, inspect the frame tree and publisher configuration; missing camera frame tf transform: fix the frame tree walks through that diagnosis.

Step 2: transform the detection into base coordinates

Your vision stage produces an object pose: a position and orientation in the optical frame. Chain the transforms in one lookup:

The second equation assumes the desired pick point is already defined in object coordinates. For a complete commanded tool pose, also include the object-to-grasp transform and the controller's tool-centre-point convention.

For picks from bins or pallets, the object pose often comes from a 3D sensor rather than a 2D camera alone. Choose among the 3D sensors according to working range, surface behaviour, occlusion, and the pose data your grasp planner requires. Transform the resulting points or poses into the base frame. Keep the orientation conversion in mind: the detected object's rotation must also be composed with the camera-to-base rotation, and roll-pitch-yaw conventions differ between vision libraries and robot controllers.

Step 3: verify with a touch-off

Never trust a calibration chain on its first commanded move. Use the robot manufacturer's commissioning procedure and the cell's risk-assessed safeguards. Keep personnel outside the hazardous motion area; reduced speed alone does not make a move safe. Use noncontact verification where physical contact could damage equipment or create a hazard.

1. Place a calibration target or a marked block at an independently measured base-frame position, at the same height as your real picks.
2. Detect it with the camera and compute the pick pose. Compare the coordinates numerically before commanding motion.
3. Select an approach point with clearance from the target, tooling, and surrounding equipment. For example, 50 mm above the target may be suitable only after the geometry and approach path have been checked.
4. Using the documented commissioning mode and safeguards, approach in controlled increments. Observe the tool centre point relative to the mark without entering the hazardous area.
5. Record the position and orientation errors. Repeat at several locations and heights, then compare the worst error with your application's acceptance limits.

This same verification procedure is what you repeat after any cell change; the article eye-to-hand calibration: verify the camera-to-base result covers the verification math in depth, and vision calibration recovery after camera or robot changes covers what to redo after a crash or a moved bracket.

Worked example with numbers

Suppose the solver returned a translation of roughly (130, 1460, 1170) mm and some rotation between base and optical frames. A detected part sits at (0.20, -0.05, 1.10) metres in the optical frame. First convert the point to (200, -50, 1100) mm. Applying the rotation to that vector and adding the translation gives the base-frame position.

With a purely illustrative identity rotation, the arithmetic is (130 + 200, 1460 - 50, 1170 + 1100), giving approximately (330, 1410, 2270) mm in base coordinates. The rotation term changes these numbers in a real cell. A sign, axis, or transform-direction error can produce a grossly incorrect target rather than a small offset you can tune out.

For a planar image, estimate lateral sampling from the field width and image width. A 400 mm field spanning 1600 pixels gives 400 / 1600 = 0.25 mm per pixel at that plane. This is a sampling estimate, not a depth-accuracy specification. Use the field-of-view calculator to explore the optical geometry:

Hardware notes for the cell

A 3D camera that streams dense clouds at pick rates produces real processing load. Choose a controller with compatible camera interfaces, supported software, and enough measured processing capacity for the rest of the cell. The MV-VC3301H-128G66 and MV-VC5525-256G70-NN are vision-controller options to compare. Validate your actual acquisition and pose-estimation workload before selecting either.

Browse the full range in the vision controllers category, and the 3D sensors category for the camera side. Compare the cataloged controller specifications here:

All 2 share: Humidity 20% RH to 95% RH (no condensation); Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

If you also run an HMI or cell controller next to the vision machine, the eye-to-hand calibration and XY tablet mapping guide covers how calibration mapping is configured on the controls side.

Common mistakes that produce a wrong pick

• Wrong frame parent or transform direction. Publish the camera optical frame under the robot base using the camera pose expressed in base coordinates. Print the frame tree and verify the matrix direction separately.
• Optical frame versus camera frame. The optical convention can differ from the housing convention; mixing them rotates your picks by that offset.
• Conflicting or unreliable publishing. A bolted camera normally uses a static transform. Remove duplicate broadcasters and confirm the relationship remains available after restart.
• Height-dependent error. A planar image mapping does not generally transfer to another height. For a full 3D system, investigate depth scale, intrinsics, and calibration geometry rather than assuming the rigid transform is valid only on one plane.
• Units mismatch. Millimetres from the robot controller and metres from the vision side differ by a factor of 1000. Convert explicitly before composing translations.
• Incomplete orientation conversion. Applying only the translation leaves the grasp angle wrong even when the position appears plausible. Compose rotations and verify the controller's angle convention.

For cells where the robot carries the camera instead, the frame chain changes completely; see eye in hand calibration pose selection: robot positions that work for that variant.

Quick checklist before the first production pick

• Calibration result saved with the documented fixed-camera configuration and an identifiable namespace.
• Static transform published and confirmed after a full restart.
• Detection pipeline confirmed to output poses in the expected optical frame.
• Transform lookup and multiplication direction verified from camera coordinates into base coordinates.
• Object-to-grasp transform and tool-centre-point definition checked.
• Verification performed across the working area and relevant heights, using the cell's commissioning safeguards.
• Position and orientation errors recorded against application-specific acceptance limits.
• Reverification trigger defined for any camera, robot, tooling, or mounting change.

When the usual answer is wrong

If the verification is repeatable but consistently offset, do not edit calibration values by hand to hide the error. Investigate frame conventions, transform direction, tool-centre-point definition, and calibration bias. If results vary, also examine part presentation, depth noise at your working distance, timing, mechanical movement, and robot repeatability. Calibration quality can contribute to either pattern, so use measured tests rather than ruling it out from one symptom.

Fix the layer where the evidence points, and use the camera pose reference-frame guide when frame definitions are unclear.

Ready to specify the camera and controller for the cell? Start from the 3D sensors category and the camera finder, or contact us through the contact page for help matching parts to your cycle time.