Camera mounted on the gripper: which tool-frame offsets actually work

Flange space is scarce. Between the gripper, its air lines and the part-present sensor, the natural home for the camera is the gripper body — behind or beside the jaws, looking at the pick point. Whether that works is not a mechanical question but a mathematical one: which offset from the tool frame does your vision system accept? Many wrist-camera workflows take a distance along the tool Z axis and nothing else. This guide gives you the arithmetic for what a side offset costs when you enter Z alone, a two-pose test that catches the error before the first production pick, and the hand-eye calibration that recovers the full offset when you keep the side mount.

The decision in three lines

• Camera on the tool axis, looking straight down it. Enter the measured Z distance; the standard wrist-camera workflow is then exact. Mount it on the gripper and move on.
• Side offset or tilt, and your software accepts only Z. Either re-mount the camera so the offset becomes pure Z, or keep the mount and recover the full camera-to-tool transform with your own hand-eye calibration.
• Side offset or tilt, and you enter Z anyway. The un-modelled offset returns as a pick error equal to its own size — 40 mm of side offset is a 40 mm miss — in a direction that rotates with the wrist.

The rest of the guide puts numbers under those lines: the worked example behind the third case, the test that exposes it, and the procedure for the second.

What the offset entry actually does

Every vision result starts life in the camera frame: a part centre at some pixel, converted to a point in front of the lens. The robot cannot use that number directly, so the controller chains two transforms to bring it into the base frame: base-to-tool, which its own forward kinematics know exactly, and tool-to-camera, which is the offset you enter.

A camera riding on the gripper is still an eye-in-hand set-up — the camera moves with the wrist, exactly as described in eye-in-hand vs eye-to-hand camera mounting for robots. The chain only asks that the tool-to-camera term is the true, constant pose of the camera in the tool frame: three translations and three rotations. A Z-only entry asserts a translation of (0, 0, -z) and an identity rotation. When the physical mount matches that assertion the chain is exact; when it does not, the mismatch rides through every pose the robot ever visits.

What a Z-only entry costs: the worked example

Take a realistic mount. The tool frame origin sits at the TCP with +Z pointing at the part. Space past the jaws is blocked, so the camera sits behind and beside the TCP, tilted to see past the fingers. Measured on the bench:

• camera origin at (X, Y, Z) = (-40, 0, -85) mm in the tool frame — 40 mm to one side, 85 mm behind the TCP;
• optical axis tilted 15° about the tool +Y axis.

The software offers one field, so you enter the Z term only: translation (0, 0, -85) mm, rotation identity.

The un-modelled part is the difference, (-40, 0, 0) mm. Rotation preserves length, so the error magnitude is √((-40)² + 0² + 0²) = 40 mm at every orientation — only its direction rotates with the wrist. Roll 0° puts the miss 40 mm toward tool -X; roll 90° turns it 40 mm toward -Y; roll 180° flips it to +X. Same part, same program, different misses.

The tilt adds an orientation error. A feature imaged 300 mm in front of the camera maps through an assumed optical axis 15° away from the true one, so the lateral error is roughly 300 mm × tan(15°) ≈ 300 × 0.268 ≈ 80 mm. That is a first-order estimate, but the scale is the point: frame errors of tens of millimetres swamp any sub-pixel precision the camera itself delivers.

If picks already miss in a direction that changes with wrist angle, that is this signature; the fault-finding path is laid out in diagnose an eye-in-hand pick point offset.

The two-pose orbit test

Measure the mismatch directly before the first production pick. Fix a part, or the centre mark of a calibration board, on the table. Move the TCP over it in two poses that differ only by wrist roll: 0° and 90°. Trigger the camera at both and read the part's base-frame coordinates from the vision result.

If the entered offset is right, the two reports agree to about 1 mm. If a side offset a is missing, each report carries an error R·Δt that points differently in each pose; for a 90° roll the two errors are perpendicular, so the reported positions jump by √2 × a. The arithmetic runs backwards cleanly: a jump of 57 mm means a = 57 / 1.41 ≈ 40 mm of un-modelled side offset — precisely the mount in the worked example.

No jump means the entered transform is consistent with the physical mount in translation, and the remaining risk is repeatability, not frame error.

When a Z offset alone really is enough

Three conditions, all checkable at the bench:

• the camera origin sits on the tool Z axis — concentric with the TCP, no X or Y term;
• the optical axis is parallel to the tool Z axis — a machinist's square against the barrel checks this well enough at mounting time;
• the Z you enter is the true camera-origin distance, measured from the TCP touch-off point rather than scaled off the bracket drawing, where shims, spacer stacks and barrel lengths quietly move the number.

Meet all three and the Z-only entry is exact to your measurement accuracy — rule and depth gauge get within about 1 mm, and the orbit test confirms it in place.

Dry-run the geometry before anything goes on the robot: lay the board where the part will sit, hold the camera at the proposed standoff, and check the view. A camera stand with a glass tray turns that into a ten-minute bench exercise instead of a robot-cell experiment.

Recovering the full offset yourself

When the mount keeps its side offset or tilt, the fix is a hand-eye calibration of your own. The solver returns the camera's true pose in whatever reference frame you choose, so the camera does not care which side of the gripper it is bolted to. Work in this order:

1. Bolt the camera into its final bracket and torque-mark the screws. Calibrate in the mounted state; unbolting afterwards changes the answer.
2. Fix the tool frame. Calibrate the TCP you actually pick with and select that tool in every pose. Calibrating in a different tool frame than production uses is the classic source of offset errors — hand-eye calibration wrong robot frame: match tool0 to the sample walks through it.
3. Lay a calibration board flat in the work envelope, sized to stay fully in view across all poses; hand-eye calibration target size: how big a board your poses need gives the sizing rule.
4. Record 15–20 image-and-pose pairs around the board, tilting the wrist across at least 30–45° in two different rotation axes. Poses that only translate, or only roll, starve the solver — AX=XB solver fails: motion degeneracy and how to fix it shows what that looks like. Save the exact robot pose with every image, as in hand-eye calibration dataset acquisition with saved robot poses, and choose the poses with eye-in-hand calibration pose selection: robot positions that work.
5. Solve eye-in-hand for the camera-to-flange transform, then compose it with the flange-to-tool transform to express the camera in the tool frame. Camera to robot transform calibration: stop measuring by hand covers the chain, and convert a hand-eye quaternion to a transform matrix gets the result into the rotation format your controller wants.
6. Check the residual. Aim for an average reprojection error under about 0.5 pixels; near or above 1 pixel usually means blur, a warped printout or a loose bracket. The targets are set out in camera calibration residual values: average and max targets.
7. Enter the full transform, then verify: the orbit test first, then the pick-based checks in hand-eye calibration accuracy verification: test before you trust it.

A camera referenced directly to the TCP rather than the flange suits gripper mounts just as well, since the offset you produce is the one your picks consume; the variant is covered in robot-held camera calibration with a TCP tool frame.

Mount details that protect the result

The calibration is only as good as the mechanics holding it:

• Rigidity. Bracket deflection appears one-for-one as camera motion, so prefer a plate bolted straight to the gripper body over a post on stacked adapters.
• Standoff inside the depth of field. Part height changes between grip, lift and place; set the camera-to-part distance so the whole range stays sharp. The depth-of-field calculator pairs standoff and f-number for your sensor and lens.
• Jaws out of frame. Rear and side mounts can put the gripper in its own image; mount far enough back that the jaw tips drop below the field of view, or mask them.
• Collision envelope. The bracket extends the gripper footprint — re-check reach poses and keep-out zones with the camera fitted.
• Cables. A wrist flexes every cycle; leave a strain-relieved loop, not a taut line.

If you are choosing the camera itself for a gripper mount, structured-light 3D cameras such as the MV-DPS200P-02 return the part pose from a single capture, which is why they are the usual eye for bin picking on the wrist — and their calibration is the same camera-to-tool problem solved above.

Keep the focus locked after calibration and mark the ring: any later refocus or lens swap moves the effective camera pose even though the bracket has not moved, as calibrate a camera with autofocus: lock the focus first explains.

When the flange mount is still the better answer

Keeping the camera above the tool coupler wins in three cases:

• Grippers change often. A gripper-mounted calibration dies with the gripper swap; a flange-mounted camera keeps its camera-to-flange transform and only the TCP changes. Recalibrating a TCP is minutes; a hand-eye dataset is a longer job.
• The vision stack is closed. If the vendor workflow accepts only Z and you would rather not maintain your own calibration scripts, re-centre the camera — a centred mount makes the Z-only entry exact.
• Payload and space. Camera and bracket add mass at the wrist and enlarge the gripper footprint; when the arm's payload margin is thin or the cell is tight, the flange side of the coupler usually has room.

Before the first production pick

Calibration

• Production TCP calibrated and selected in every pose
• 15–20 poses with 30–45° of tilt spread across two rotation axes
• Average reprojection residual under about 0.5 pixels

Entry and verification

• Full camera-to-tool transform entered; rotation is identity only for a camera looking straight down the tool axis
• Two-pose orbit test agrees within about 1 mm
• Twenty test picks across varied wrist rolls hold the pick tolerance

Mechanics

• Bracket bolts torque-marked; camera not unbolted since calibration
• Focus locked and ring marked; no optical change since calibration
• Cables strain-relieved; collision envelope checked with the camera fitted

Next step

If the two-pose test jumps, fix the entry before touching anything else — it is the cheapest correction you will ever make. When you are speccing the camera side, the camera finder filters the stereo and structured-light cameras that live on wrists, and the configurator walks the chain from camera to mount; the robotics topic page collects the rest of the robot-side set-ups.