2D camera to robot calibration without vendor tools

A fixed overhead camera can steer a robot accurately without a vendor's one-button suite. The whole job is one distortion fit and one homography, and an integrator with a checkerboard, a rigid stand and any vision library can run it in an afternoon. This walkthrough covers the fixed-camera, flat-plane case: the camera bolted off the arm, looking straight down at the work surface. You will size the field of view, remove lens distortion, capture 9-16 plate points in robot coordinates, fit the pixel-to-robot transform, and prove it with an offset test before the first live pick. The worked example uses a 5 MP 2/3" camera, a 25 mm lens and a 300 mm working distance; substitute your own numbers where they appear.

What does a 2D camera to robot calibration actually compute?

One 3 × 3 homography — eight independent numbers — maps corrected pixel coordinates (u, v) to robot base coordinates (X, Y), fitted by least squares from point pairs whose robot positions you already know, and no robot maker's software is needed to produce it.

The calibration plate is the bridge between the two coordinate sets. Place a checkerboard in view, detect its inner corners in pixels, and record where those same points sit in the base frame — read from a fixture drawing or touched off with a pointer in the gripper. With the camera bolted to the frame rather than the wrist, one flat mapping covers the entire plane, so a single capture session finishes the job. A camera riding on the arm is a different layout: camera to robot transform calibration without hand measurement and the eye-to-hand setup for a camera above the robot base treat those cases in full.

How accurate can a plate-based 2D calibration be?

Budget about ±0.1 mm end to end: at a 300 mm working distance a 25 mm lens on a 2448-pixel 2/3" sensor resolves 0.041 mm per pixel, and pixel scale, fit residual, target quality and robot repeatability combine to 0.08 mm by root-sum-square.

The field-to-focal-length ratio is the thin-lens estimate. The blister inspection walkthrough of the working distance and field of view formula runs the same arithmetic in detail, and this guide to field of view versus working distance shows how the scale moves as the camera rises or drops.

Add the contributions as independent errors:

That total is a budget, not a datasheet limit, and the robot term dominates on most cells. Rescale the sensor and focal length in the calculator and the pixel-scale terms follow directly:

What do you need on the bench before you start?

Four items decide the result: a stand that cannot creep, a lens of known focal length with a published distortion figure, a flat 9 × 6 checkerboard covering most of the field, and lighting driven at a fixed level from capture through production.

Bolt the camera square to the plane on a locked column — after calibration, 1 mm of camera motion is a 1 mm bias on every pick. The LS-MNT-0040 is a slide-table stand with a 700 mm column on a 400 × 450 mm base, sized for bench cells of exactly this kind.

Pick a known prime from the machine vision lenses category. The LS-FFL-0679 is a 35 mm, 2/3", 6 MP C-mount lens. Swap it into the worked example and the field becomes 8.45 mm × (300 / 35) = 72 mm, or 0.030 mm/px — tighter pixels over a smaller field at the same stand height.

Print or etch the board matte black on white, on rigid stock. At this scale each 10 mm square spans 10 / 0.0414 = 242 pixels, so subpixel corner detection to a tenth of a pixel is routine; tape the board flat, because 0.05 mm of bow spends the whole target term of the budget.

Lock brightness between calibration and production. Drive the light from a digital light controller at a fixed percentage — the LS-CTL-0290 is a 60 W, 4-channel unit with 24 V constant-voltage output — and turn every auto-exposure and auto-iris loop off in the camera; what automatic exposure and gain modes do to a settled image explains the drift they add.

How do you remove lens distortion before the fit?

Fit distortion first and the homography second: capture 15-20 checkerboard images over the whole field, solve the radial and tangential coefficients, and remap every detection, because a lens at the common 0.3% distortion figure shifts a field-edge point 0.30 mm — 7 pixels at 0.041 mm/px.

The Brown-Conrady model — radial k1, k2, k3 plus tangential p1, p2 — describes machine-vision lenses well: fitting distortion from point pairs alone shows the solve, and the distortion-correction walkthrough for pixel-to-mm measurements covers the remap. OpenCV's calibrateCamera returns the coefficients plus an RMS reprojection error in pixels; accept the fit at 0.5 px RMS or better, discard the frames that push it up, and lock the coefficients. Every image afterwards, calibration and production alike, is remapped once, and all detection runs on corrected coordinates.

How do you capture point pairs the robot can trust?

Collect 9-16 pairs spread to the field edges, each one a detected image point matched with its base-frame coordinate read to 0.05 mm or better; four points solve the homography, but edge coverage is what holds the fit residual near one pixel.

Three capture layouts work:

1. Fixed plate, pointer touch-off. Bolt the board flat on the table, detect its inner corners, then jog a pointed tool in the gripper onto each corner and read base-frame X, Y from the pendant. Pair accuracy follows the tool point, about 0.02-0.05 mm.
2. Plate in the gripper. Grip the board and jog the wrist to nine commanded poses, each placing the plate at a known base position. This is the geometry of nine-point hand-eye alignment minus the rotation solve, which a fixed camera does not need.
3. Known-fixture plate. If the board sits in a machined nest, read its corners from the drawing. Fastest, and only as good as the nest.

Run corner detection subpixel on the remapped image, with the light at the production drive you locked earlier.

What are the exact fitting steps, in order?

Eight steps run from mounted camera to released transform: square the mount, fit and lock distortion, remap, collect the pairs, solve the homography by least squares, scan residuals, prove one unplanned point, then store the transform with its conditions — the solve itself is one OpenCV call.

1. Square the camera to the plate. Level the mount or scan a straight edge along both image axes; hold tilt under about 1° so the perspective terms stay small.
2. Fit distortion. 15-20 frames, RMS at 0.5 px or better, coefficients locked.
3. Remap once. All later detection runs on corrected coordinates.
4. Collect 9-16 pairs. Record base-frame X, Y in millimeters, three decimals where the pendant gives them.
5. Solve. With clean pairs, the homography is a plain least-squares solve: OpenCV's findHomography over the full pair set, no outlier rejection needed.
6. Scan residuals. Back-project every plate point; each should land within 1 px, about 0.04 mm here. A point beyond 1.5 px is a bad corner detection — re-detect it rather than refitting around it.
7. Prove one unplanned point. Set a 10 mm dowel somewhere the fit never saw, detect it, command the robot over it, and measure the pick offset. Accept 0.1 mm or less at this scale.
8. Store the transform with its conditions. Camera position, lens, focus setting, working distance, light mode. Any of those moving voids the fit; refit rather than nudging numbers.

At production time the pick logic detects the part's feature point on the remapped image, applies the homography, and hands base-frame X, Y to the motion program; Z comes from the fixture, not from this calibration. Running the chain without a robotics framework shows the solve in plain OpenCV and ViSP. The identical math carries into controller integration: a pixel-to-robot-base walkthrough for a six-axis controller applies the same transform end to end.

How do you prove the calibration before the first live pick?

Three checks, in order: a residual scan over all plate points (each within 1 px), proof picks at three unplanned positions (offset 0.1 mm or less), and a drift re-check after 24 hours of running (under 0.05 mm of movement on a rigid stand).

The residual scan catches detection mistakes; the proof picks catch everything else — tilt, scale error, a stale working distance. Spread the probes across the field: one near the center, one at an edge, one at a corner. If only the edge probe misses, distortion residuals are still alive; if every probe misses by the same vector, the camera or the working distance moved. Centimeter-scale misses point at a frame or sign mix-up rather than a bad fit — the failure the 1-2 cm translation error diagnosis dissects — and this accuracy verification procedure formalizes the test plan. Log the day-one offset results beside the transform file so the next shift knows what the cell achieved.

When does a flat-plate calibration stop being enough?

Height breaks it first: the mapping is exact only on the calibrated plane, and a part top 20 mm above a plate fitted at 300 mm reads 7.1% large — a feature truly 50 mm from center maps to 53.6 mm, a 3.6 mm miss.

Three remedies, simplest first: clamp parts to the calibrated plane; keep one transform per working height and select it by part program; or scale pixel distances by WD / (WD − Δz) about the image center, honest for small Δz. Past a few tens of millimeters of height spread, or with tilted parts, one homography is the wrong tool: move to a depth view — the 3D camera working distance and positioning guide sizes that step — or to an on-arm chain where the robot pose joins the math, as in eye-in-hand pixel-to-world transform chaining. Refit after lighting changes too: a different light geometry shifts where edges sit on shiny parts, and the coefficients will not follow on their own.

To stand the bench up, shortlist mounts in the camera stand finder, match a prime in the lens finder, and let the build-a-system configurator pair camera, lens and light in one pass; more coordinate-chain guides sit on the systems topic page. If a second pair of eyes on your accuracy budget would help, contact us with your working distance and target tolerance.