2D vision guided robot pick place grip: how the cycle works

A robot cell that assumes every part sits in the same spot works until the first shifted tray. 2D vision closes that gap with one overhead camera and three numbers per part: position and angle for the pick, an offset for the place, and the in-plane geometry that sets the grip. This guide is for integrators and robot programmers who need to size, calibrate and time that loop with their own numbers. It walks the three stages, sizes an overhead camera with the arithmetic shown, and ends with the decision for each cell type.

The three outputs the vision system owes the robot

One camera, one result message, three uses. The robot program keeps one taught nominal pose per part and applies the measured delta every cycle. The camera never replaces the teach; it corrects it.

These are planning values for a calibrated fixed overhead camera on a rigid mount; the worked example below shows where they come from.

Calibrate the camera to the work plane

Mount the camera fixed overhead and calibrate against the plane where parts actually sit, usually belt or pallet height. Move the robot tip to nine points in a 3 × 3 grid spread across the field of view, record pixel position and robot position at each, and fit scale, rotation and offset:

Here (u, v) are image pixels and (X, Y) robot millimetres; the four coefficients carry the scale and the in-plane rotation between the two frames. The fit is true only at the teach height, so height control carries the whole method — see the limits section. Re-teach after any bump to the mount: a small tilt of the camera moves every result by millimetres.

The same idea applies on the arm: the hanging robot vision calibration guide teaches the work plane directly, the eye-in-hand calibration pose selection guide covers cameras that move with the robot, and fixing a camera calibration warning clears the error most cells hit first.

Find the part with rotation-invariant matching

Match on a stable printed or machined feature — a logo, a notch, an edge corner — not the whole silhouette, which shifts with every printing variation. Search the full circle, ±180°, only when parts can arrive in any orientation; a ±30° window on a presented part examines six times fewer angle hypotheses and returns faster. Accept matches at or above 0.8 score on a 0-1 scale and confirm with a second feature when the background carries similar shapes. Sub-pixel edge interpolation then places the pose well inside one pixel, so the error floor is calibration, not matching.

Two failure modes deserve their own reading: template matching a part larger than the field of view when the search region runs past the frame, and color pattern matching for reliable part localization when colour is what separates the feature from the background.

Time the trigger and the strobe

Trigger from a photo-eye or PLC output and strobe the light during the exposure, on a global-shutter camera — a rolling shutter scans the frame while the part moves and smears it. Set the exposure from the blur budget:

Freeze motion at half a pixel, then buy brightness afterwards. One caution while tuning:

Worked numbers: at 300 mm/s belt speed and the 0.07 mm per pixel scale from the worked example below, t_exp ≤ 0.5 × 0.07 / 300 = 0.00012 s, so set 100 µs and accept 0.03 mm of blur (0.43 pixel). Continuous light at that exposure needs impractical LED power; a strobe driven only on trigger does it within duty-cycle limits. Strobe overdrive, duty cycle and timing gives the drive limits, matching LED strobe duty cycle pairs light to controller, and triggering strobe lighting from robot digital outputs shows the wiring.

One four-channel box usually covers the cell; two channels strobe on trigger for the overhead view and two run constant for alignment lights. The LS-CTL-0070 is a four-channel digital 24 V controller rated 200 W, with strobe output and an RS232 port for sequencing.

Size the overhead camera: a worked example

Given: a 60 mm part, ±25 mm position wander, a 300 mm/s belt, ±0.3 mm placement tolerance, 400 mm working distance, and a 1/1.7"-type sensor about 7.5 mm wide with 1600 × 1200 pixels.

1. Field of view: 60 + 2 × 25 = 110 mm minimum.
2. Magnification: m = 7.5 / 110 = 0.068.
3. Focal length: f = 400 × 0.068 / (1 + 0.068) = 25.5 mm, so order the standard 25 mm lens.
4. Real field of view at 25 mm: 7.5 × (400 − 25) / 25 = 112.5 mm, clearing the 110 mm need.
5. Pixel scale: 112.5 / 1600 = 0.07 mm per pixel.
6. Vision error: 0.07 mm per pixel plus roughly 0.05 mm calibration residual, so ±0.1 mm in-plane — inside the ±0.3 mm placement budget with room left for robot repeatability.

The diagram shows how sensor, lens, working distance and field of view set each other; the calculator below runs your own numbers.

For the camera itself, that arithmetic calls for a global-shutter C-mount GigE body: the MV-CU020-90GC pairs a 2.5 MP, 1/1.7" sensor with a GigE interface and a C-mount.

From camera to gripper: applying pick, place and grip offsets

Send one record per part: dX and dY in millimetres and dRZ in degrees, robot-frame, plus a part ID. The pick adds it to the taught approach pose and nothing else in the program changes — the structure in structuring a vision-guided pick and place program around drop targets is exactly this. For placement, a second view of the carrier measures the seated part and the drop gets the same correction; vision-guided placement accuracy budgeting pushes the same loop to far tighter tolerances.

The grip uses the same measurement twice: the measured RZ rotates the wrist about Z, and feature offsets across the part body — an edge, a connector, a boss — shift the grasp point along the part and set the opening width. All three corrections are in-plane; height, out-of-plane tilt and stack state never pass through a 2D result, so the cell controls them mechanically with presenters and seating plates.

Budget the loop: at a 250 mm part pitch and 300 mm/s, the cell owes a pick every 300 / 250 = 0.83 s, and a match plus transfer in 50-100 ms leaves most of that window for the move. The robot side stays an ordinary program that consumes offsets; the pick and place robot programming guide for multi-conveyor cells shows one structure. Whether those moves fit the window is drive sizing, which the servo versus VFD cycle time comparison for pick and place lays out.

Smart camera or area scan camera with a controller

Choose the smart camera when the cell needs the three offsets and nothing more: see the smart cameras or the camera finder. Choose an area scan body with a vision controller when you need several cameras on one cell, image archiving, or logic the device cannot hold: start at area scan cameras and the light controllers that drive their strobes. The no-PC route in detail is robot pick and place vision without an external PC.

The MV-SC3050XC-16M-ATN-H is a 5 MP color vision sensor with a global shutter, an M12 lens mount and a Fast Ethernet interface — the on-device route for a single station.

When 2D is the wrong tool

A single 2D view has no depth. Every measurement above assumes the part sits on the calibrated plane, and each millimetre of height error changes magnification by 1/399 ≈ 0.25% at the 400 mm working distance, putting a feature 50 mm off-centre out by about 0.125 mm. Keep total height variance within about ±1 mm — roughly ±0.06 mm at that offset — or move to 3D. Go 3D as well when parts arrive stacked or overlapped, when the robot needs a six-degree-of-freedom pose, or when the surface itself must be inspected. The 2D or 3D camera choice for conveyor pick and place makes that call systematic.

The decision, per cell

• Presented parts on a belt or pallet with controlled height: the fixed overhead 2D loop above — one camera, a 25 mm class lens at 400 mm working distance, a 100 µs strobe.
• Random orientation in a tray: the same loop with full-circle angle search and a light geometry that kills glare.
• Height variance beyond ±1 mm, stacked parts, or six-degree-of-freedom grasping: a 3D sensor instead.
• Several cameras or custom logic on one cell: area scan bodies with a vision controller.

Size the camera and lens for your own part in the configurator below, or browse the robot vision topic page for calibration, strobing and error recovery in the rest of the cell.