Vision locate for robot part picking: set it up to survive rotation

A vision locate that passes at 0° and fails at 30° is the most common way a pick cell disappoints once it reaches the floor. The part arrives rotated, the matcher scores low, or it locks onto a look-alike feature and hands the robot an angle wrong by tens of degrees. This guide is for the engineer wiring that cell. It gives the arithmetic that decides whether the camera resolves rotation at all, the teaching choices that keep the score up as the part turns, a worked example running from field of view to slip at the gripper, and the verification pass that finds the failures before the first live pick. The same logic holds whether the locate runs on a smart camera at the belt or on a vision controller beside it.

What the locate step hands the robot

A locate returns three values: the X-Y position of the taught feature frame, an in-plane angle about it, and a score for how well the image matched the taught model. The robot takes the taught pick point, rotates it by the returned angle around the returned position, and moves there through the camera-to-robot transform. That transform is a one-time job; the nine-point alignment behind most robot hand-eye calibrations is the standard route, and Robot hand-eye calibration: how nine-point alignment works walks the arithmetic.

Two failure classes follow, and they need different fixes. A no-match means the score fell below threshold: part present, model not recognised. That case, and the empty belt next to it, is handled by part-present logic in Vision locate with a part detected flag variable: stop the empty belt. A wrong match is worse: the matcher returns a confident pose at a look-alike feature and the robot picks air. Rotation raises the odds of both, so the rest of this guide works on keeping the match honest as the part turns. If the cell architecture itself is still open, 2D vision pick and place: selecting a planar guidance system frames the fixed-camera versus wrist-camera choice.

The numbers that decide whether rotation survives

Four numbers decide it: pixel size, field of view, the pixel budget granted to the matcher, and the lever arm from located feature to grasp point.

Grant the matcher 2 to 3 pixels. Matchers interpolate sub-pixel, but this budget is what survives a soiled lens, an ageing LED and a slightly soft focus. The angle term is the one engineers skip: angle error is pixel error divided by the radius of the geometry the model keys on, so a model taught on a 10 mm boss rotates six times worse than one taught on a 60 mm span at the same pixel size. More pixels shrink both terms in proportion, which is why the worked example below starts from the tolerance and works backwards to a pixel size, not forwards from a camera.

Teach the model so rotation keeps the score up

Teach on geometry, not on shading. A greyscale template keyed on a highlight changes its contrast as the part turns; a contour model keyed on the outline does not. Three rules cover most cells:

1. Teach the full silhouette or its largest asymmetric feature. Angle accuracy scales with feature radius, so a 60 mm span outperforms a 10 mm boss six-fold on the same sensor.
2. Set the rotation search range wider than the arrival spread. If parts can land any way up, cover a full turn; the range costs search time, not accuracy, because only the found pose gets refined.
3. Audit the part for self-similarity. Round flanges, evenly spaced hole patterns and square bodies return several equally good poses at fractional turns; no threshold fixes that, the geometry has to break the tie. The way to measure the tolerance symmetry actually allows is worked through in Symmetric part orientation matching: rotation tolerance in picking.

Worked example: 150 mm bracket picked to ±0.5 mm, arriving 37° off nominal

Inputs. Belt-to-nest cell: drawn bracket with a 150 mm × 100 mm silhouette; grasp point 40 mm from the centroid; largest asymmetric span R = 60 mm; the nest takes the part to ±0.5 mm, of which robot, gripper and nest fit together are allowed ±0.35 mm; the part stops anywhere in a 200 mm × 160 mm window, rotated up to 37° from the taught pose; camera class 5 MP global shutter with the common 2448 × 2048 array; matcher budget n_px = 3 pixels; working distance 300 mm. The bracket, tolerances and window are the declared case values; the 3-pixel budget is standard practice for a calibrated 2D cell.

Rule in words. The field must cover the window plus margin; the pixel size must keep 3 pixels inside the vision share of the tolerance; and the angle error times the grasp radius must fit that same share.

Arithmetic.

1. Vision share of the tolerance: e_budget = sqrt(0.5² − 0.35²) mm = sqrt(0.1275) mm ≈ 0.36 mm.
2. Largest usable pixel: q_max = 0.36 mm / 3 px = 0.12 mm/px.
3. Field of view: the 200 mm × 160 mm window plus 10 % margin gives 220 mm across the width; q = 220 mm / 2448 px = 0.0899 mm/px, inside the 0.12 mm/px limit with 25 % room, and the array's 2048 rows cover 184 mm of height.
4. Position error: e_pos = 3 px × 0.0899 mm/px ≈ 0.27 mm.
5. Angle error: angle_err = 3 × 0.0899 / 60 rad ≈ 0.0045 rad ≈ 0.26°.
6. Slip at the grasp point: e_ang = 40 mm × 0.0045 ≈ 0.18 mm.
7. Combined vision error: e_vision = sqrt(0.27² + 0.18²) mm = sqrt(0.1053) mm ≈ 0.32 mm.

Result. 0.32 mm against a 0.36 mm budget: the 5 MP class locates this part to tolerance with about 10 % margin, and a 37° arrival sits far inside a full-turn search range.

Decision. Order the 5 MP class and spend the leftover margin on lighting, not pixels. Watch two thresholds. If the robot-gripper share erodes past ±0.35 mm or the window grows beyond 220 mm, rerun at 12 MP (4096 × 3000 array): q = 220 mm / 4096 px ≈ 0.054 mm/px, e_pos ≈ 0.16 mm, e_ang = 40 mm × (3 × 0.054 / 60) ≈ 0.11 mm, e_vision ≈ 0.19 mm — a 45 % margin, and the version to buy for a line that runs for years. Past roughly 0.05 mm/px the remaining error lives in calibration and lighting, so more pixels stop paying.

The lens side closes the set-up: at the 300 mm working distance, a 220 mm field on a 2/3" sensor (8.8 mm wide) needs f = 300 × 8.8 / 220 mm ≈ 12 mm, which is why 8-16 mm M12 lenses are the usual fit for belt cells — the lens focal length calculator runs this per station.

Lighting, shutter and triggering

Rotation changes which faces the light hits, so a locate that passes in daylight and fails on night shift is telling you the room is part of the model. Fix the light to the camera, not the building: a diffuse ring or flat light enclosing the shot keeps edge contrast constant as the part turns, while spotlights aimed from the machine frame do the opposite. Compare scores across a full shift before trusting one good hour; the plays in Ambient light variation in pick and place vision: control it cover shielding, exposure strategy and the polarizer case.

Shutter follows motion. A stopped conveyor presents a static scene, where a rolling shutter sensor matches a global shutter one pixel for pixel; on-the-fly picking, a wrist camera that has just stopped, or a vibrating fixture needs global shutter and a settle delay before trigger. Trigger from a part-present photoelectric sensor or the stopper position, never from a timer.

Cameras and controllers that fit the arithmetic

The example's field needs 1,833 pixels across at the 0.12 mm/px limit (220 mm / 0.12 mm per pixel); the 2,448 pixels of a 5 MP class sensor deliver it with 25 % room. Mono beats colour for locate: colour sensors interpolate the colour sites and soften exactly the edges a matcher keys on, so colour earns its place only when hue itself is the feature.

The MV-SC5060XM-08M-WBN smart camera matches the example's camera class with margin: 6 MP mono, 8 mm M12-mount lens, GigE, rolling shutter — right for the stopped-belt case above.

For a compact cell that wants one head and one cable, the LS-SMC-0063 integrates a polarized white light with its 16 mm lens and reads up to 60 fps, so the silhouette lighting travels with the camera.

Size the controller by ports: a LAN port per camera plus one for the robot link, and one light channel per light. The MV-VC2140-128G40-1T covers a single-camera cell with 3-4 LAN ports, 4 built-in light channels and an Intel N-series processor.

Cells with two or three cameras, PoE wiring and heavier matching favour the MV-VC5525-256G70-NN: a 12th-gen Intel Core i5 with 32 GB of memory, PoE camera ports, 8 built-in light channels and 5+ LAN ports.

Handshake with the robot over discrete I/O or the cell's vision interface: trigger in, busy, results valid, pick acknowledged — the same signal-for-signal pattern as this robot vision I/O handshaking for multi-part picking. The full ranges sit in the smart cameras and vision controllers categories, with pixel filters in the camera finder.

Verify at the rotation extremes before the first live pick

Bench first. Present the part at 0°, then at ±20°, ±37° and 90°, and log three numbers at each pose: best match score, returned X-Y against a measured datum, and returned angle against a fixture stop or protractor. The angle log catches what the score hides — a confident match can sit 5° off and still pass a loose gate, and the miss reappears as drift at the gripper, the pattern traced in Vision pick error under rotation: diagnose position drift. Set the accept threshold from those logged scores, not from the tool default.

One robot-side check belongs in the same pass: the pick offset is only as good as the tool frame it rotates into, so confirm the TCP before blaming vision; the robot-side TCP set-up for correct part picking is laid out in this TCP configuration guide.

Then prove the reject path: missing part, upside-down part, two overlapping parts — each must end in a clean no-pick, never a best guess. When a live cell throws a bad-position alarm after a located pick, the causes sit on a short list, worked through in this vision picking alarm guide.

When the usual answer is wrong

• Round or symmetric parts: the pose is ambiguous by geometry, not by software. Measure the tolerance the symmetry allows (the method sits in the symmetric-part article linked above) or give the part or fixture an asymmetric datum.
• Unknown height or jumbled parts: a 2D locate returns no Z and no out-of-plane tilt; bulk parts are bin-picking territory, and a 3D sensor is the right tool.
• Shiny or black parts: the taught model is probably the highlight pattern. Move to diffuse or dark-field lighting so edges carry the match instead of reflections.
• Parts that never stop: rolling shutter smears a moving scene. On-the-fly picking needs global shutter, short exposure, more light and encoder-triggered acquisition.

And when the locate looks right but picks drift, suspect the camera-to-robot transform rather than the matcher: recalibrate instead of nudging pick offsets, following Camera to robot transform calibration: stop measuring by hand. With a wrist-mounted camera, the poses fed into its calibration decide the accuracy everywhere else, as Eye in hand calibration pose selection: robot positions that work shows.

Order checklist

• Field of view covers the arrival window plus 10 % margin, measured at the real working distance.
• Pixel size at or under the tolerance budget (0.12 mm/px in the worked example).
• Mono sensor unless hue is the feature; global shutter if anything moves at exposure.
• Light fixed to the camera; scores compared across a full shift, not one hour.
• Model taught near the worst-case rotation; search range wider than the arrival spread.
• Symmetry audit done, ambiguous poses listed and handled.
• Threshold set between measured pass and fail scores; reject path proven live.
• Angle sign checked against the robot frame; robot TCP confirmed.
• Controller ports counted: LAN per camera plus the robot link, one light channel per light.

Run your own part through the arithmetic, then step through the system configurator camera stage or send the drawing through the contact page and we will return the worked bill of materials. The rest of the cell-integration set lives on the systems topic page.