Auto-pick taught objects with a vision model: the generic method

Teaching waypoints and gripper actions for every new part is the slow way to build a picking cell. The generic method does the work once: teach the vision system the object and its grasp point, then let one program turn every vision match into a pick. Here you get the method in build order, the pixel math that decides whether your camera is good enough, a worked example with the arithmetic shown, and where a vision sensor, a smart camera or a vision controller fits. It is written for line engineers and integrators who need the second and third part in a cell to cost minutes of teaching, not a shift of programming.

What a taught-model auto-pick does

A taught-model pick has two phases. In the teach phase, you present the part under production lighting, draw the model region, set the rotation range the matcher may expect, and mark the grasp point relative to the model center. Vision software that stores these parameters — shape region, angle range, grasp offset — rather than a single reference picture is usually called parametric teaching, and it is what lets one model follow the part around its arrival angles.

The run phase is the same every cycle: trigger the camera, match the taught model, accept the match on score, convert the returned position and angle to robot coordinates, then run the shared pick-and-place moves with the computed pick pose. The waypoints are shared; only the pick pose changes. That is why the second part costs a teach session and not a program rewrite, and why the program structure never grows with the product list.

A 2D taught model returns X, Y and in-plane rotation only. Z comes from geometry, not from the image: the part height is a taught parameter, and the pick height is computed from the fixture surface minus that height. That single assumption — one height, small tilt — is the boundary of the whole method, and it decides when you need a 3D sensor instead.

The pixel math that decides whether the camera is enough

Pick accuracy is a chain of three errors added in quadrature: the matcher's detection error, the hand-eye calibration error, and the robot's own repeatability. Detection error starts from the pixel size, which follows from the field of view and the sensor's pixel count. Choose the field after the arrival spread is known: a field sized to the spread plus 10% margin keeps pixels dense where they matter.

The subpixel factor of 3 to 5 is what a grayscale correlation matcher typically resolves on a well-contrasted, sharply focused part; soft focus or weak contrast pushes the effective factor toward 1. A fixed-mount hand-eye calibration over a few hundred millimetres of workspace lands around 0.05-0.15 mm on a rigid stand, with target quality and stand rigidity setting where in the range you land. Small six-axis arms print repeatability between ±0.02 and ±0.05 mm. Two design rules follow: keep the predicted pick offset under one third of the part's locating feature tolerance, and give the gripper fingers a lead-in chamfer of at least five times the offset so the fingers correct what the pose misses.

Worked example: 5 MP over a 200 mm field

The part is a connector cap, 40 × 25 mm on the flange, presented flat and singly on a matte fixture, pick tolerance ±0.5 mm. A typical 5 MP sensor resolves 2448 × 2048 pixels, and the stand gives a 200 mm wide field.

• Pixel size: 200 / 2448 = 0.0817, so 0.082 mm/px.
• Detection error: 0.082 / 3 = 0.027 mm, and 0.082 / 5 = 0.016 mm; take the conservative 0.03 mm.
• Hand-eye calibration: 0.10 mm, fixed mount, calibrated at the production working distance.
• Robot repeatability: ±0.05 mm.
• Pick offset: sqrt(0.03² + 0.10² + 0.05²) = sqrt(0.0009 + 0.0100 + 0.0025) = sqrt(0.0134) ≈ 0.12 mm.

The decision: 0.12 mm against a ±0.5 mm budget passes, because it sits under the 0.167 mm that one third of the tolerance allows, and the chamfer rule asks for 5 × 0.12 = 0.6 mm of finger lead-in, so specify 0.6-1.0 mm. The cycle side: exposure 10 ms + readout and transfer 15 ms + match 80 ms + pose hand-off 20 ms = 125 ms of vision overhead, which is 6% of a 2 s robot cycle (0.125 / 2.0 = 6.25%). A 5 MP camera over 200 mm auto-picks this cap to about ±0.12 mm, and the camera is nowhere near the bottleneck.

Vision sensor, smart camera or vision controller

One taught part on a fixed stand needs very little compute. The choice widens with part families, camera count and whether AI checks will join the taught models. Mono is the usual pick for taught matching: the model trains on grayscale structure anyway, so mono sensors put their pixel budget into resolution rather than color. Color earns its place when the matcher must separate same-shape parts by color, or a color check rides along with the pick.

The three camera-side models in the table, side by side:

All 3 share: Humidity 20% RH to 95% RH (no condensation); Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

A vision sensor is the smallest bill of materials for one part on one station: the MV-SCB007EC-12S-WBN is a 1.6 MP color vision sensor with a built-in 12 mm lens and manual focus, made for a close, fixed field. A smart camera adds on-board vision libraries and interchangeable M12 lenses: the MV-SC5060XM-08M-WBN is a 6 MP mono model with an 8 mm lens on GigE, its rolling shutter at home on parts that hold still for the picture, while the global-shutter MV-SC3050XC-16M-ATN-H, a 5 MP color sensor with an M12 mount, covers presentation that moves during exposure.

Past 3 or 4 cameras, or once AI checks join the taught models, put the matching on a vision controller and keep the cameras as sensors. Two controllers that cover the range:

The MV-VC2140-128G40-1T runs an Intel N-series N97 processor with 8 GB of memory, a 128 GB SSD plus 1 TB HDD, and 4 built-in light channels — a two-camera taught-model cell with light control in one box. The MV-VC5525-256G70-NN moves to a 12th-gen Intel Core i5 with 32 GB of memory, a 256 GB SSD, PoE camera ports and 8 built-in light channels for a larger camera set. The camera finder filters by resolution, shutter and mount, and the system configurator builds the whole bill of materials around them.

The generic program, block by block

1. Shared constants. The camera scan pose, a safe home, the drop pose, and an approach height above the tallest part you handle. These never change when the part changes.
2. Taught data per part. Model ID, grasp offset from the model center, grip force or stroke, and the part height that sets Z.
3. Vision request. Trigger the capture, match the model ID, and accept only at score 0.7 or better. Below that, send a not-okay signal to the PLC instead of a pose; in enclosed, controlled light with a clean background, 0.5 is a safe floor.
4. Coordinate transform. Push X, Y and the angle through the hand-eye calibration into robot base coordinates, and add the angle to the tool orientation command.
5. Computed pick. Move to the pose raised by the approach height, go down in a straight line, close the gripper with the taught force, retreat straight up.
6. Shared drop. Release at the one drop pose. If each part drops somewhere else, that single case statement is the only place the program grows — the drop-target program structure guide shows how to keep even that generic.

For the 2D case, Z is bookkeeping. The cap sits on a fixture at Z = 800 mm in base coordinates and stands 12 mm tall, so the pick height is 800 − 12 = 788 mm and the approach point sits at 788 + 30 = 818 mm. Both numbers are computed from taught values at teach time; nothing is re-taught at runtime.

None of this needs a PC in the cell. A smart camera or vision sensor closes the loop on the device and talks to the robot directly, which is the set-up in the pick-and-place without an external PC guide.

When the generic answer is wrong

• Unknown tilt or stacked parts. A 2D taught model assumes one height and small tilt; a random bin needs Z, out-of-plane tilt and grasp planning, which is point-cloud matching on a 3D sensor. The 3D vs 2D decision guide draws that line, and vision-guided flexible feeding is the usual fix on the presentation side.
• Several part types in one image. Running every model on every frame slows the cycle and invites first-result mistakes; the multi-model scene guide covers the branch logic.
• Picks that drift as production runs. Drift traces to calibration or to arrivals outside the taught angle range far more often than to the model itself; the vision pick rotation drift checklist works through both.
• Scores that swing with factory lights. Enclose the station, drive the light from a controller channel, and use the ambient light control guide to test how much variation is left.

Checks before you order

• Field: the field of view covers the whole arrival spread plus a 10% margin, and the smallest feature the matcher must hold is at least 3 pixels across (field width / horizontal pixels).
• Mount: a rigid stand that shares no vibration with the press or feeder, camera distance recorded, calibration run at the production working distance.
• Light: enclosed against factory lighting, driven from a controller or camera channel so every exposure is identical.
• I/O: one output for the gripper, one not-okay signal to the PLC, a hardware trigger input; count light channels against the controller's built-ins.
• Commissioning: ten consecutive auto-picks at reduced speed with guarding closed, then measure the spread of picked positions with calipers and hold it near the predicted offset — the 0.12 mm from the worked example — and inside one third of your tolerance.

Start the build at the camera: the configurator below filters cameras to the pixel size your tolerance needs. If you want the pick math checked against your part drawing, send the layout through the quote page.