2D vision pick and place: selecting a planar guidance system

A camera can locate a part accurately in an image while still sending the gripper to the wrong physical point. The missing information is often height, tilt, an ambiguous orientation, or motion since exposure. Those limits should determine the imaging method before you select hardware.

For tray loading, flat-component handling, and conveyor picking, start by defining what the mechanics already constrain. This guide shows you how to decide whether planar guidance is sufficient, translate pickup tolerances into imaging requirements, and run an acceptance test before committing to a system.

1. Decide whether the task is genuinely planar

A conventional planar guidance system reports position across a reference plane and rotation around its normal. The robot still needs a pickup height, approach direction, collision-free path, and valid tool geometry. Those requirements do not disappear because the camera supplies coordinates.

Planar imaging is a strong candidate when the fixture, tray, or presentation mechanism controls the missing dimensions. It becomes less attractive when the system must discover those dimensions independently for every part.

An apparent outline does not prove that the top surface is accessible. Likewise, a visible mark may identify orientation without revealing whether a part is tilted.

Do not treat imaging as a safety function unless the complete system is designed and validated for that purpose. Machine safeguarding remains separate from ordinary part-location logic.

2. Allocate the pickup error before choosing pixels

Start with the allowable error at the actual tool contact point, not the repeatability of a detected feature. Include the camera, calibration, part presentation, motion timing, robot, and gripper.

For an illustrative lateral allowance of 0.60 mm, a preliminary allocation could be:

These are design targets, not measured capabilities. Adding them conservatively gives 0.60 mm. A root-sum-square calculation needs justified statistical assumptions; it is not a substitute for handling systematic offsets.

Rotation also produces translation when the pickup point is offset from the feature used for orientation.

At a 40 mm offset, an orientation error of 0.2° contributes approximately:

That contribution can consume most of an image-localization allowance. For errors that change with part angle, use the diagnostic sequence in vision pick error under rotation.

Check height-induced perspective error

With a conventional perspective lens, a calibration for one plane does not automatically remain accurate above that plane. A useful pinhole estimate is:

For r = 100 mm, D = 500 mm, and h = 2 mm:

This estimate assumes an aligned camera and an ideal perspective model. It also shows why a center-only test misses a serious selection problem. The effective projection center is not necessarily the front of the lens, so do not substitute a mechanical working distance without checking the geometry.

3. Size the field of view and image sampling

The required field of view includes the complete search region, placement variation, and clearance around features used for localization. A part touching the image boundary can lose the edge or mark needed to establish its pose.

The lens projects that region onto the sensor. Changing working distance, sensor dimensions, or focal length changes the coverage relationship.

Record the usable field after any cropping, rectification, or image rotation. For planning those margins, use calibration target coverage and lens margins. The nominal sensor image is not always the image your detection algorithm receives.

Calculate object-space sampling independently along each image axis:

Sampling is not accuracy. Edge contrast, focus, distortion correction, motion blur, and feature geometry determine how repeatably a location can be measured. Subpixel output alone does not demonstrate subpixel physical accuracy.

Use camera resolution for dimensional inspection to distinguish pixel sampling from the uncertainty of a physical measurement. For the optical selection that follows, choosing a lens focal length explains the relationship between sensor size, working distance, and coverage.

4. Work through a complete preliminary selection

Consider an illustrative tray application with a 240 × 180 mm search region. The parts rest on a controlled surface, and the pickup feature is 0.60 mm wide. Assume a usable image of 4000 × 3000 pixels for the first calculation.

If experiments show a localization spread of 2 pixels under the intended conditions, its object-space equivalent is:

That is below the illustrative 0.15 mm image-localization allocation. However, it leaves little room for additional bias. Test the complete pipeline rather than assuming that a feature spanning 10 pixels will always be adequate.

Now apply the 0.10 mm height-error allocation at a lateral distance of 100 mm. Rearranging the earlier perspective relationship gives:

Under those assumptions, approximately 0.50 mm of upward height variation reaches the allocation. A presentation process allowing several millimeters would need a different geometry, mechanical control, height compensation, or depth measurement.

The decision is therefore not simply to buy more pixels. Extra resolution cannot remove an incorrect assumption about the part plane.

5. Select the camera, lens, and processing hardware

Compare area scan cameras using the camera finder only after establishing usable resolution, shutter requirements, interface, and acquisition rate.

The MV-CU020-90GC and MV-CH120-11UC are global-shutter candidates. The LS-CMR-0166 uses a rolling shutter. These are acquisition alternatives to investigate, not equivalent selections for the worked example.

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

A global shutter is generally the simpler starting point for moving parts because it avoids geometric skew from sequential row exposure. It does not eliminate motion blur during exposure. A rolling shutter can be suitable for stationary capture when the part remains still throughout the relevant exposure and readout sequence.

Do not select a color camera solely because the displayed image is easier to interpret. Color is useful when it separates otherwise similar parts or features. If location depends only on silhouette or brightness contrast, evaluate whether color adds information that the task actually needs.

For machine vision lenses, check sensor coverage, mount, working distance, focus range, aperture, distortion, and performance across the full image. The lens finder helps narrow candidates, but a nominal format match does not establish edge sharpness or pickup accuracy.

The LS-FFL-0567 is a C-mount lens candidate. Confirm its coverage and working geometry against the selected camera before ordering.

For processing, vision controllers should be evaluated on camera connectivity, processing latency, software compatibility, and integration requirements. Port count alone does not establish simultaneous acquisition or algorithm throughput.

The MV-VC3301H-128G66 is a vision-controller candidate. Verify the required acquisition software, robot communication, and application licensing as part of the selection.

Benchmark representative images with the actual algorithm. Include difficult orientations and incomplete detections, which can take longer than easy matches.

6. Establish stable images and a verifiable coordinate mapping

Choose illumination around the feature that defines the pickup point. A silhouette may provide stable outside edges while hiding orientation marks. Surface illumination may reveal those marks while introducing reflections that move as the part rotates.

Test actual production variation rather than one clean sample. Include finish changes, printed variation, shadows from neighboring parts, and the least favorable allowed orientation.

Lock exposure, gain, focus, and aperture during geometric validation unless a changing setting is explicitly part of the validated process. Automatic adjustments can change feature appearance, while refocusing may change magnification or other calibration-dependent geometry.

Calibration should cover the usable search region and correspond to the physical plane used for guidance. Record units, axis directions, coordinate origin, tool offset, and the definition of the reported angle. A correct image location can still become an incorrect robot target through a convention mismatch.

Use independent check positions that were not used to fit the mapping. The guide to extrinsic calibration accuracy beyond reprojection error explains why a small fitting residual does not prove accurate physical coordinates.

7. Size exposure and coordinate timing together

For moving parts, exposure time determines blur, while the age of the coordinates determines where the part will be when the gripper arrives. These are separate constraints.

At 500 mm/s and 0.060 mm/pixel, limiting blur to 1 pixel gives:

This is a blur allocation, not a guaranteed acquisition setting. The illumination must provide sufficient signal within that exposure, and the feature detector must tolerate the remaining blur.

At the same speed, 2 ms of uncompensated timing uncertainty contributes:

A stable delay can sometimes be compensated. Variable delay, an incorrect exposure timestamp, or mismatched encoder data is harder to correct. Tie every detection to the exposure event and, where applicable, the corresponding conveyor position.

Use vision request timing and overlapping processing to plan the acquisition sequence. For moving-scene data validity, troubleshooting stale conveyor pick data addresses why a valid detection can still describe an outdated scene.

Transmitted targets should include a detection identifier, coordinate frame, orientation convention, and validity state. Define what happens when processing finishes after the usable pickup window.

8. Run an acceptance test before committing

Agree on pass criteria before collecting images. Otherwise, a successful demonstration can conceal a narrow operating envelope.

1. Establish independent reference positions. Use known physical locations and a measurement method suitable for the tolerance. Do not validate a mapping using only its own fitted points.
2. Distribute positions across the image. A preliminary grid of 5 horizontal by 4 vertical positions gives 20 locations, including the regions where perspective and distortion effects are often most visible.
3. Exercise the orientation range. Include angles that challenge symmetry, feature visibility, and the tool offset. A regular 45° spacing provides 8 starting orientations for a full-turn screening test.
4. Exercise the presentation limits. Repeat at the minimum and maximum allowed heights, with permitted tilt and neighboring-part arrangements. Record which conditions the system cannot distinguish.
5. Measure repeatability and bias separately. Repeated captures at a fixed position characterize spread. Comparison against the independent reference reveals systematic position error.
6. Repeat under production timing. Include normal processing load, intended trigger rates, and the shortest valid pickup window. Test rejected detections as well as successful ones.
7. Verify the physical pickup. Measure performance at the contact point, including tool alignment and approach height. A correct feature overlay is not an acceptance test for gripping.

For a preliminary repeatability screen, 30 captures per stationary condition can reveal obvious instability. This is not a universal sample-size requirement or proof of a low failure probability.

Report the maximum observed error, signed position bias, repeatability, orientation error, missed detections, and incorrect detections separately. An average error can hide a repeatable corner failure.

9. Check the order and recognize when planar guidance is wrong

Before ordering, confirm the following against the tested operating envelope:

• Presentation: Height, tilt, overlap, and part separation remain within documented limits.
• Optics: The lens covers the sensor and retains the required feature quality across the usable field.
• Sampling: The final cropped image meets the localization allocation in both axes.
• Motion: Shutter behavior, exposure, timestamps, and tracking meet the timing allocation.
• Integration: Coordinate conventions, units, software support, licensing, and rejected-target behavior are defined.
• Acceptance: Independent physical measurements meet the pickup tolerance with usable margin.

Choose another approach when the pickup requires unknown height or surface normal, hidden contact geometry, or reliable separation of stacked objects. Depending on the task, the answer may be controlled presentation, an additional view, a separate height measurement, or 3D imaging.

Do not assume depth sensing removes every ambiguity. Reflective surfaces, occlusion, calibration, and timing still need application-specific verification.

Start with the camera configuration step when the imaging requirements are established. If the height or timing budget remains unresolved, send the field of view, pickup tolerance, presentation limits, and representative images through application support before selecting hardware.