Bin picking non planar parts with 3D vision guidance

Picking small, non planar workpieces -- clips, brackets, springs, moulded connectors -- out of a bulk bin is one of the harder vision-guided robotics jobs. The parts overlap, they can sit in any orientation, and there may be no accessible flat face to grab. A 2D camera can locate visible features without resolving the depth and orientation needed for a reliable grasp.

A 3D pipeline captures the bin as a point cloud, matches it against the part's CAD model, estimates six-degree-of-freedom poses, and evaluates grasps against the surroundings. The sections below explain the stages, the numbers that govern feasibility, and how to structure a reusable workflow. If your parts are flat or presented in known orientations, simpler guidance may be enough; see the section near the end on when bin picking is the wrong tool.

Why planar methods fail on non planar parts

For a flat washer or a gasket presented at a known height, a top-down image can provide position and rotation. The gripper approaches a predictable face, provided that the presentation and gripping method support it. That assumption collapses for a clip lying on its side, hooked over another clip, or standing on edge. Two image-plane poses can look similar while needing grips on opposite faces.

The fix is to stop treating the part as a shape in an image and treat it as a solid in space. A 6-DoF pose (three translations, three rotations) plus the part's known geometry lets the software reason about which faces are exposed, which way the gripper must approach, and whether the wrist will collide with neighbouring parts. For unconstrained bulk parts, a single greyscale image generally cannot resolve those questions reliably.

Capturing the bin as a point cloud

The first stage converts the bin contents into XYZ points. Two sensor families are common. Projected structured-light and stereo cameras acquire an area of the scene without sweeping a laser line across the bin. Some acquisition methods require multiple exposures, so check the complete capture sequence and motion sensitivity. A projected structured-light 3D camera is one candidate to evaluate.

Laser profilers acquire the scene line by line, normally requiring controlled relative motion to build a surface. Evaluate the actual material and viewing geometry rather than assuming one method handles reflective or dark parts better. A binocular single-line profiler such as the [MV-DP3062-01D V2.0](product:MV-DP3062-01D V2.0) represents that acquisition approach. The 3D sensors category carries both families, and the camera finder filters them by type.

Whichever family you choose, three numbers decide the outcome:

• Point spacing on the part surface. As an initial sampling target, aim for 15 to 20 points across the smallest feature the gripper touches. For a 10 mm feature, that means roughly 0.5 to 0.7 mm spacing. Validate this target against real matching and grasp results.
• Field of view versus bin width. The sensor must see the working area of the bin. Include the walls in the scene model, and check where they obscure parts near the edges.
• Measurement uncertainty versus gripper clearance. As an initial screening rule, reserve no more than one third of the clearance for depth uncertainty. With 1 mm clearance, that allocation is about 0.33 mm. Calibration and angular pose error still need their own allowances.

Here, L is the distance from the estimated part origin to the grasp point, and θ is angular pose error in radians. For example, an angular error of 0.01 rad at a 50 mm offset contributes approximately 50 × 0.01 = 0.5 mm of grasp-point displacement. This small-angle estimate explains why a stable depth reading alone does not prove adequate pick accuracy.

If the numbers do not close -- the sensor that covers your bin cannot resolve the feature -- consider a smaller field of view with multiple captures or a scan axis. Matching parameter tuning cannot recover geometry that was never measured.

CAD-based pose estimation by point-cloud matching

With the cloud captured, the software matches it against a reference model of the part, often derived from CAD. A common approach is surface-based matching: sample the CAD model into a cloud, then search for rigid transformations that align that sample with observed surface points. Feature descriptors and hypothesis scoring help limit the search, but processing time depends on scene size, model complexity and implementation.

For a part like a clip, three practical points matter:

1. Symmetry and self-similarity. A part with near-symmetric features yields multiple near-equal matches. Define equivalent orientations where appropriate, and reject ambiguity when it changes the required grasp.
2. Occlusion modelling. The matcher should score matches against the visible portion of the CAD surface, rather than demand the whole model be present. Partially buried clips may match, but they still need a separate graspability check.
3. Confidence thresholds. Validate the acceptance threshold against realistic appearances -- oily, tangled, edge-on -- rather than only the clean sample part used during setup.

The output per accepted instance is a transformation matrix from the part's CAD frame into the camera frame. Everything downstream depends on consistent units, frame definitions and calibration. The 3D fine matching input checks explain why matching clouds and poses must use compatible coordinate conventions.

Collision-aware grasp selection

A pose is not a grasp. Between the vision output and a successful pick stands grasp planning: for each detected instance, generate candidate gripper poses in the part frame, then score each one against the live scene. Candidates can come from a taught set or suitable CAD-based generation methods.

The scoring usually checks, in order:

1. Reachability. Can the robot put the gripper there within joint limits? Candidates outside the reachable volume of the mounted robot are discarded before anything else.
2. Collision with the scene. Test the gripper body, fingers and swept approach path against the bin geometry and point cloud, with a clearance allowance. Treat unseen volume conservatively rather than assuming missing points mean empty space.
3. Grip quality. Does the contact geometry actually hold a non planar part -- opposed contacts where required, enough closure travel, and adequate retention under acceleration?

The highest-scoring acceptable grasp becomes the pick candidate. A runner-up can support recovery only if the scene remains valid; otherwise, capture and plan again. A vision controller must be sized against measured processing time and peak memory use, not point count alone. The MV-VC5525-256G70-NN has 32 GB of memory and PoE camera ports. Confirm camera interface, power requirements and software compatibility separately. Compare the available vision controller options against those requirements.

One pipeline, many parts: build it as a workflow

The mistake that turns a working demo into a maintenance burden is hard-coding the part into the pipeline -- fixed regions of interest, thresholds tied to one CAD model, and grasp poses baked into a robot program. A better structure separates reusable processing stages from part-specific recipe data:

With that split, adding a new clip starts with importing its CAD model, teaching suitable grasp poses and validating acceptance settings. The pipeline structure and communication protocol can remain consistent, although each recipe still needs qualification. It also makes recognition testable on recorded clouds, so you can regress a new part against existing recipes and catch conflicts before deployment.

Verify the pipeline before you trust it

Before the first production pick, run these checks:

• Capture a bin with parts at deliberately awkward poses: on edge, tangled and near walls. Confirm that each result is either an acceptable pose or a deliberate rejection. A confident wrong pose is more dangerous to the process than a missed detection.
• Measure pose repeatability using one static part captured ten times. Compare both translation and rotation variation with your grasp error budget. Repeatability does not establish absolute accuracy, so also check against an independent reference.
• Test rejected collision candidates in offline simulation or a non-moving planning mode. Verify the rejection reason without commanding the robot into the rejected path.
• Time the full loop -- capture, match, grasp planning and robot motion -- with margin for re-capture after disturbed picks.
• Verify grip detection, dropped-part handling and the response to missing or stale vision results before enabling automatic production.

A 3D inspection checklist mindset pays off here. Frame and calibration mistakes can resemble recognition failures; the 3D ROI coordinate-frame troubleshooting guide covers checks for incorrectly oriented geometry. For communication, calibration and controller compatibility, use the 3D camera and robot integration checks.

When bin picking is the wrong answer

Three cases where the usual answer should be questioned:

• Presented parts. If the parts arrive in a bowl feeder or on a dunnage tray, a 2D check and a fixed pick can be simpler, faster and more repeatable than full 3D matching.
• Parts that tangle irrecoverably. If clips nest so badly that no grasp is reliably reachable, fix the bulk presentation first; vision cannot untangle a mechanical knot.
• Reflective or transparent parts. Structured light and stereo can struggle with clear or mirror-like surfaces. Test actual production samples early, before committing the cell design.

A short selection checklist

• Field of view covers the working bin area, with wall occlusion assessed.
• Point spacing starts near feature width / 20 and is validated on the gripped feature.
• Depth uncertainty fits its allocated share of the clearance budget on the real surface.
• Calibration, translation error and angular error are included in the total grasp error budget.
• CAD models for every part family use consistent units and frame conventions.
• Taught grasp sets include collision allowances for unseen volume.
• Recorded clouds from difficult bins are retained as regression tests.
• Recovery includes re-capture after disturbance and rejects stale grasp candidates.

If you are assembling a cell from scratch, use the system configurator to begin selecting the vision components. Before settling the capture geometry, review the 3D camera working distance and positioning guide. For questions about a specific part, contact us with the CAD model, bin dimensions and required cycle time. Reviewing the geometry before selecting sensors can prevent an unsuitable cell layout.