Grasp candidate selection and ranking in 3D bin picking

You have one 3D scan, thirty parts, and a robot that can attempt one grasp. The matching stage tells you where every part sits; it does not tell you which grasp survives contact with the bin. That choice is the difference between a cell that runs all shift and one that faults on collisions. This guide gives a scoring rule that ranks grasp candidates from a single point cloud — height, tilt, clearance and point density — with default thresholds, a worked example on a 600 × 400 mm bin, and the checks to run before you commit hardware. It is written for integrators building bin-picking and palletizing cells from a 3D camera, a vision controller and a robot.

The rule: send the robot to the highest clear grasp

Rank, do not hunt. Score every grasp candidate the scan produced, send the robot to the top score, and re-scan before the next attempt. Height carries the largest weight because most collisions come from reaching past a taller neighbor, not from a bad pose. A published automotive palletizing cell selects candidates exactly this way — by height, orientation and grip-point position — and sustains 8 s per part while placing parts into guide bars at 0.5 mm tolerance. Ranking is production-proven, not a research nicety.

The rule, in full:

• Reject any candidate regardless of score when tilt exceeds 15°, clearance falls under 10 mm, or grasp-face point density falls under 50%.
• Score the survivors on height, tilt, clearance and density (formula below) and pick the top score.
• Re-scan after every grasp attempt in a bin. On a single-layer conveyor where parts cannot shift, keep ranking from the one scan.
• On ties within 5 points, take the candidate with more clearance.

From one point cloud to ranked candidates

Ranking sits between matching and motion planning. The pipeline in a working cell runs in five steps:

1. Cut the cloud. Fit the bin floor, delete walls and floor, keep the points that belong to parts. The plane fit on the empty floor also gives you the height reference for the height term.
2. Match poses. Surface-based matching of the part CAD model against the cloud returns a 6-DOF pose per visible instance — the same approach 3D point cloud template matching uses for deviation measurement. Flat stamped parts need extra care here; 3D bin picking for flat parts covers that geometry class.
3. Generate approaches. For each pose, test the gripper approach axes the wrist can reach: straight down first, then the tilts. Variable pick approach offsets shows how to shift the grasp point when the ideal axis is blocked.
4. Collision-test each approach. Sweep the gripper envelope along the approach vector and measure the distance to the nearest point that does not belong to the candidate. That distance is the clearance term.
5. Score the survivors and return the winner's coordinates in robot frame, through the hand-eye calibration.

Published graspability methods compute a comparable score from a single depth map alone, which is why height-map scoring still works when you lack a full CAD match. Occlusion is the other input problem: shadowed faces read as missing points, and merging multiple depth frames into one gap-free cloud fills those holes before you score.

The four numbers and their default thresholds

Here h is the grasp-point height, h_max the tallest point in the scan, θ the tilt in degrees and c the clearance in millimetres. The tilt penalty is deliberately steep. A part held at angle θ presents a height spread of

over its length L, so a 120 mm bracket at 15° varies 31 mm along its face (120 × sin 15° = 31 mm) — the limit of what a parallel gripper still seizes square. Past that, rigid fingers contact edges instead of faces. Compliant fingers or a magnetic gripper tolerate about 20°; rigid vacuum cups keep the default. Magnets need more room, not less: they lift neighbors, so raise their clearance floor to 15-20 mm.

Worked example: four brackets in a 600 × 400 mm bin

Inputs, each with its source:

• Bin interior 600 × 400 mm; stamped steel brackets 120 × 60 × 20 mm — a typical flat-part case.
• 3D camera assumed at 1.0 mm XY point spacing and ±0.2 mm height noise on a flat plate; measure both on your own sensor before trusting any threshold.
• Parallel gripper, 80 mm envelope, rigid fingers: tilt floor 15°, clearance floor 10 mm, density floor 50%.
• Expected grasp-face points: (120 / 1.0) × (60 / 1.0) = 120 × 60 = 7,200 points at full visibility.

One scan returns four matched candidates:

The tallest point in the scan is A's grasp point, so h_max = 83 mm.

A: H = 100 × 83 / 83 = 100. T = 100 − 6 × 6 = 64. C = min(100, 4 × 25) = 100. S = 0.40 × 100 + 0.25 × 64 + 0.25 × 100 + 0.10 × 85 = 40 + 16 + 25 + 8.5 = 89.5.

B: H = 100 × 61 / 83 = 73.5. T = 100 − 6 × 3 = 82. C = min(100, 4 × 40) = 100. S = 0.40 × 73.5 + 0.25 × 82 + 0.25 × 100 + 0.10 × 90 = 29.4 + 20.5 + 25 + 9 = 83.9.

C: H = 100 × 78 / 83 = 94. T = 100 − 6 × 19 = −14, floored to 0 — and 19° fails the 15° reject anyway. Its c of 8 mm is under the 10 mm floor, so C is out twice over; its score of 52.6 is irrelevant.

D: H = 100 × 55 / 83 = 66.3. T = 100 − 6 × 1 = 94. C = min(100, 4 × 60) = 100. Density of 30% is under the floor: 7,200 × 0.30 = 2,160 points on a face that should carry 7,200, so the pose behind it is not trustworthy. D is rejected despite S = 78.0.

Decision: the robot takes A (89.5) over B (83.9). The 5.6-point margin is height-driven, exactly what you want in a first grasp. After A leaves the bin, re-scan: B usually rises to the top, and C may clear its floors once its tall neighbor is gone. Height noise of ±0.2 mm is 2% of the 10 mm clearance decision, so noise does not reorder these candidates.

The camera that feeds the ranking

Every threshold above resolves against the sensor's point spacing and height noise. To hold 1.0 mm spacing across a 600 mm bin you need 600 depth pixels across the field, 600 × 400 = 240,000 over the floor. Double that for margin after filtering and you are specifying about 1 MP of depth resolution (1200 × 800). The camera resolution calculator runs this arithmetic for your own bin size.

For the snapshot itself, a structured-light stereo camera returns the whole cloud while the robot waits outside the bin:

The MV-DPS200P-02 projected structured-light camera is the alternative when parts are matte and the bin is shallow:

Shiny stamped steel is the hard case: bin picking reflective metal parts covers the exposure and geometry tricks that keep density above its floor. On a moving belt the sensor choice changes — 2D or 3D camera for conveyor pick and place trades snapshot cameras against profilers — and a wrist-mounted sensor shortens the camera-to-part distance at the price of single-snapshot limits.

Volume, placement and the controller

The same cell carries two smaller vision jobs. First, stop feeding the output when the layer is full: a volume-measurement 3D camera over the target container measures remaining capacity directly, the trick a published palletizing cell uses to decide when a bin can still take parts. Mount it overhead, treat its volume output as a PLC bit, and the robot stops before it buries the layer.

Second, confirm placement. With a 0.5 mm insertion tolerance, a fixed 2D camera over the guide bars catches a tilted insert before the next cycle buries it. A 20 MP mono USB3 camera such as the MV-CS200-10UM does that at normal C-mount working distances.

Compute cost is not the bottleneck. Scoring four candidates against a few hundred thousand points is roughly one million point-to-envelope distance tests — single-digit milliseconds on one current x86 core. The surface match dominates the vision budget, and robot motion dominates the cycle. The controller also sequences camera, light and PLC:

The MV-VC3301H-128G66 brings 8 GB of memory and 4 built-in light channels to that job. Two calibration details decide whether the winner's coordinates mean anything: hand-eye calibration between camera and robot (3D camera robot integration checks), and height-aware XY scaling, because a tall stack shifts the apparent position of parts below it (scale compensation for part height variation). For the cell-level picture — camera mount, lighting, network, safety — the bin picking 3D vision technical requirements page walks the same integration path with a robot-vision stack.

When the highest part is the wrong pick

• Leaning parts. A part resting against a taller neighbor can pass the grasp-plane clearance test and still drag the neighbor along. Test the full swept volume from approach start to grasp, and reject when foreign points sit inside it, not just beside the fingers.
• Magnets. A magnetic gripper lifts whatever the field touches. Raise its clearance floor to 15-20 mm and re-rank; the highest part is often the one whose neighbors the magnet drags up.
• Sparse returns. Reflective or thin parts push density below its floor and the ranking empties even though parts are visible. Fix the cloud, not the score: 3D matching on thin point clouds and the reflective-parts guide above both attack the input.
• Tight single layers. When every candidate clears the floors, order matters: pick the perimeter first so each grasp frees its neighbors instead of boxing them in. Rejecting pick candidates too close together is the tiebreak that implements it.
• Motion between scan and grasp. On a belt the part travels while the robot moves; scan upstream and aim ahead, or the score was true of a position the part has already left.
• A ranking that is right and a robot that lands wrong is an alarm, not a vision bug: a vision bin-picking robot position alarm usually traces to calibration or frame hand-off, not to the candidate list.

Checks before you commit the cell

Run these eight checks with numbers in hand before ordering hardware:

• Flat-plate height noise at or under one tenth of the clearance floor — at a 10 mm floor that is 1 mm, and the 0.2 mm above passes with room.
• Point spacing at or under one fifth of the smallest gripped feature — 1.0 mm against a 20 mm bracket thickness is one twentieth.
• Tilt floor matched to the gripper: rigid default, about 5° more for compliant fingers or a magnet.
• Clearance floor of 10 mm for parallel grippers and 15-20 mm for magnets, measured to the modeled envelope including hoses and cables.
• Density floor applied after preprocessing, not on the raw cloud.
• Re-scan wired as a PLC event after every grasp attempt, success or fault.
• Hand-eye residual at or under one fifth of the placement tolerance — 0.1 mm against a 0.5 mm insert.
• Tie rule logged: candidates within 5 points go to the one with more clearance.

Start the hardware from 3D sensors or the camera finder, run the whole stack through the system configurator, and browse the 3D topic page for the related guides. If your part mix breaks the defaults above, the contact page reaches the applications team with your scan data.