Check part presence with a fixed 3D ROI over each hole

A holder with five holes, three of them used, has to leave the station with three clips seated, and when one is missing the robot needs to know which hole to refill. You get that answer from the point cloud you already capture: one small cuboid ROI over each used hole, one point count per ROI, one threshold. This walkthrough gives you the box sizes, the arithmetic that sets the threshold, the step sequence for a holder sitting in a nest, and the 3D-matching variant for a holder that arrives loose. It also shows where the check breaks — glossy clip heads, recessed clips — and what to do instead. No stickers, no QR codes.

Why point counting beats labels

An empty hole and a seated clip differ in the most direct signal a depth camera produces: the number of points between two heights. A clip that is home puts its head inside the box and returns hundreds of points. A missing hole returns dust and hole-wall glints. The count answers presence, and the ROI index answers which hole — the two things the robot needs.

Stickers and codes anchor a 2D search and add failure modes of their own: print wears off, glare hides a triangle, a code peels. A 30 mm radius search window is also the wrong shape for the job. π × (30 mm)² ≈ 2,827 mm² of scene; even after a 13 mm hole takes its 133 mm², about 2,700 mm² of holder surface stays inside the window. At 0.5 mm point spacing that is roughly 10,800 floor points, and the seated clip's 450 points drown in them. Hug the part instead.

Height is what makes a 3D box beat a 2D mask here: the holder surface and hole walls sit below the box, the clip head sits inside it. The 2D vs 3D camera ROI guide walks through when each kind of ROI wins.

What you need

• A 3D camera that returns a registered point cloud of the whole holder in one shot. On a benchtop station, projected structured light gives the densest cloud at working distances under a meter; the 3D sensors range covers the types.
• A mount above the station at the working distance the camera is specified for; the 3D camera working distance and positioning guide covers the geometry.
• Hard stops for the holder if it will sit still, or a matching model of the holder if it will not.
• A controller that runs capture, matching, the ROI steps and the robot connection.

The MV-DPS200P-02 projected structured-light 3D camera covers a station like this from a single overhead mount and feeds the point cloud straight into the ROI steps.

If the fixture is wider than a benchtop and the check can live with a coarser surface, the MV-DT01WMU ToF sensing camera runs the same ROI logic over a larger scene.

The counting, matching and robot hand-off run on a vision controller. The MV-VC3301H-128G66 carries 8 GB of memory, 5+ LAN ports and 4 built-in light channels, enough for this check plus a projector light if you add one.

Capture with the robot parked so no gripper or cable drops into the ROI volume mid-shot.

Size the box to the clip

The example running through this guide: a clip whose head is 12 mm across, seated so the head stands about 8 mm proud of the holder.

XY: half the head is 6 mm; add 2 mm of clearance per side and the box is 16 × 16 mm. The 2 mm buys placement tolerance. With the holder against nest stops repeating within 1 mm, a worst-case shift puts the head edge at 6 mm + 1 mm = 7 mm from the hole center, still inside the 8 mm half-width of the box.

Z is the decision that makes the check work: keep the box bottom above the holder surface. With a seated head at +8 mm, run the box from +5 mm to +15 mm, 10 mm tall. An empty hole then contains nothing but glints, so its count collapses to single digits. A half-inserted clip tops out below +5 mm and reads as missing — the correct answer, because it is not seated.

Give every used hole its own box and its own ROI step. One box across all three holes tells you a clip is missing but not which hole failed, and the robot needs the hole identity.

Set the threshold from a floor and a signal

A is the projected area of the clip's top face and s is your camera's point spacing at your working distance. N_floor is the highest count an empty ROI returns over 50 frames; N_signal is the lowest count a seated clip returns over the same 50, shot with the worst-looking clip on the line. The 3× multiplier absorbs frame-to-frame floor variation, and the one-fifth division keeps a clip present even if glare strips most of its points.

Fixed holder: the three-ROI pipeline

1. Locate the holder on hard stops — two dowel pins and a back stop — so placement repeats within 1 mm. That repeatability is what the 2 mm ROI clearance is paying for.
2. Capture one acquisition with the robot parked and the fixture settled.
3. Run your existing mask and filter steps to strip background and keep the holder.
4. Add one Extract 3D Points in 3D ROI step per used hole. Set each box to 16 × 16 mm in XY and +5 mm to +15 mm in Z above the holder surface, centered on the hole.
5. Count the points each ROI step returns.
6. Compare every count with the threshold — 90 in the example.
7. Output the pass/fail set as an indexed list: all pass, holder complete; any fail, that index names the hole to refill.
8. Prove it before release. Pull one clip and run 50 cycles: exactly that ROI must fail every time. Re-seat it and run 50 more: all three must pass.

Holder moves: match first, then cuboids

A holder that arrives off a conveyor or on a pallet kills fixed ROIs, because their whole basis is the nest. Match the holder, then hang the boxes on the match.

1. Keep your 3D coarse matching and 3D fine matching on the holder model, and set the model's reference point above the holes, not on a pick edge. Every box offset hangs off that reference.
2. Replace the fixed ROI steps with Extract 3D Points in Cuboid steps whose poses are defined relative to the reference, keeping the same 16 × 16 mm XY and the same +5 mm to +15 mm Z band.
3. Keep the threshold: the boxes hug the same geometry, so the counts mean the same thing.
4. Re-measure the floor with matching active. Fine-matching jitter walks each box a fraction of a millimeter, and the 2 mm clearance absorbs it; confirm the empty-ROI count stays single-digit over 50 runs spread across the holder's whole placement area.

If the matching step hands on an empty or mismatched cloud, fix that first — the 3D fine matching input mismatch checklist sorts out clouds and poses before the counting logic is worth debugging.

Several holder variants on one line call for one ROI-and-threshold recipe per part number, each validated on a known-good part — the same discipline behind thread presence checks bound to validated part recipes.

A station that adds cameras on top of matching and robot comms in the same cycle is where memory and ports start to bind: the MV-VC5525-256G70-NN steps up to 32 GB of memory, PoE camera ports and 8 built-in light channels.

Checks before the first shift

• Ambient light. Bright light raises the floor of IR-based depth. Log empty-ROI counts across a full day if the station sits near windows or hot lamps; the autoexposure noise floor on IR depth article explains why the floor climbs.
• Depth holes. Shiny or black clip heads can drop out entirely and drag a seated clip under the threshold. Merging 3 frames before counting roughly triples the surviving points while the floor stays put; missing points in a 3D camera point cloud lists the causes, and merging multiple depth frames into one gap-free point cloud covers the registration.
• Decimation. Do not decimate between capture and counting. Keeping every second point in both axes divides each ROI count by about 4, and a threshold copied across arrives wrong by the same factor — recompute T on the decimated cloud or drop the step. Point cloud downsampling distortion also shows why decimation shifts points, which matters when the same cloud places clips.
• Timing. Capture between robot moves. Vibration mid-shot shows up as missing points, not as blur, and it hits exactly the counts you are thresholding.

Hand the missing-hole index to the robot

Keep the map from ROI index to hole position with the recipe: for a nested holder the hole centers are constants, and for the matching path they are the reference point plus the same offsets you used for the boxes. The robot picks a replacement clip and places it at that position. If the spare clips come from a bin, 3D bin picking for flat parts covers the pick side, and robot vision for locating part center points covers the placement target.

Then close the loop: re-run the three-ROI check after placement. The extra cost is one acquisition, and it catches a clip that bounced out of its seat as well as confirming the repair.

When point counting is the wrong tool

• Fully recessed clips. Nothing protrudes, so the box never sees a part that is present. Tilt the camera to look into the recess, or anchor the check to a lip that does stand proud.
• Transparent clips. Depth cameras return smeared or missing points on clear plastic; neither a threshold nor frame merging rescues the count.
• Glossy domed heads that stay invisible even after merging. Image the head directly instead. The MV-PSA-12UM-C4D30-WDM area-scan dome-light vision inspection system looks down on the holder with a 300 mm dome light and a 12 MP sensor, so a shiny head reads as contrast rather than as depth, and presence becomes a 2D blob check in the same ROI footprints.

For holders with proud, opaque clips, the three-box count is the leanest answer on the bench: no labels, no extra hardware, and a per-hole diagnosis the robot can act on.

Sizing the station

Size the camera to the cloud density the threshold needs and the controller to the camera and robot connections. The camera finder filters by mounting distance and resolution, the 3D topic hub collects the set-up guides around this one, and the contact page is open for a layout review against your holder drawing and cycle time.