Circle occlusion picking: unknown parts on a printed detection sheet

Vision-guided picking usually starts with teaching: one model per part, retrained every time the part list changes. This guide shows the opposite approach. You print a detection sheet — a grid of black pick circles with fiducials at the corners — fix it under a camera, and let the robot pick whatever hides the circles. Nothing about the parts is taught: any part big enough to cover a circle becomes a pick point, and a full circle list means the sheet is empty. The scheme suits staged layers of mixed or unknown parts, prototype cells and bins whose contents change daily. You will get the layout rules, the lens and working-distance math, an error budget, and the failure modes that decide whether it survives on your line.

How absence detection replaces part teaching

The trick is to stop looking for parts and start looking for empty space. Print a sheet with a grid of black pick circles and a fiducial at each corner. The camera then runs two locate steps. The first finds the four corner fiducials and returns the sheet's position and rotation in the camera frame. The second sweeps the sheet for pick circles and returns the list of circles it can actually see.

The robot controller compares that list against the nominal grid. For each printed position it asks one question: is there a located circle within 5 mm of this spot? If yes, the cell is empty. If no, a part is covering the circle, and that grid coordinate becomes a pick point. The robot dives, grips, lifts, drops the part at a side station, and scans again. Every missing circle feeds the next dive until the circle list comes back complete, which is the signal that the sheet is empty.

No model, template or training step touches the parts. The only fixed geometry is the printed grid, so a new part type costs a printout, not a teaching session. Frame to grip is the same two-locate structure behind any 2D vision guided robot pick and place cycle; the sheet simply replaces taught part models with printed ones. The pose math from camera to robot is the standard eye-to-hand object pose transform.

Sheet layout: circles, pitch and fiducials

A3 landscape, 420 mm by 297 mm, carries the reference layout: a 5 by 5 grid of circles at a 60 mm pitch, spanning 240 mm square in the middle, and one 10 mm fiducial circle 12 mm in from each corner. Print black toner on matte white paper, 200 to 300 g/m², at 600 dpi; toner reads under 10% reflectance against roughly 85% for the paper, so the circles separate cleanly in a threshold image. A typical office laser printer holds registration to about ±0.5 mm over the sheet, small against a 60 mm pitch. Print two sheets from the same file: the pose is measured from the fiducials every cycle, so a swapped sheet needs no reteach.

Two size rules decide the circles and the pitch. First, every circle must be smaller than the smallest part footprint, so a resting part hides the ring completely. The placement tolerance follows from that:

A 30 mm part over an 18 mm circle tolerates ±6 mm of placement error; a 45 mm part tolerates ±13.5 mm. Second, set the pitch at least as large as the biggest footprint, so one part never straddles two circles at pick time — 60 mm clears the 45 mm worst case with margin. A part that lands between cells leaves its own circle visible and the layer scans as empty; the program guard below catches exactly that case.

Camera, lens and working distance: the worked example

Pick the field first, then the lens. The sheet is 420 × 297 mm; add 10 mm of margin on every side and the field of view must cover 440 × 317 mm. Take an illustrative 5 MP camera with 2592 × 1944 pixels of 2.8 µm on a 1/1.8-inch sensor, whose active area is about 7.2 × 5.4 mm. Magnification and focal length follow from:

m = 7.2 / 440 = 0.0164. Mount the camera 625 mm above the sheet: f = 625 × 0.0164 / 1.0164 = 10.1 mm, so a 10 mm lens is the pick. Back-check with FOV = sensor × (WD − f) / f: 7.2 × 615 / 10 = 443 mm wide and 5.4 × 615 / 10 = 332 mm high — both clear the sheet with margin to spare. The lens focal length calculator runs the same two lines for your own sheet size.

Pixel density sets the detection budget: 2592 / 443 = 5.8 px/mm, or 0.17 mm per pixel. The 18 mm circles measure 105 pixels across and the 10 mm fiducials 59, both comfortably above the 40-pixel floor where ring matching stays reliable.

Exposure and aperture are easy because nothing moves during the frame: set 10 ms at f/4 with a flood light and leave them fixed. Black circles on white paper give a two-peak histogram, so the threshold sits in a wide valley; letting auto-exposure chase that white sheet as parts cover it is what drifts thresholds — the feedback loop behind auto exposure ROI instability.

Ambient light through windows moves the same histogram; the fixes are the usual ones from ambient light control in pick and place vision: shroud the station and keep the exposure fixed.

Focus is not the constraint. At f/4 with a 2-pixel blur budget the depth of field is about ±85 mm (2 × 4 × 0.0056 mm × 615² / 10² ≈ 170 mm total), so parts stacked 80 mm above the sheet stay sharp. What does bite is parallax: a part of height h standing d mm off the camera axis hides a shadow shifted by h × d / WD on the sheet. At the sheet corner d = 277 mm, so a 40 mm tall part shifts its shadow 40 × 277 / 615 = 18 mm. The nearest neighbor circle edge sits 60 − 9 = 51 mm away, more than the shift, so a tall part cannot falsely hide a circle it does not sit on. Keep the pitch at least three times the worst-case shift: 3 × 18 = 54 mm against the 60 mm pitch.

The program: two locates and a nearest-point check

1. Locate the sheet. The first locate step searches the four corners, fits the fiducial centers and returns the sheet pose. A fixed overhead camera sees the whole sheet every frame; an on-arm camera first moves to a taught pose that frames the sheet.
2. Locate the circles. The second step returns the center of every pick circle it can match, already in sheet coordinates through the pose.
3. Run the grid check. For each of the 25 nominal positions, a short script function searches the located list for any circle within a 5 mm tolerance — a nearest-point test, a dozen lines in any robot language. A miss marks the cell occupied.
4. Dive in grid order. Approach 60 mm above the sheet, descend, grip, and confirm the grip with a vacuum or force check before lifting. The approach offset logic for vision-guided picks carries over unchanged.
5. Drop and rescan. Place the part at the side station, rescan, rebuild the pick list, and repeat until the list is complete.
6. Guard the empty case. If a freshly loaded layer scans complete — every circle visible — stop and alarm instead of pulling the sheet away; parts are resting between cells. The same alarm catches a sheet printed from a bad file.

The 5 mm tolerance has headroom by design: found circles land within about ±2 mm of nominal once distortion is handled, and neighboring grid points sit 60 mm apart, so the test cannot bridge. When picks come out consistent on most positions but drift on a few, the intermittent localization errors in sheet picking checklist isolates whether it is pose, threshold or grip.

Where the pick error comes from

Add the terms before trusting the first dive:

Short M12 lenses commonly carry around 1% distortion, and 1% of a 220 mm half-width is about 2 mm — toward the corners it dominates an uncalibrated cell. Corrected, the budget lands at roughly ±1 to ±3 mm anywhere on the sheet; uncorrected, expect ±5 mm or more at the edges. If corner picks miss while center picks land, distortion is the first suspect, not the robot.

Cameras for a sheet station

Two architectures run this program. A smart camera keeps both locates and the grid logic on the device and hands the robot a finished pick list over one Ethernet cable — no PC in the cell and no image traffic to move. A PC with a GigE Vision camera buys processing headroom and an easier path to adding 3D later, at the cost of a computer to maintain. Browse the smart cameras category or the camera finder to filter by resolution and interface.

The MV-SC5060XM-08M-WBN is a 6 MP mono smart camera with an 8 mm M12-mount lens on GigE; its rolling shutter is harmless here because nothing moves during the exposure. For a half-sheet station, the LS-SMC-0063 pairs a 1.6 MP sensor with a 16 mm lens and polarized white light at 60 fps, which removes a separate light entirely. If one station serves several sheet layouts, print a DataMatrix in a corner and let one of the fixed code readers report which grid is loaded before the locates run.

The MV-SC3050XC-16M-ATN-H adds color: a 5 MP color vision sensor on Fast Ethernet with a global shutter and an M12 mount, useful when part families differ by color and the program branches on what the sheet holds.

When a printed sheet is the wrong tool

The scheme holds while parts rest on the sheet and cover circles. It loses when:

• Orientation matters. Absence gives a position, never an angle; an asymmetric part gets gripped blind. If the grip needs orientation, that is 3D or teach-per-part territory — the 3D versus 2D system decision guide draws the line.
• Parts do not rest. Tangled, hanging or nested parts leave their circles visible and strand themselves; a 3D bin picking approach for non-planar parts sees the pile instead of the floor.
• Parts run smaller than the circles. Below the covering tolerance the layer scans empty while parts remain. Shrink pitch and circles with the tolerance formula, or feed small parts separately.
• The layer is already staged. Known sheets in known stacks do not need absence detection; the one snapshot position per layer in tray destacking method is simpler there.

Shallow piles still iterate: each dive lifts the top part and the rescan reveals the next circle, provided the grip check rejects empty dives.

First-day checks

• Threshold view: every circle solid black with no gray fringe; a gray edge means the exposure or threshold has no margin.
• Empty sheet: 25 of 25 circles found with lights on, off and ambient-only — a miss on an empty sheet is a false pick waiting to happen.
• Loaded sheet: a full layer must hide all 25 circles; any visible circle means a part or a shadow is leaking, find it before the first dive.
• Corner dives: pick at all four corners onto a marked target and measure the offset with calipers; hold ±3 mm before production.
• Grip confirm on: vacuum or force verification rejects the empty dives that parallax and piles eventually produce.
• Spare sheet: a second print from the same file, so a torn or stained sheet swaps in with zero reconfiguration.

The decision, case by case

Staged or poured layers of unknown parts, 20 to 60 mm footprint, resting flat: run the scheme as laid out above — A3 sheet, 5 by 5 grid at 60 mm pitch, 18 mm circles, 10 mm fiducials, a 5 MP camera on a 10 mm lens at 625 mm, fixed 10 ms at f/4. Expect ±1 to ±3 mm picks and 2 to 3 seconds per part. Parts below 20 mm: shrink pitch and circles using t_max = footprint / 2 − diameter / 2 and re-run the covering numbers, or feed them separately. Orientation-critical grips, tangled parts or deep mixed piles: step up to 3D vision rather than forcing the sheet. When the cell is being designed from zero, the robot vision building blocks for object location and gripping pair with this sheet as the part-detection layer.

To configure a station, start with the camera configurator and add light and trigger steps as the finder walks you through. The applications topic page collects the sibling guides, and the contact page reaches the applications team when your parts sit outside these envelopes.