Multiple object detection in one snapshot: pick a full tray per cycle

Hovering the camera over a tray and shooting one part at a time is the slow habit in tray picking. Every extra snapshot buys a robot move, a settle wait, an image transfer and a locate pass. When a layer holds 15 parts on one plane, multiple object detection collapses all of that into one snapshot: the camera finds every part that matches the taught object model, and the robot picks until no match is left. This guide explains the edge-then-color locate, how to teach a model that matches every part, what the detection scores tell you, and the cycle-time arithmetic that tells you when one snapshot wins.

Why one snapshot wins the cycle

Teach the object model once, shoot the layer once, pick until no match remains. A per-part routine costs one snapshot-and-locate pass per part; multiple object detection costs one pass per layer. On a 15-part plane that removes 14 camera moves per layer, about 93% of the locate passes, and the deleted item is the expensive one: the robot move. Detection itself is the fast half. Published measurements of multi-object detection on embedded hardware average around 200 ms per frame, and locate time grows with the number of objects, the background complexity and the object's feature complexity - a few hundred milliseconds per layer, not seconds.

The pick loop does not change. Locate, transform the pose, approach, grip is the standard sequence of a vision-guided pick-and-place cycle; the only difference is that the locate hands back a list of poses instead of one. If the layer will not fit one frame, the plan changes: cover it with several taught snapshot positions, which is the case in the guide to multiple wrist camera snapshot positions over a large table. This guide handles the case you should try first - one plane, one frame.

How two-step detection finds every part

Detection runs in two steps, and the order matters. Edge detection runs first: the software matches the taught part contour against the edges in the frame, and that match carries position and orientation. Color validation runs only after edges have succeeded, so color never rescues a failed edge match - it rejects look-alikes such as an unprinted version of the same part, a gasket next to it, or the tray pattern itself. The division of labor is the same as in color pattern matching for part localization: geometry places the part, color confirms it.

After a successful match, the overlay outlines the part and shows a detection score as a percentage. Other detected objects that are not model matches get their own outline, and you can select each one to read its score. A score far below the rest of the layer points at lighting drift, a changed surface finish, or a model taught from an unrepresentative sample - retouch the model or fix the light before trusting the pick.

Teaching a model that every part matches

Teach from a snapshot of a good part lying where parts actually lie: on the tray, in the layer, not held in a fixture at a different angle. Use the three view modes while you tune. The raw view is for framing, the edge view shows the contour the system will match, and the color view shows the color signature as the system sees it. Toggling between them is the fastest way to catch glare, a background that competes with the part, or a light that drifted since the last teach.

Color validation has two teaching forms. The automatic method paints layer masks over the taught model: you keep the colors that belong to the part and erase the ones that belong to the background with marker, eraser and rectangle tools. The parametric method, used for symmetric objects, stretches or shrinks a color selection zone that follows the object's contour.

Worked example: 15 caps on one tray layer

Inputs, each with its source:

• N = 15 parts per layer, all on one plane and all inside one field of view - the case this guide is built around.
• Locate passes per layer today: 15, one hover-and-shoot per part.
• Locate passes per layer with the one-snapshot method: 1.
• Locate processing per pass: about 0.2 s, from published embedded-vision measurements of multi-object frames averaging around 200 ms; treat it as a class, not a spec.
• t_reposition, your measured time per hover-and-shoot pass for travel, settle, trigger and transfer; the illustrative value below is 2 s, and you replace it with your stopwatch number.

Rule in words: one snapshot replaces N locate passes with one, so the layer saves the N - 1 deleted passes, each worth its reposition plus its locate, minus the small extra cost of locating 15 objects inside a single frame.

Arithmetic for the 15-cap layer:

• Passes removed: 15 - 1 = 14 per layer, and 14 / 15 ≈ 0.93, so 93% fewer locate passes.
• Locate processing removed: 14 × 0.2 s = 2.8 s per layer.
• Reposition time removed at the illustrative t_reposition = 2 s: 14 × 2 s = 28 s per layer.
• Total: 28 s + 2.8 s ≈ 31 s per layer, which is 31 s / 15 ≈ 2 s per picked part.

The saving scales linearly: at t_reposition = 5 s the same layer saves 14 × 5 s + 2.8 s ≈ 73 s. The threshold sits on the other side of the comparison. The single locate that carries 15 objects costs a few hundred milliseconds more than a single-object locate, so any reposition you measure above about half a second already loses to the one-snapshot plan, and real wrist repositions run in seconds. Commit to one snapshot on any single-plane layer whose parts fit the field of view.

Check the field of view and the plane first

Three conditions make one snapshot legal. All parts sit on one plane, because the camera looks straight down and a stacked or leaning part presents a different contour. All parts fit inside one field of view with a clear border, because a part cut by the frame edge cannot match. And no parts overlap, because overlapping edges break the contour match for both.

Field of view follows the pinhole estimate FOV ≈ WD × sensor width / f. At a 300 mm working distance, an 8 mm lens and a 1/2-inch-class sensor about 6.4 mm wide, the estimate gives 300 mm × 6.4 mm / 8 mm = 240 mm of coverage. A 400 mm tray needs a wider lens, a longer standoff or a larger sensor, so run your own numbers before fixing the mount.

Lighting and color validation that hold up in the cell

A color signature is only as stable as the light on it. Windows, overhead LEDs and a neighboring cell shift colors between the teach and the run, and the symptom is low scores on parts that taught perfectly. The fixes are the standard responses to ambient light variation in pick-and-place vision: shield the snapshot area, keep exposure fixed instead of auto so the colors stay comparable, and re-sample the color zones after every lighting change. On glossy caps and labels, a polarizer on the light tames the specular highlights that move with the robot's approach angle.

Cameras and controllers for snapshot picking

For a single-camera station the locate runs on the device, no PC in the loop. The models below are color, global-shutter parts sized for snapshot work: the MV-SC6050C-08M-XBN adds a built-in multi-color light source for the color-validation half, the MV-SC3050XC-16M-ATN-H keeps the same 1/1.45-inch sensor format over Fast Ethernet, and the MV-SCB007EC-12S-WBN stays at 1.6 MP color with a 12 mm manual-focus lens. Compare them in the smart cameras category and the camera finder.

Cells that grow past one camera - a second station, a code reader, several snapshot positions - move the locate onto a vision controller. The MV-VC2140-128G40-1T carries an Intel N-series processor with 8 GB of memory, four built-in light channels and 3-4 LAN ports, so the locate and the robot handshake live in one box.

Send us your tray size, part size and cycle target through the contact page and we will size the snapshot position and return a parts list, or start from the camera finder and configure from there.

End-of-pick logic and empty layers

The loop is locate once, then pick each returned pose until a locate returns nothing. The empty result is the normal end of a finished layer, not a fault. The path that needs teaching is the unplanned one: a misfeed, a single stray part, or a layer that ran out early. Give the no-match result its own exit so the cycle ends cleanly instead of hunting; the restart pattern in the guide to recovering from a vision no-part error maps straight onto this loop.

When one snapshot is the wrong plan

The method assumes one plane, one field of view and non-overlapping parts. Break an assumption and pick a different plan:

• Stacked layers in a tray: teach one snapshot position per layer, the way tray destacking vision handles a full tray.
• Mixed SKUs on one layer: separate them with vision item types, as in sorting parts by face into tray layers.
• A table wider than one frame: spread coverage over several taught snapshot positions rather than forcing a single shot.
• Symmetric parts: rotation tolerance and mirrored poses need their own handling, covered in symmetric part orientation matching with rotation tolerance.
• Shiny bare metal with no stable color signature: drop color validation, rely on edges, and matte the background with the fixture.
• Overlapping or tangled parts: separate them mechanically first, or move the problem to 3D.

Checks before you commit

Run this list on a full trial layer before you release the program to production:

• One plane confirmed: no stacking, leaning or overlap anywhere in the snapshot.
• All N parts inside the field of view with a clear border margin.
• The edge view shows a clean, closed contour on a part at the frame center and on one at the frame edge.
• The color view separates part from tray under the run lighting, not the bench lighting.
• Scores across the layer sit close together; if one part sits far below the rest, re-teach instead of lowering thresholds.
• t_reposition measured with a stopwatch and worked into the formula, with the one-snapshot decision confirmed.
• The no-match exit taught and tested once with an emptied layer.

When the list passes, order the layer trial: one taught model, one snapshot position, a full tray and a stopwatch. More build notes like this sit on the systems topic page.