Wrist camera locate empty tray slot: teach the cavity as a part

A tray lands where the operator drops it, at any angle, and the robot must still fill each pocket exactly once. The hard part is not finding the part to pick — it is proving a pocket is empty before you release into it, because a part dropped onto an occupied pocket means scrap and a jammed cell. The fix is to teach the open pocket as its own template on a contrasting background, then nest a second camera locate for that template inside the pick-and-place cycle, so the robot releases only into a slot it has positively identified as empty.

This guide sizes the snapshot geometry, sets exposure and f-number from worked numbers, and puts a placement error on the table before you order anything. The method runs on any camera that teaches a template and returns a pose — a smart camera on the wrist or a fixed camera over the cell.

How do you locate an empty tray slot with a wrist camera?

Teach the open pocket as its own template: back one empty cavity with a contrasting sheet, mask every occupied pocket, and run the normal teach, so the locator reports the empty slot's position and angle instead of a part's position.

A template matcher scores each candidate window against the taught image and returns position and angle with the score. An occupied pocket changes the window's interior completely — part surface instead of background sheet — so its score collapses, while a genuinely open pocket scores high. Emptiness becomes a positive detection with coordinates, not an absence you have to reason around.

The classic teach scene uses a yellow sheet behind the open pocket and paper masking over the filled ones, then the ordinary teach steps from there. Start with the accept threshold near 70% and let a score log tighten it; the tuning procedure is in the steps below.

Why nest the slot locate inside the pick sequence?

Run one camera locate for the part at the pick snapshot and a second for the empty pocket at the tray snapshot, and let the second run only after the gripper confirms it is holding a part.

Nested this way, the slot locate cannot fire without a part in the gripper, which closes the two failure paths that matter: no part found at the pick end, and no open slot found at the place end. Each path gets its own branch — skip the pick, or keep the part and skip the place — instead of a blind release.

The cycle reads: snapshot over the pick area, part locate, grip, hold check, move to the tray snapshot, slot locate, place. That is the same skeleton as the 2D vision guided pick and place cycle, with the slot locate replacing a fixed drop offset; the layer-by-layer version of the same discipline is one snapshot position per tray layer.

What does the background around the open pocket need to look like?

Give the locator a background that matches nothing else in the scene: a matte yellow or black sheet under or behind the open pocket, with paper or caps masking the filled pockets, is what lets one taught image work at any tray angle.

Check the teach image histogram before you accept it: background and pocket rim should sit in two groups at least 30 grey levels apart at the working exposure, or the margin between empty and occupied stays thin. Matte beats gloss, because a specular rim mirrors whatever stands opposite and can imitate the background colour.

On shiny trays, polarized light or a diffuse source at a steep angle flattens those highlights; the underlying logic is the camera-light-part geometry in lighting geometry for image contrast. If the tray floor is light and the parts dark, invert the scheme: dark sheet, bright rim, and the score still separates.

How high above the tray should the snapshot position sit?

Work backwards from the frame you need: with a 12 mm lens on a 1/2.9"-format sensor, a snapshot 240 mm above the tray rim sees a 90 × 68 mm frame, which holds a 44 mm pocket plus about 23 mm of tray wall on every side.

Worked example, assumptions on the table: a 1.6 MP, 4:3 sensor of the 1/2.9" format — about 4.5 mm wide, 1,440 × 1,080 pixels — a 12 mm lens, camera 240 mm above the rim.

FOV = 4.5 mm × 240 mm ÷ 12 mm = 90 mm wide; 3.4 mm × 240 mm ÷ 12 mm = 68 mm high. Pixel scale = 1,440 px ÷ 90 mm = 16 px/mm, so the 44 mm pocket spans 44 × 16 = 704 px — far above the 4 to 8 px a matcher needs across the smallest edge it must distinguish.

Context margin = (90 − 44) ÷ 2 = 23 mm of tray per side; keep at least 10 mm so the template carries pocket walls and cannot match bare background. Magnification m = 4.5 ÷ 90 = 0.05. Height scales the frame linearly: the same lens at 300 mm covers 113 mm, so raise the snapshot instead of changing glass when a bigger tray arrives.

The MV-SCB007EC-12S-WBN comes with a 12 mm manual-focus M12 lens over a 1/2.9" sensor, 1.6 MP colour, global shutter, Fast Ethernet.

For a tray that must fit one frame, the MV-SCA002M-03S-WBN-SR pairs a 3 mm lens with a fixed 240 mm focus — the same snapshot height as this example.

How do you teach and nest the two locates, step by step?

Prepare one teaching scene with a single open pocket, teach the part and the empty slot as two separate templates, then wire the slot locate to run only after the gripper reports a confirmed hold.

1. Bound the tray zone. Measure where trays can land and how they rotate. If the zone exceeds the frame from the example, add an approach point per region; multiple wrist camera snapshot positions to cover a large table shows the pattern.

2. Set the tray snapshot position. Camera axis square to the tray plane, 240 mm above the rim in the example, height repeatable to ±2 mm; a hard stop or a checked move beats a free-space teach.

3. Build the teaching scene. One pocket open over the contrasting sheet, the rest masked with paper or caps, shot from the snapshot position at the working exposure and f-number.

4. Teach the empty-slot template. Keep the pocket rim and the background inside the window, exclude the neighbouring pockets, and mask anything that moves between cycles if the tool allows it.

5. Teach the part template at the pick snapshot. Pick a stable feature rather than the whole silhouette, as the guide on teaching long thin parts by feature argues.

6. Program the cycle. Pick locate → grip → hold check → tray snapshot → slot locate → place pose from the slot's position and angle → release. The slot locate is a second camera locate inside the same program, not a separate pass.

7. Set the threshold from data. Log every score across 30 random tray orientations and part positions, then set the threshold halfway between the lowest true empty-slot score and the highest occupied-pocket score, starting near 70%.

8. Write both failure branches. No part at the pick locate follows the no-part path, the same logic as restarting the robot cycle after a vision no-part error; no slot at the place locate keeps the part in the gripper and skips the release.

9. Prove it. Run 30 cycles with tray and parts at random orientations; require zero releases onto occupied pockets and zero open slots left unfilled while a part waited.

How accurate is the final placement?

Budget ±0.06 mm from the image plus a few tenths from the wrist: at 16 px/mm a one-pixel error is 1 ÷ 16 = 0.063 mm, a ±2 mm height error adds ±0.33 mm at 40 mm off-centre, and 1° of tilt adds 0.17 mm across a 10 mm pocket depth.

The arithmetic: height error changes scale by 2 ÷ 240 = 0.83%, which displaces a target 40 mm from the frame centre by 40 × 0.0083 = 0.33 mm; tilt of 1° shifts the apparent centre of a feature 10 mm deep by 10 × tan 1° = 0.17 mm. Worst case lands near ±0.6 mm, so a pocket with 1 mm or more of clearance per side places cleanly.

Below that, buy pixels: the MV-SC5060XM-08M-WBN raises the count to 6 MP over GigE with an 8 mm M12 lens. More pixels over the same frame halve the 0.063 mm term directly.

What exposure and f-number keep the snapshot sharp?

Stop motion blur at one pixel and cover the pocket with depth of field: at 16 px/mm that means 2.5 ms or shorter if the wrist may still carry 25 mm/s of residual motion, and f/8 for about 22 mm of depth at the 240 mm snapshot.

The arithmetic: one pixel is 1 ÷ 16 = 0.0625 mm, so 0.0625 ÷ 25 = 2.5 ms. With c = 0.0032 mm and m = 0.05, depth of field is 2 × 8 × 0.0032 × 1.05 ÷ 0.0025 = 21.5 mm at f/8 and 10.8 mm at f/4; a 10 mm pocket plus its rim needs the f/8 stop so rim and floor stay crisp together.

A 2.5 ms exposure needs real light on the tray, which is the argument for light at the wrist. The LS-SMC-0063 carries a 16 mm lens, its own white light with polarized and transparent-light options, 1.6 MP and 60 fps over GigE, running inspection on the device.

Trigger the snapshot after motion settles. A locate shot mid-deceleration breaks the blur budget no matter how short the exposure is.

When does the open-slot locate lie to you?

The template still scores high on an occupied pocket when a part overhangs the rim just enough to hide the background, or when a glossy tray wall mirrors the sheet's colour — so judge by score margin, never by a single passing frame.

• Overhanging neighbours. A part leaning across a pocket shades its background. Use steeper light and a taller teach mask, treat scores within 10 points of the threshold as uncertain, and re-shoot or skip; fixing wrist-camera teach failures on small parts covers the teach side.
• Specular rims. Gloss imitates the background colour. Polarized light or a diffuse source at an angle kills the reflection.
• Frame clipping. A pocket half outside the frame cannot match. Keep the tray zone inside the field of view or add snapshot positions.
• Mixed tray types. A template from another tray design can false-match. Identify the tray first — dimensions alone are enough, as part identification by dimensions shows — then load that tray's template.
• Clear or mirror-finish parts. The background may show through and hold the score up. Polarized light helps; when 2D cannot separate it, a 3D sensor decides.

A locate that returns no candidate at all — not a low score — usually means the template never survives the search; why a wrist camera locate finds no match sorts those causes.

Next step: run your own tray through the arithmetic above, then shortlist hardware. The camera finder filters smart cameras and vision sensors by mount, resolution and shutter, the configurator builds camera, lens and light as one kit, and the applications topic page carries the neighbouring patterns. If the cell needs a second opinion, contact the applications team.