Tray destacking robot vision: teach one snapshot position per layer

Emptying a stack of trays sounds like a pallet-pattern job: a line pattern from the full stack down to the last tray, the approach offset by the pallet sequence, and a Camera Locate node to find the parts. On a Universal Robots controller that fails on the second pallet point, because a Camera Locate node keeps exactly one taught snapshot position and refuses to capture from anywhere else. What works instead: one calibrated snapshot position per tray layer, a layer counter in place of the pallet pattern, and the numbers — lens focal length, field of view, pixel size, exposure, layer spacing — to teach those positions once and empty the stack cleanly.

Why the pallet pattern cannot move the snapshot

A Camera Locate URCap stores what you taught it: one TCP pose for the snapshot and one calibration plane under it. Before every capture it compares the robot's actual pose with that taught pose, and it stops with a "not a snapshot position" fault when they differ. The pallet sequence changes the waypoints around the node, never the taught pose inside it, so the second layer photographs from one tray thickness lower and the check fails.

The geometry behind the offset idea is sound. When the camera and the stack both drop by one thickness, their relative geometry is unchanged and the locate result stays valid. The limit is the software: the node takes a taught position, not a computed one. If the fault persists after re-teaching, the pose comparison itself is the place to look — fixing Camera Locate snapshot position faults walks those checks in order.

Do not work around it by leaving the camera high and only lowering the picks. A downward camera turns image positions into XY on one taught plane, and a part sitting Δz below that plane lands off-axis by:

Here r is the part's distance from the optical axis and WD the lens height above the taught plane. At 460 mm of height and a part 200 mm off axis, each millimetre of height error moves the pick 200 / 460 ≈ 0.43 mm sideways. One uncorrected 40 mm tray thickness is about 17 mm at the tray edge. The snapshot position has to follow the top surface; that is the whole design.

The structure that works: a layer counter, not a pallet pattern

Give every layer its own Camera Locate node and its own taught snapshot position, then sequence them with a counter:

layer = 1
while layer <= LAYERS
    MoveJ  Snapshot[layer]     # taught poses CL_1 ... CL_N
    wait   0.25 s              # let the wrist settle
    Locate[layer]              # the node taught for this layer
    pick every part it returns
    wait   TRAY_REMOVED        # operator confirm or photocell
    layer = layer + 1

The loop photographs a layer, picks it empty, waits for the tray to leave, and drops to the next taught position. An operator can pull the trays by hand or the robot can stack them off; the counter does the bookkeeping the pallet node would have done, and vision re-finds the parts on every tray, so a tray sitting a few millimetres off needs no waypoint touch-up. It is the same habit as covering a large table with several wrist camera snapshot positions: each region gets a pose taught for its own geometry and the program picks between them. When the layers hold different part faces, the same structure drives sorting parts by face into tray layers. The pendant-side settings for this layout are covered in configuring multi-position destacking on a wrist camera.

Step by step: teach one position per layer

1. Measure the tray thickness with a caliper across five trays and use the mean. A spread of ±0.5 mm or less is fine for computed heights; a wider spread means teaching each position from the real stack instead of a number.
2. Build the full stack, lay the calibration grid on the top tray, and run the camera calibration once. The camera-to-flange result serves every layer; the poses that give a stable calibration are their own subject, covered in eye in hand calibration pose selection.
3. Jog to the first snapshot height — 460 mm above the top tray in the example below — and teach it as the position of node CL_1.
4. Remove one tray, lay the grid on the new top surface, jog down by one thickness, and teach CL_2. Repeat to CL_N. What fixes each position is the plane of the grid, not the arm posture you jogged through: match the snapshot position to the work plane, not the robot pose.
5. Name the nodes CL_1 to CL_N and store the tray thickness as a variable, so a tray-type change is a jog and a confirm per layer, not a rebuild.
6. Program the counter loop and the tray-removed input, then pick from the returned positions using variable pick approach offsets for clearance above each part.
7. Confirm layer by layer with parts present: the located pick points must repeat within 1 mm over ten captures.
8. Run the full stack once with trays removed by hand in reduced-speed manual mode before the first automatic cycle.

Worked example: six trays of 40 mm parts

The stack: six trays, each 40 mm thick, each holding 40 mm parts in a 400 × 300 mm footprint. The photo must cover the tray plus 20 mm of margin, so the target field of view is 420 × 320 mm, taken from 460 mm above whatever tray is currently on top.

Take a 1-inch, 20 MP sensor: 5472 × 3648 pixels of 2.4 µm, a chip 13.1 × 8.7 mm across. Magnification is m = 13.1 / 420 = 0.031, so f = 460 × 0.031 / 1.031 = 13.9 mm. Fit the next stock focal length down, 12 mm, and the field opens to 13.1 / (12 / 448) = 489 × 325 mm — the tray sits inside with margin on both axes.

Resolution: 489 mm across 5472 pixels is 0.089 mm per pixel. A 40 mm part spans 448 px, and a ±1 mm pick tolerance is ±11 px of it. Hold 0.15 mm per pixel as the coarsest you accept for picking work.

Focus and aperture: at m = 0.027, f/4 and a circle of confusion of two pixels (4.8 µm), the depth of field is 2 × 4 × 0.0048 × 1.027 / 0.027² ≈ 55 mm, so ±28 mm about each taught layer. Tray tilt and sag stay inside that; the next layer down does not, which is a second reason each layer gets its own taught height.

Exposure: the robot is stationary at the taught pose, so cycle time, not motion, sets the limit. Start at 10 ms with gain at minimum under a white LED ring light, and raise light output rather than exposure when the histogram peak sits below half range. Glossy trays want a dome or crossed polarisers instead of a bare ring, or the glare moves with the layer and every re-teach becomes guesswork.

The layer ladder — six positions, one tray thickness apart:

Total travel of the snapshot positions: (6 − 1) × 40 = 200 mm. Run your own tray size and height through the same arithmetic with the calculator.

Choosing the camera and lens

Two decisions set the wrist camera: the pixel size it delivers over the tray, fine enough that your pick tolerance spans several pixels, and the cable, which must survive millions of flex cycles at the wrist. Area scan cameras in the 12–20 MP class cover a 400 × 300 mm tray from under half a metre. The MV-CS200-10UM puts 20 MP on a 1-inch sensor behind a C-mount — the sensor class the worked example uses.

Prefer one cable standard across the cell? The same 20 MP sensor also comes as a GigE camera, so the locate processing keeps a single interface.

All 2 share: Sensor type CMOS, rolling shutter; Sensor model Sony® IMX183; Pixel size 2.4 μm × 2.4 μm; Sensor size 1"; Resolution 5472 × 3648; Dynamic range 65.5 dB; SNR 41.5 dB; Gain 0 dB to 24 dB. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

A smart camera moves the locate onto the device and hands the robot finished pick points, with no vision PC in the cell. The LS-SMC-0017 carries 8.8 MP on a 1-inch sensor with a C-mount and a global shutter. The hand-eye step still has to happen — smart camera hand-eye calibration when no wizard exists covers it without a vendor wizard.

Checks before the first automatic cycle

• Re-run every node after the last one is taught: each must return its grid origin to within 1 mm, because teaching later positions can nudge the camera bracket.
• Keep every taught pose clear of wrist singularities; if a position needs a flipped wrist, re-teach it from a different approach direction rather than fighting joint limits at runtime.
• Confirm the tray-removed input (photocell or HMI button) blocks the loop; a timer that assumes removal speed will drop the camera onto a tray that is still there.
• After any collision or camera knock, re-check the stack against one target using robot cell vision recalibration with one reference target before resuming.
• Show the layer counter on the HMI so the operator knows which tray to pull; the counter, not the vision result, is the record of stack height.

When one snapshot position is enough

Teach per layer unless the uncorrected height change is negligible. The test is the parallax rule from the first section: r · Δz / WD must stay under a third of your pick tolerance. At 200 mm radius and 460 mm height, a ±1 mm tolerance allows Δz ≈ 0.8 mm — thinner than any real tray. So: single-layer trays, shim sheets or film stacks get one position with picks computed on its plane; any stack of trays 10 mm or thicker gets one taught position per layer.

Two cases change the answer. A height-aware 3D snapshot camera returns each part's Z with the image, so one pose serves the whole stack and the layer counter only sequences the picking. A fixed camera above the stack cannot be re-taught per layer at all: it needs the stack height known in advance, or the parallax arithmetic will move its edge picks by millimetres per layer.

From here

Size the wrist camera against your own tray in the camera finder, or lay out the whole cell — camera, lens, light, controller — in the system configurator. More robot-side vision guidance sits on the robotics topic page, and for stacks that need height per part, tell us the tray and part sizes and we will size a 3D snapshot set-up.