Vision picking parts stacked in trays inside a cardboard box

A robot-tending cell often receives parts the way suppliers ship them: seated in trays, trays stacked inside a cardboard box, the whole thing opened only at the machine. This guide is for the integrator who has to empty that box automatically with a 2D camera. It gives the structure that works — one snapshot position and calibration per tray layer — the scale-error arithmetic that shows when a single calibration would put the gripper millimetres off, the standoff and field-of-view numbers for a 400 mm box, lighting that survives the box walls, and the checks to run before ordering hardware.

Can a 2D camera pick parts from trays stacked in a cardboard box?

Yes — a 2D camera guides this pick layer by layer: it returns X, Y and rotation for each part on the current tray, and it stays accurate to about ±1 mm only when every tray height has its own snapshot position and calibration.

A 2D locate works on one plane at a time. The Z comes from the robot: you teach each layer's height once, and the gripper drives to that Z while the camera fixes the in-plane position. What the camera cannot do is see through the stack — the layers below are hidden — so the cell removes each empty tray to expose the next one. Locate, pick, clear, remove, locate again: that rhythm is the whole application.

The pattern matches tray destacking with one snapshot position per layer, applied to a box that arrives sealed. If the parts are large and the tolerance generous — a 200 mm housing picked to ±5 mm, say — a single snapshot position can survive small height differences. Work the arithmetic in the next section before committing to that shortcut.

Why does one snapshot position put the gripper in the wrong place on lower layers?

Because a 2D calibration is only true on its teaching plane: a part deeper by Δh projects smaller pixels, so the pick lands short by r × Δh / (H + Δh) — 28 mm at the far corner of the worked example below.

The calibration converts pixels to millimetres with a scale fixed at the teaching distance H. A part on a plane Δh deeper sits farther from the lens, so its image shrinks by H / (H + Δh). Converting that smaller image back at the old scale returns a position too close to the camera axis:

H is the standoff from lens to the calibrated plane, Δh the depth of the part below it, r the part's true distance from the camera axis. The error grows in proportion to r: a part on the axis never moves, a part at the field edge moves most.

Worked example. The camera hangs 600 mm above the top tray and is calibrated there. Four layers at 25 mm tray pitch put the bottom layer's parts Δh = 75 mm below the calibration plane. The box interior is 400 × 300 mm, so a part in the far corner sits r = √(200² + 150²) = 250 mm from the axis. Error = 250 × 75 / 675 = 27.8, call it 28 mm — 70 % of a 40 mm part's width, an automatic miss.

The same arithmetic in inches: 1 inch of unmodeled height on a 24 inch standoff shifts the pick by 1/24 = 4.2 % of the off-axis distance, so a part 8 inch off-axis lands about 0.33 inch short.

Two escapes exist. Teach one snapshot position per layer, as the next section shows, or keep one calibration and multiply the located offset from the image centre by (H + Δh)/H — the approach behind scale compensation for part height variation. The scaling shortcut only holds while the tray pitch is exactly known and every tray is present; a missing tray changes Δh and the scale quietly lies, while a per-layer snapshot fails loudly and stops the cell.

What standoff and field of view does the box need?

Size the field to the box interior at the deepest layer, then set the standoff from the lens: for a 400 mm wide box, a 6 MP 1/2.4-inch sensor and an 8 mm lens, a 650 mm standoff gives a 460 mm field at 0.15 mm per pixel.

Take a typical 6 MP, 1/2.4-inch sensor — about 5.7 mm wide, 3072 × 2048 pixels — with an 8 mm M12 lens. To cover a 400 mm interior with 30 mm of margin per side you need a 460 mm field, so WD = 460 × 8 / 5.7 = 645 mm; call it 650 mm. Pixel scale at that field: 3072 / 460 = 6.7 pixels per millimetre, or 0.15 mm per pixel. A 30 mm part then spans 200 pixels, far above the 15-25 pixels a locate wants across its smallest feature.

Any camera near these numbers works. Mono is enough because locating runs on edges, not colour, and a static snapshot tolerates a rolling shutter since nothing moves during exposure. Prefer global shutter only if the stand shakes when the robot passes. One camera matches the arithmetic:

The MV-SC5060XM-08M-WBN is a 6 MP mono GigE smart camera with an 8 mm M12-mount lens over a 1/2.4-inch rolling-shutter sensor, and it runs the locate on the device. The smart cameras finder filters the rest by resolution, mount and shutter.

How do I set up one snapshot position per layer?

Teach the box once: measure the tray pitch, mount the camera so the field covers the deepest layer, then store one snapshot position with its own calibration per layer and verify with a part at the far corner — a 1-2 mm residual there means the cell is ready.

1. Measure the stack once: tray pitch, layer count, and the smallest tray's interior. For the example box that is 25 mm pitch, four layers, 400 × 300 mm.
2. Mount the camera over the box centre at the standoff from the last section — 650 mm above the top layer here — on a stand rigid enough that a passing robot cannot tilt it.
3. Calibrate the top plane with a target or a tray of known pocket spacing, and store the result with snapshot position 1. The short-range hand-eye calibration target and standoff guide covers target choice at these distances.
4. Give every layer the same treatment: after the robot removes the empty tray, calibrate the newly exposed plane — or apply the scale factor from the previous section — and store it as the next snapshot position. Each position pairs one plane with one taught robot Z.
5. Trigger still snapshots, not a live stream: the robot requests, the exposure runs while nothing moves, and a finished pose comes back. If locates pass in teach but drift in production, the stream-versus-snapshot fix is the usual culprit.
6. Verify at the worst point: put a part at the far corner of the deepest layer, run the locate, and command the robot to hover 50 mm above it. Measure the offset by touching off; 1-2 mm is a passing cell. If the error grows with part rotation, follow diagnosing position drift under rotation, and teach the model so it survives the rotation the pockets allow.

Is lighting inside a cardboard box a problem?

Usually no — corrugated board and most tray faces are matte and diffuse, so a single bar light mounted beside the camera illuminates every layer; the real risks are shadows across deep layers and glare on glossy plastic trays.

The box is not a dark tunnel when the camera looks straight down into it: the image shows tray floor and parts, both matte. Shadow control matters more than direction. At a 650 mm standoff with walls 200 mm from the axis, light entering steeply and close to the lens is what reaches the bottom layer; a bar along the near rim throws grazing light that the walls cut up across the deep layers. Mount the bar tight under the camera, and if the camera's own shadow band crosses the field, split into two bars on opposite sides.

Lock the exposure so every layer images at the same brightness; deeper layers lose some light to wall shadowing, and the locate threshold should not have to chase it. Start near 10 ms with a steady 24 V bar, adjust until the part edges sit mid-grey, and keep the drive constant through the box.

Glossy polystyrene trays are the one real glare risk. If highlights blind the locate, cross-polarize: polarizing film on the bar, the analyzer rotated 90° on the lens, and 3-4× the exposure budgeted for the filter loss.

Bars sized to box interiors:

The LS-BAR-1276 is a 372 x 28 mm white diffused bar light on 24 V. Across a 400 mm interior it covers the long axis from one side, and the diffuser softens the shadow the camera casts. The LS-BAR-1047 is a 290 x 30 mm standard bar light on 24 V. It suits boxes near 300 mm on the lit axis. Match lit length to the interior, not the outer carton.

How does the cycle handle removing empty trays without collisions?

The locate only places the gripper on the part — it does not clear anything, so the program must swing in above the box rim, enter vertically, and lift straight out: keep 50-100 mm of approach clearance above every part and above the tray you pull out.

A workable cycle for the example box: locate on snapshot position 1, pick all parts on the top layer, grip the tray by its edge recesses, lift it out above the rim, and set it on a tray stack or conveyor. The cell drops to snapshot position 2 and repeats until the box is empty, then signals for the next one.

The trigger matters more than it looks: a discrete output from the robot or PLC that fires the exposure removes the timing variation a software snapshot carries. When the box itself is presented by sensor — a photo-eye seeing it in the fixture — the same handshake questions return, and the sensor-triggered robot box picking troubleshooting guide walks that side of the cell.

Collision planning is yours, not the vision system's. Keep the approach and retract vertical above the part, hold 50-100 mm of clearance above the rim on every entry and exit, and give the tool 10 mm beyond the worst wall bow, because corrugated boxes swell as humidity rises. Run the first cycles at reduced speed override and watch the tray-removal move in particular: the tray is the widest thing that leaves the box.

When is 2D over trays the wrong choice?

Stay with 2D while parts sit in known pockets on known planes; switch to 3D when parts arrive loose, tangled or face-down in the box, or when the pick needs the part's height — depth search handles poses a 2D locate cannot name.

The pockets are what make this a 2D job. Once parts are dumped in loose, a single planar locate cannot return a graspable pose, and per-layer snapshots buy nothing because there are no planes. That is bin picking, and the decision moves to depth: the 3D approaches for flat parts in bins and the practical limits of wrist-camera bin picking frame that choice, and the broader 2D or 3D comparison for pick and place lays out the full decision tree. A middle case exists: pockets hold two or three part types, which separate models and per-layer snapshot positions still handle in 2D — see picking a full tray per cycle with multiple object detection.

What do you check before ordering the hardware?

Five checks decide the order: field over the deepest layer, standoff within the stand's height, lit length over the box, snapshot accuracy at the far corner, and tray-removal clearance — walk them in that order and the box feeds the machine without a redesign.

• Field at the deepest layer covers the interior plus 25 mm of margin per side — the deepest layer sets it, not the top.
• Standoff fits the mount: a 650 mm lens-to-top-layer distance must clear the stand column above a tray sitting 50-100 mm above the bench.
• Lit length matches the interior long axis; two bars opposite if the camera's shadow crosses the field.
• Pixel scale at or below about 0.3 mm per pixel for parts 20 mm and up, so the locating feature keeps enough pixels.
• Verified offset at the far corner of the deepest layer within 1-2 mm before the cell ships to production.
• Tray-removal path lifts 50-100 mm above the rim, and the tool clears the walls by 10 mm through the whole descent.

A stand sized for the standoff:

The LS-MNT-0040 is a slide-table camera stand with a 700 mm column and a 400 x 450 mm base. A 650 mm lens-to-layer standoff leaves about 50 mm of column over a top tray sitting just above the bench, and the slide table squares the camera over the box centre without shimming. Finish the list in the camera stands category and the bar lights category.

Next step: put your own numbers into the two formula blocks — standoff, field, per-layer error — then build the station from the lighting configurator or the camera finder. Send the stack drawing with a quote request if you want the arithmetic checked against your tray pitch before you order.