3D vision for robot palletizing: sizing boxes for pick

A palletizing cell that handles mixed or randomly placed boxes needs three numbers for every pick: where the box is, how it is rotated, and how big it is. A 3D vision palletizing system delivers all three in one acquisition, then hands them to the robot controller as a pick target. This guide is for integrators and controls engineers planning such a cell. You will choose between snapshot 3D sensors and PC-based processing, size the camera geometry from the pallet and cycle time, run a worked example, and leave with a checklist for the robot handoff.

What the cell must output

Start from the robot program, not the camera. Most palletizing robots need, per box:

• Pick point X, Y, Z in a shared coordinate frame (robot base, conveyor frame, or pallet frame).
• Yaw rotation about the vertical axis, so the gripper can square up to the box.
• Box dimensions, if the layer pattern is generated from measured sizes rather than a fixed SKU table.
• A quality flag, so the robot can reject or divert an unreadable box instead of picking thin air.

Decide the frame early. Every millimetre of error you accept in the camera-to-robot calibration adds directly to your grip tolerance. A reference target at a known robot position, re-checked after any mechanical change, is the cheapest insurance in the cell. The workflow behind converting image results into robot-frame coordinates is the same one described in bounding box depth: mapping image pixels to 3D points, applied at pallet scale.

Two architectures: onboard sensor or vision controller

There are two workable architectures, and the right one follows from your cycle time and camera count.

Snapshot 3D sensor with onboard measurement. A structured-light or ToF sensor captures one depth image per trigger, finds the topmost box, and reports dimensions and pose over industrial Ethernet. One camera, one cable, no separate compute. This covers most single-station palletizing cells with cycle times of a few seconds per box.

PC-based processing with a vision controller. When you run two or more cameras, custom detection on irregular stacks, a SKU database, or buffering while the robot finishes a pick, move processing to a controller. A machine such as the MV-VC5525-256G70-NN pairs a multi-core CPU with multiple LAN ports, so cameras and the robot link live on separate networks — the segmentation logic is the same as isolating a vision cell network from a busy plant LAN.

For a second camera added for coverage, the sizing math in embedded stereo vision system sizing for two cameras applies directly.

Camera geometry: coverage and standoff

Two constraints set the mounting height: the camera must see the whole search area, and the depth error at that standoff must stay inside your grip tolerance.

For coverage, a rough estimate for a sensor with a horizontal field of view covering width W at height H is:

The depth error of a structured-light sensor grows roughly with the square of the standoff distance, so doubling the mount height costs you about four times the depth noise. Mount as low as the robot's reach envelope and safety scanning allow.

Worked example. A pallet layer is 1.2 m by 1.0 m and boxes arrive anywhere on the pallet. You budget a 15 % margin, so the camera must cover about 1.4 m by 1.2 m. If the sensor's diagonal field of view allows full coverage at 2.5 m standoff, check the corners: the diagonal of the search area is √(1.4² + 1.2²) ≈ 1.84 m, so the useful depth image must stay valid at that diagonal, not just across the centre. Many depth sensors lose coverage at the image edges — this is the same effect described in depth image field of view: why stereo coverage is smaller. If the corners are noisy, either raise the sensor and accept more depth noise, or shrink the search area and add a pre-palletizing control point.

Choosing the sensor type

Match the sensor to the boxes, not the other way round.

• Structured light, speckle or projected pattern. Best general choice for corrugated boxes: dense depth on matte, low-texture surfaces, good height measurement from a single snapshot. The speckle structured-light volume-measurement camera is intended for volume measurement, while the MV-DPS200P-02 is a projected structured-light 3D camera. Verify depth completeness on your actual box surfaces before choosing either approach.

• ToF. Faster acquisition and a wider scene tolerance, but lower depth resolution; fine for locating boxes, weaker for precise dimensioning of small cases.
• Laser profiling, scanned or line-scan. When boxes pass under a fixed arch on a conveyor and you need dimensioning in motion, a profiler or 2.5D line-scan system builds the height map line by line. The MV-HLVS-16K100-L60 is a line-scan 2.5D system with a 16K resolution sensor and a 600 mm emitting area length, suited to overhead dimensioning of wide conveyor lanes.

Reading a single depth value at a known pixel — useful for verifying height at a reference point — is covered in center pixel depth measurement: read a fixed image point.

Sizing the compute for the cell

If you go PC-based, size for the peak rate, not the average. Estimate the image-processing budget as:

At a 6 s cycle with a 0.5 factor, vision has 3 s to trigger, acquire, detect and publish. A single snapshot detection runs in a few hundred milliseconds on a modern CPU; multi-camera fusion, SKU matching and logging consume the rest. Controllers in the vision controllers range from compact N-series models for one camera to Core i5 and i7 machines with PoE camera ports for multi-camera cells — pick by LAN port count first, processor second.

The robot handoff

The interface, not the vision algorithm, is where palletizing integrations usually fail. Get these right:

1. One direction of truth. The robot requests or waits; the vision system publishes one result per trigger with a sequence ID. Duplicates and stale results cause the classic double-pick.
2. Handshake before motion. The robot acknowledges receipt before starting the approach, so a lost message stops the cell instead of picking at a stale position.
3. Define failure. Agree what the robot does on a quality flag of zero: divert, skip, or fault. An undefined failure path is how a blocked box stops the whole line.
4. Timestamp the trigger, not the result. If the box moves between trigger and pick, track the conveyor or freeze the section; an integration walkthrough of case palletizing with an integrated robot vision option shows one full handshake sequence in practice (case palletizing vision integration walkthrough).

If the vision or robot hardware changes later, the recovery procedure in vision calibration recovery after camera or robot changes tells you what must be re-verified before restarting.

Checks before you order

• Confirm the search area dimensions and the worst-case box size, including lean or overhanging flaps.
• Compute required standoff from the coverage formula and check corner depth quality against the sensor's working range.
• Time one full trigger-to-result cycle on the supplier's data, then double it for your budget.
• Count cameras, robot links and any plant-network connection, and choose LAN port counts accordingly.
• Agree the result message format and failure behaviour with the robot programmer before wiring.

When the usual answer is wrong

A single overhead snapshot camera is the standard answer, and it fails in three cases. Random stacks — boxes dumped into a bin or half-collapsed layers — need full 6D pose estimation and often two viewpoints; a snapshot dimensioning sensor gives you size, not a graspable pose.

Clear or dark stretch wrap on top of boxes can drop out of structured light entirely; a change of wavelength or a dark-field illumination test tells you quickly. And if the robot must build layers from measured dimensions, dimension accuracy, not location accuracy, becomes the binding specification — verify the sensor's dimensioning accuracy on your worst box, not on the vendor's test target.

For those cases, look at laser profilers or multi-camera stereo instead of a single snapshot sensor; the 3D sensors finder filters by type, and the camera finder narrows by resolution and interface. When you are ready to specify the compute side, contact the applications team with your camera count, PoE requirements and processing workload.

Quick checklist

• Search area and margin defined in millimetres.
• Standoff computed from coverage; corner depth verified, not assumed.
• Dimension accuracy tested on the worst real box.
• Result message: position, yaw, dimensions, quality flag, sequence ID.
• Robot acknowledgement before motion; defined failure path.
• Camera-to-robot calibration re-checked after any mechanical change.

Work through the list in order and the palletizing cell starts up with the vision system as a data source the robot trusts — which is the whole point.