Vision-guided flexible feeding: architecture for random pick

Flexible feeding replaces a tooled bowl for every part with one vibrating tray, a camera above it, and a robot that picks whatever the camera finds. The imaging itself is rarely the hard part. What decides whether the cell runs is the loop around the camera: image the settled tray, send one pose to the robot, pick, re-image. This guide gives that architecture end to end: the timing budget, the handshake signals, the camera and lens arithmetic for the tray, and the checks to make before ordering hardware. It applies equally to a smart camera working alone and to an area scan camera driven by a vision controller.

The loop that sets every number

A flex feeding cell repeats one sequence: vibrate the tray to redistribute parts, let it settle, image the pick window from a fixed top-down camera, compute poses, hand one pose to the robot, pick, then either pick again from the same frame or restart.

The number that governs the design is image-to-pose time: from trigger to the robot controller holding a valid pose. Budget 200 ms for it. The basis is mechanical. Poses are trustworthy only while the tray is undisturbed; every feeder run, gripper contact or dropped part ends that window. Vision that answers in 200 ms leaves the robot picking for most of the cycle. Vision that answers in 600 ms makes the robot wait and starves the takt.

At 30 parts per minute the takt is 2.0 s. A realistic split:

The vision chain is 80-160 ms of that, inside budget. When processing regularly overruns acquisition, fix the pipeline first: acquisition faster than processing: fix the frame rate gap covers the usual causes.

The handshake: four signals and one rule

The cell lives on four signals between feeder, vision and robot:

1. TRIGGER - the robot or PLC requests an image, or the vision system triggers itself when the settle timer expires.
2. POSE_READY - vision returns one pickable candidate: x, y, angle, part type, score, and the image number it came from.
3. PICK_DONE - the robot confirms the grip; vision marks that pose consumed.
4. FEEDER_RUN - any feeder pulse invalidates the whole pose cache before the next image exists.

The single rule that prevents empty-gripper picks: never serve a pose computed from an image older than the last feeder event. Version every pose with its image number and make the robot echo it back; a mismatch means re-image, not pick.

A one-line TCP frame carries it: img=1024 x=212.4 y=98.1 th=41.5 part=brk score=0.93. Fixed field order, newline-terminated, trivial to parse in any robot language. Socket handling, heartbeats and reconnect logic are covered in send camera part coordinates to a robot controller over TCP; where the cell standard is a PLC, vision controller Modbus communication: a PLC integration guide does the same job with registers. The robot-side program that receives these frames has a worked pattern in configuring a robot controller for vision-guided pick cycles.

One startup trap: a smart camera that starts answering requests before its inspection tool is loaded drops the first frames. Gate it as described in smart camera misses first images after load: fix the ready gate.

Camera position and the numbers that size it

Mount the camera top-down over the pick window on its own rigid stand, never on the feeder frame: the frame shakes with the tray and the blur lands in every image, permanently.

Three numbers size the optics:

• Field of view: the pick window plus 5-10 % margin per side, so a part at the edge is still fully imaged.
• Pixel scale: the smallest locating feature at least 3 pixels across. Two pixels detect a feature, three locate it, five measure it.
• Working distance: whatever the gantry allows. The lens follows from the other two.

The third line is the blur budget. At 0.1 mm per pixel and 20 mm/s of residual tray motion, exposure must stay under 5 ms; a 1 ms strobe tolerates 100 mm/s.

Check the pixel count for your own tray with the calculator below; its button filters the shop finder to cameras with enough pixels.

Worked example: 400 × 300 mm tray at 30 parts per minute

Inputs, each from a real source:

• Pick window 400 × 300 mm, from the feeder specification.
• Smallest part 12 mm across with a 0.3 mm locating notch, from the part drawing.
• Takt 2.0 s per part, from the line rate.
• Mounting height limited to 700 mm, from the gantry clearance.

Rule: three pixels across the notch sets the pixel scale at 0.3 / 3 = 0.1 mm per pixel.

The arithmetic:

1. Field of view with 5 % margin: 400 × 1.05 = 420 mm wide, 300 × 1.05 = 315 mm high.
2. Pixels needed: 420 / 0.1 = 4200 wide. A 4000-pixel-wide sensor gives 420 / 4000 = 0.105 mm per pixel, and the notch spans 0.3 / 0.105 = 2.9, call it 3 pixels. Passes.
3. A 12 MP sensor at 4000 × 3000 on a 1/1.7-inch format is about 7.5 mm wide, so m = 7.5 / 420 = 0.0179.
4. Lens: f = 700 × 0.0179 / (1 + 0.0179) = 12.3 mm, so a 12 mm lens. Check it: m = 12 / (700 - 12) = 0.0174, field of view = 7.5 / 0.0174 = 431 × 323 mm, covering the window with margin.
5. Exposure: 1 ms strobe. Blur at 20 mm/s of residual motion is 20 × 0.001 = 0.02 mm, under a fifth of a pixel.
6. Transfer: 4000 × 3000 × 1 byte = 12 MB. USB3 at roughly 300 MB/s effective moves it in about 40 ms; GigE at roughly 100 MB/s takes about 120 ms.

Budget check: 1 + 40 + 60 + 2 + 30 = 133 ms from strobe to robot motion, inside the 200 ms budget with margin. The same sensor on GigE lands at 213 ms and fails it.

Decision: a 12 MP USB3 camera, 12 mm lens at 700 mm working distance, 1 ms strobe, poses cached per image and invalidated on every feeder run. Two cameras that cover this window:

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

The LS-CMR-0166 outputs 4000 × 3000 frames at 30 fps over USB3 from a 1/1.7-inch rolling-shutter sensor. A rolling shutter is acceptable here because the tray is stationary when the strobe fires. The MV-CH120-11UC reaches 12 MP with a global shutter, a 1.1-inch sensor and a C-mount on USB3; its larger sensor widens the window for the same lens and working distance.

Smart camera or vision controller

Pick the architecture by part mix and camera count, not by preference:

For one tray, smart cameras remove the PC from the cell. The MV-SC6050C-08M-XBN is a 5 MP color smart camera with a global shutter, a GigE interface and an 8 mm M12-mount lens. Its multi-color light source lights and images the tray from one housing. Rules versus trained models is its own decision; see AI vs rule-based smart camera inspection: how to choose.

When a second camera, AI models or image archiving enter the cell, move the processing into one vision controller and feed it from the area scan camera finder. The MV-VC3301H-128G66 carries an Intel Core i3 processor with 8 GB of memory, 5+ LAN ports, four built-in light channels and a VM6200Pro license, so one box holds the cameras, the strobes and the robot socket. The MV-CU020-90GC is a 2.5 MP camera with a global shutter, a 1/1.7-inch sensor and a C-mount on GigE. A 2.5 MP frame at the same 0.105 mm pixel scale covers about 160 mm square, so it belongs on small trays only.

Lighting: flat, diffuse and strobed

Light the tray with a flat, diffuse field from above so parts in random orientation give a stable silhouette and stable surface features. Shiny stamped or drawn parts want a dome; machined faces with specular glare want cross-polarization. Backlighting supports silhouette-only pose and gives up part identity, so use it only when every part on the tray is the same.

Drive the light as a strobe, synchronized to the trigger, from the controller's light channels or the smart camera's illumination. A 1 ms pulse freezes 20 mm/s of residual motion to 0.02 mm, a fifth of a pixel. The strobe does not shorten the settle wait, because parts must be stationary for their poses to mean anything; it removes blur as a constraint and protects the first frames after settle ends.

Programming the robot side of the handshake

Structure the robot program around fixed drop targets and let vision supply the only variable, the pick. Structure a vision-guided pick and place program around drop targets gives the layout. Approach heights and per-part grip offsets belong in that program rather than recomputed per image, as variable pick approach offsets for vision-guided picking shows. Calibrate pixel coordinates into the robot frame once at commissioning; the calibration sequence for a planar set-up is in 2D vision pick and place: selecting a planar guidance system.

Put one piece of intelligence in the vision step: the gripper envelope. A candidate is pickable only when no neighboring part lies inside the gripper width plus 5 mm. Blocked candidates are marked, not deleted; they often free up after the next neighbor is picked.

When a 2D tray camera is the wrong answer

• Parts stack or overlap after feeding. A top-down 2D image cannot give a safe grip height. Move to a 3D sensor; the decision logic in 2D or 3D camera for conveyor pick and place: choosing the system transfers directly.
• Parts tangle: springs, wires, spiral shavings. No camera fixes the feeding step; read up on picking flexible wires from bundles with vision before specifying hardware.
• Takt under about 1 s per part. Settle, image and handshake cannot fit; move imaging off the pick path with conveyor tracking and a continuously imaging camera.
• Flat shiny parts wash out. A saturated highlight carries no pose information at any resolution; fix the lighting first.

Checks before you order

• Field of view equals the pick window plus 5-10 % per side, confirmed with the lens formula at your real working distance.
• Pixel scale no coarser than the smallest locating feature divided by 3.
• Global shutter, or settle plus strobe with computed blur under half a pixel.
• Frame transfer under 50 ms: frame size divided by effective bandwidth, about 300 MB/s on USB3, 100 MB/s on GigE, 1000 MB/s on 10GigE.
• Strobe channels on the controller, or illumination built into the smart camera.
• TCP or Modbus proven with a loopback test on the robot controller before the camera arrives.
• Camera on its own stand; jitter between two images of the settled tray under 1 pixel.
• Gripper envelope plus 5 mm clearance rule running in the pose filter.

To review a specific cell with a takt target, part mix and feeder model, bring the numbers to the contact page. More cell architecture write-ups sit in the systems topic.