Structure a vision-guided pick and place program around drop targets

Four drop targets, four wait loops, four near-identical pick routines: this is the shape most first vision-guided cells grow into, and it holds until the fifth target appears. The fix is architectural. Your vision sensor already answers the only question that differs between the loops — which part did you find? Read that answer once, look up where that part goes, and run a single pick-and-place routine whose only variable is the row you looked up.

This how-to collapses a per-target loop program into one state machine driven by a target table: the result record both sides agree on, where the logic should live, the timeout arithmetic, and the pre-run checks. You need a vision sensor or smart camera that outputs a part ID with a position, and a robot program you can edit. If the guidance approach itself is still open, start with 2D vision pick and place: selecting a planar guidance system.

The decision: one wait loop and a target table

Your four loops differ in exactly one line: the drop waypoint. Everything else — the wait, the approach, the grip, the retreat — is the same code pasted four times. So make the waypoint data, not code. One wait loop reads the sensor: a valid flag, a part ID, a position. The ID indexes a table of drop poses, and one pick-and-place routine takes its pose, release height and gripper rotation from that row.

Collapse to this structure the day the second drop target appears. A fifth target then costs one table row and one taught pose instead of a fifth copy of the loop. Miss handling exists exactly once, and an empty window can never fall through into the next loop with a stale located flag — the failure your current version avoids only by accident.

Collapse the loops in seven steps

1. Agree the result record first: valid flag, part ID, X, Y, angle, and a sequence counter that increments with every new result. Fix the units now — millimeters or pixels, degrees or radians — and convert in exactly one place.
2. Declare the target table beside the robot program: one row per part ID, columns for drop pose, approach height, release height, gripper rotation and speed percentage.
3. Replace the four loops with a single wait that exits only when the flag is valid and the ID falls inside the table range.
4. Treat an out-of-range ID as a miss: log it and keep waiting. Never fall back to the nearest known pose.
5. Rewrite the drop as one parameterized routine that consumes the table row; variable pick approach offsets for vision-guided picking keeps the approach vector out of hard-coded waypoints.
6. Add the miss path: on timeout, log a no-part entry and stay waiting. Define the alarm — in the worked example below, a part present at the infeed with no valid result for 1 s.
7. Test at reduced speed override in this order: every part type, then an empty window, then a stale result.

Where the state machine runs

WAIT, PICK, PLACE, back to WAIT — the logic is small, and it has three legitimate homes. The choice follows what the cell already contains, not the program shape.

For a single sensor and a single robot, a smart camera keeps the cell at two devices:

The LS-SMC-0063 is a 1.6 MP smart camera with a 16 mm lens, polarized white light and a 60 fps rate. One frame every 16.7 ms means result freshness never bottlenecks the cell — the robot cycle does.

When strobed lights, several cameras or recipes join the cell, a vision controller is the natural home: it sequences the lights, collects every result and hands the robot one line per cycle:

The MV-VC3301H-128G66 pairs a Core i3 with 8 GB of RAM, drives 4 built-in light channels and includes the VM6200Pro license, so the sequencing software runs where the images arrive.

One result record, read once

Whatever carries the logic, the sensor writes one record and the robot reads one record — anything else invites torn data, half old and half new. The sequence counter is what makes this safe: read the counter, read the fields, read the counter again. If it moved, a new result overwrote the old one mid-read, so read again. At this frame rate you lose a millisecond, never a cycle.

Poll or push: below roughly 10 updates per second, let the robot poll on a fixed period; above it, have the sensor push and use the counter to drop late arrivals. Either works — pick one and write it into the record definition. The camera-side half of the same discipline is in frame timeout: one wait call per frame, and asynchronous vision result frame matching with timestamps covers pairing a result with the physical part that produced it.

Serial links follow the same rule — one command code per action, as RS232 vision recipe selection shows. If the robot controller side needs attention, communication set-up for vision-guided pick cycles is a configuration task, not a logic change.

A basic vision sensor is fully equal to this job. The HK-TD032-12-WBN is a 1.6 MP mono vision sensor on Fast Ethernet with a global shutter; set it to send ID, X and Y in one telegram and the state machine never asks what produced them.

The target table

Every row is taught once with the pendant and stored as data:

Keep IDs contiguous from zero, so an ID outside the table is one range check. The rotation column earns its place the first week: a wrist that turns 270° where the table says -90° still hits the coordinates and still puts the part down wrong — gripper rotation between pick and place: find the full turn is the diagnostic. The X and Y in the result are robot coordinates only because someone taught the mapping; robot hand-eye calibration: how nine-point alignment works is that step.

The table is only as stable as the sensor mount. Fix the sensor relative to the robot base — an eye-to-hand mount — and neither the mapping nor the table drifts; eye-in-hand vs eye-to-hand camera mounting for robots lays out when the camera-on-wrist alternative wins. The LS-MNT-0040 is a slide-table stand, 700 mm high on a 400 x 450 mm base, with X/Y travel to center a pick window without redrilling the cell.

Worked example: four drop targets, a 60 fps sensor, a 5.8 s cycle

Inputs, with the source of each: four part types and four nests, from the cell layout. A sensor running at 60 fps, from the smart camera above. A robot poll period of 25 ms, your controller's scan time. A 320 mm pick window, from the fixture. Move times from the pendant: approach 1.2 s, grip 0.5 s, retreat 0.8 s, transfer to nest 1.6 s, release 0.4 s, return 1.3 s.

Latency: 1 / 60 = 16.7 ms per frame. Two frames from exposure to output plus the 25 ms poll gives 2 × 16.7 + 25 = 58.4 ms — call it 60 ms normal. Timeout = 10 × 60 ms = 600 ms: after 0.6 s without a valid record the robot logs a no-part entry and keeps waiting. At 60 fps that is 36 skipped frames, so one blurred image cannot stop the cell.

Position: 1440 pixels across the window (the horizontal count of a 1.6 MP sensor) over a 320 mm pick area gives 320 / 1440 = 0.22 mm per pixel. Allow ±2 pixels of detector jitter and vision contributes ±0.45 mm against a ±2.5 mm nest clearance — under a fifth of the tolerance, leaving the rest for robot repeatability and part variation. A ±0.5 mm clearance would instead demand about 0.10 mm per pixel, roughly 3200 pixels across the window; the calculator below converts that requirement into camera pixels in one step.

Throughput: 1.2 + 0.5 + 0.8 + 1.6 + 0.4 + 1.3 = 5.8 s per part, so 60 / 5.8 = 10.3 parts per minute. Decision: feed this cell up to 8 parts per minute — a 22% margin. At 12 parts per minute, overlap the return move with the next wait or bring the nests closer; the structure does not change, only the numbers in it.

The whole program, for reference, is shorter than the loop version it replaces:

state = WAIT
loop:
    if state = WAIT:
        r = read_result()            # valid, id, x, y, angle, counter
        if r.valid and 0 <= r.id <= 3: target = TABLE[r.id]; state = PICK
        elif r.valid: log('unknown id')
        elif waited > 0.6 s: log('no part')
    if state = PICK:
        move_above(r.x, r.y); descend; grip; ascend; state = PLACE
    if state = PLACE:
        move_above(target); descend_to(target.release); open; ascend; state = WAIT

Verify before the first part

• Place each part type in turn: the robot must visit its own nest, 20 consecutive picks per type, all inside clearance.
• Empty the window: a no-part entry logs at most every 0.6 s and the robot does not move — the behavior your four loops gave you by accident, now guaranteed.
• Unplug the sensor cable mid-wait: alarm inside the timeout, no motion toward any pose; reconnect and the cell resumes without a restart.
• Present two parts back to back: the counter check rejects the torn read, and the log counts it.
• Load the second recipe: the table row set must follow the recipe ID before the first trigger fires.

Run the first 50 cycles at 20% speed override. The state machine does not care about speed; your fixtures do.

When one loop is the wrong answer

The lookup assumes each ID maps to one destination and any order will do. Break that deliberately, not by accident:

• Same part, fixed build order, such as a kit: keep the table, add a sequence index that steps after each place. Order becomes data; the machine is unchanged.
• Several parts in one image: still one loop — add a sort rule, nearest first, and pick the head of the list.
• Unknown part height, parts in a bin: a 2D pose is not enough. 3D bin picking for flat parts: choosing a vision approach is that fork in the road.
• Parts on a moving conveyor: the drop table still holds, but the pick side needs tracking; whether a delta pick-and-place cell needs conveyor vision covers when that layer is worth adding.
• Destinations that move every cycle, such as pallet layers: compute the pose from the layer model. The table becomes a function and the routine never notices.

Next steps

Rewrite the program first and buy hardware second: the structure runs on any sensor that reports an ID and a position. When the cell grows, the smart cameras, vision controllers and camera stands finders list the parts that fit each role, and the system configurator walks camera, lens, light and controller together. For a specific part-and-robot pairing, the contact page puts the numbers that matter — window width, parts per minute, part IDs — in front of the applications team. More integration write-ups sit on the systems topic page.