Vision system and robot gripper control architecture: who drives what

A pick lands short and the first argument on the bench is who was supposed to command what. In a vision-guided cell the answer is a fixed control architecture: the vision system masters the robot controller, and the robot controller masters everything hung off the arm, gripper included. This page explains why that split exists, what each hop carries, how a 24 V DC Modbus RTU gripper attaches to the robot controller rather than the vision PC, and where a smart camera shortens the chain. If you are wiring a first vision-guided pick or chasing an intermittent demo, you will leave with the chain drawn correctly, a cycle-time budget you can check and a commissioning checklist.

Who commands the gripper in a vision-guided robot cell?

The robot controller commands the gripper, always: the vision system's authority ends at the robot controller, which runs the motion program, closes the gripper over its own 24 V DC or fieldbus link and only then reports the pick complete.

The chain has two masters, one per hop, and nothing crosses a hop except between that hop's master and slave:

vision system → robot controller → gripper

The vision system's link carries pick poses and status. The robot controller's link carries motion-adjacent commands: open, close, target position, grip confirm. Setup tools blur this on the bench, because a gripper's configuration utility talks to the tool directly while you commission, and it is tempting to leave that path in place. Do not: in production the open, close and confirm steps belong to the robot program, between the approach and the lift. The same drawing scales up to a cell with a PLC and queued stations; the cell-level version is laid out in Robot vision cell control architecture for palletizing.

What does master and slave mean between a vision system and a robot?

Master and slave is a one-way command model: after the handshake program loads, the vision system sends commands such as trigger and pick coordinates, and the robot executes them and answers with a done signal, never the reverse.

The roles are assigned once, when the handshake program loads on the robot side: from then on the vision system sends and the robot executes and answers. A minimal exchange is three messages — the pose string, an acknowledgement, and the done reply that closes the transaction. The pose crosses the wire already in robot coordinates; converting camera pixels to robot-base values is the calibration work covered in use eye-to-hand calibration results to compute robot-base picks. Delimiters, units and the quoting traps that bite ASCII handshakes are worked through in send string data from a vision system to a robot over TCP.

Slave does not mean dumb: the robot controller keeps running its own program, scanning its own I/O and enforcing its own safety logic while it waits. It accepts a command and then owns everything that command sets in motion — the gripper included.

Why can't the vision software drive a 24 V Modbus RTU gripper directly?

Because the gripper is not on the vision system's side of the control chain: a 24 V DC gripper speaking Modbus RTU over RS-485 hangs off the robot controller's own serial port, and grip timing must sit inside the robot program where safeguarded stops act on it.

Three reasons, in the order they bite:

1. Wrong port, wrong protocol. A vision controller speaks Ethernet and USB; an RS-485 gripper needs a serial port and an RTU master stack, which a general-purpose PC does not bring.
2. One master per serial bus. Modbus RTU is itself a master/slave scheme: the robot controller — or a serial gateway under its control — must be the only RTU master, or two devices drive the pair at once and frames collide. The classic RS-485 figures bound the segment: 32 unit loads of devices, about 1,200 m of cable at low bit rates, 120 Ω termination at each end.
3. Timing and safety live in the motion program. Grip must interlock with axis position and speed. A second master splits ownership of one physical event across two devices, and only one of them stops on an e-stop.

If a second device genuinely needs gripper data, let the robot controller publish it upward over Ethernet: one master per bus, many listeners per cell.

How does the robot controller talk to the gripper?

Over one of three links: Modbus RTU over RS-485 at 9,600 or 19,200 bit/s into the controller's serial port, Modbus TCP over an Ethernet port, or 24 V discrete open, close and done wires.

Modbus RTU over RS-485 is the common link for electric adaptive grippers. The gripper is an RTU slave at an address set in the tool — address 1 in this example — and holds registers for target position, speed, force and an object-detection or fault status word. The RTU character frame is 11 bits per byte (start bit, 8 data bits, then parity or a second stop bit), so wire time is predictable before you power anything:

Serial settings commonly default to 9,600 or 19,200 bit/s; agree one value on both ends before the first poll.

Modbus TCP wraps the same register model in Ethernet frames on TCP port 502, the port the specification fixes, wherever the gripper or its gateway offers a network module.

24 V discrete I/O is the oldest pattern: controller outputs for open and close, gripper-side inputs back for object-present and in-position, each handshake measured in tens of milliseconds.

This is the same serial habit that carries a barcode reader into a PLC over Modbus RTU — one master, small register sets, short polls.

What is the full command chain for one pick?

One pick crosses exactly two hops, vision system to robot controller and robot controller to gripper, and the second hop carries every grip decision: approach, close, confirm, lift, place, release and the done report back to the vision system.

In order, with the owner of each step:

1. Part arrives; the vision system locates it and computes the pick pose (vision system).
2. The vision system sends the pose to the robot controller and starts a done timer (vision system).
3. The robot acknowledges and runs approach and descend (robot controller).
4. Close command to the gripper; the gripper closes at its set force and speed (robot controller → gripper).
5. The robot reads object-detect and in-position, and retries or faults if either is missing (robot controller ↔ gripper).
6. Lift, transfer, place, release (robot controller → gripper).
7. Done reply to the vision system; its timer stops (robot controller → vision system).

A cycle budget you can check. Assume a locate of 150 ms, a pose string of 32 bytes, a 1,200 ms approach and grip move, and a gripper that closes in 400 ms — a typical figure for an electric adaptive gripper with a 20-50 mm stroke.

The TCP hop costs microseconds: 32 bytes of payload plus about 54 bytes of TCP, IP and Ethernet headers is 86 bytes, so 86 × 8 = 688 bits ÷ 100,000,000 bit/s ≈ 7 µs on the wire of a 100 Mbit/s link. Allow 2 ms for the software stacks at both ends and call the hop 2 ms. The grip-confirm poll reads 4 registers: request 8 bytes plus response 13 bytes is 21 bytes, so 21 × 11 = 231 bits ÷ 19,200 bit/s ≈ 12 ms on the wire. With a 30 ms slave response the poll costs about 42 ms.

150 + 2 + 1,200 + 400 + 42 = 1,794 ms, so plan the station around 1.8 s per pick. The gripper phase is 442 ms of that, about a quarter, and it all runs while the vision system sits idle — which is the point of the architecture: the vision system can locate the next part while the robot still holds this one. Every retry of the confirm poll adds 42 ms, so three retries add 126 ms.

The first hop, pose string to robot, is written up step by step in send camera part coordinates to a robot controller over TCP.

When does a smart camera replace the vision controller as the master?

When one camera does the whole inspection: a smart camera runs the locate itself and sends coordinates straight to the robot controller over TCP or flips a 24 V trigger output, so the cell needs no separate vision PC and the gripper still stays on the robot side.

Choose it for one camera, one part family and a locate simple enough to run on the sensor: presence checks, a label position, a single pick point. The chain loses a box but not the rule: the camera masters the robot, the robot still masters the gripper, and the done signal still closes the loop. A 32-byte pose message is 256 bits of payload, trivial traffic, so the sensor's Fast Ethernet port carries it with room to spare; the same sockets pattern seen from the robot side is in send commands to a smart camera over TCP in C#.

The smart cameras and vision sensors category is where these live. The MV-SC3050XC-16M-ATN-H is a 5 MP color vision sensor with a global shutter, a 1/1.45" sensor and an M12 mount on Fast Ethernet.

When the locate grows into multi-camera guidance, dense point clouds or deep-learning checks, move the master role back to a controller-class PC — the next section.

How many Ethernet ports does a vision controller need for this wiring?

Count one port per camera, one for the robot link and one for the PLC or line network, so a two-camera cell needs four ports before you add spares, which is why five-port and larger controllers are the safe default for robot-guidance cells.

The arithmetic is per port, not per camera, and one NIC per subnet keeps locate traffic off the plant network: a two-camera, one-robot cell with a PLC needs four ports, and one spare makes five the practical floor. Built-in light channels are worth counting on the same visit, because driving the lights from the controller's own outputs removes a separate light controller and its wiring from the cell.

The vision controllers category starts at compact models. The MV-VC3301H-128G66 pairs an Intel Core i3 with 8 GB of memory, 5+ LAN ports, 3-4 USB 3 ports and 4 built-in channels, and ships with a VM6200Pro license.

When the cell adds PoE cameras, heavier models or deep-learning inference, step up a series: the MV-VC5525-256G70-NN runs an Intel Core i5 with 32 GB of memory, PoE camera ports, 8 built-in channels and PCIe expansion.

Both fit the same chain, and the port arithmetic above is the first filter to apply when you shortlist a model.

What breaks when the wiring ignores this architecture?

Timing races break it first: when vision software drives gripper timing from a second channel, an e-stop, a slow register poll or one dropped TCP frame desynchronizes grip from motion, and the symptom is intermittent short picks that no single device can reproduce.

The failures arrive in a predictable order:

1. Grip and motion race. The gripper closes from a second channel while the robot is still descending; the symptom is short picks or dropped parts that change with cycle time. Fix: move close and confirm into the robot program.
2. Done-signal deadlock. No timeout on the done wait, so one lost frame parks the cell; the symptom is a station stuck forever with no error. Fix: a done timer sized just above the slowest measured cycle.
3. Two RTU masters. Someone taps the RS-485 bus into the vision PC as well; the symptom is register timeouts that worsen as the bus gets busier. Fix: one master per segment; other devices listen to data the robot controller publishes.
4. Gripper powered past a stop. The gripper supply sits outside the cell's safety circuit; the symptom is a grip that moves after an e-stop. Fix: gripper power and signals on the controller side of the safety chain — the same reason the architecture puts the gripper there in the first place.

Each failure is invisible until it is intermittent, which is why the chain is worth drawing before the first demo rather than after it.

Which checks settle the wiring before commissioning day?

Nine checks settle it before commissioning day, and each one exists because skipping it puts a race, a deadlock or an uncontrolled grip into a cell that otherwise ran perfectly for three days.

Work them in this order:

• Draw the chain on one page: every device, every link and the master of each hop. Every later argument refers to this page.
• One master per RS-485 segment: set the gripper's slave address (1 in this example), check 120 Ω termination at both ends, and confirm nobody else polls the bus.
• Grip steps inside the robot program: close, confirm and release as numbered steps between approach and lift, never as parallel outputs from a second device.
• Agree the message contract: pose format, units, delimiter and the done timeout; log both sides with timestamps so a stuck cycle shows who was waiting.
• One NIC per subnet: cameras, robot link and plant network separated at the controller.
• PLC, if any, owns its poll loops: when several stations share one CPU, the one-loop rule in multiple vision cameras on Modbus TCP: one PLC CPU, one poll loop applies to grippers and readers alike.

More control-chain wiring lives on the systems topic page. If you are sizing the vision side of the cell, send the camera count and cycle target through the contact page and we will size it with you.