Send camera offsets to a robot over TCP socket messaging

A camera that finds the part and a robot that grips it still need one thing between them: a channel that carries the correction in the moment between exposure and pick. That channel is an Ethernet TCP/IP socket. Robot controllers expose TCP/IP sockets through a socket messaging option — commonly bundled with the PC-interface option — and it moves short ASCII strings in both directions, so a smart camera or a vision PC can send 13.7,-2.2,0.0 and the robot adds those millimetres to its taught target before the gripper closes. This guide covers who connects to whom, the message grammar, the set-up steps on both ends, how many decimals your pixels justify, and the checks that keep offsets honest.

The decision: bidirectional sockets, no fieldbus card

You do not need a fieldbus card to move pick offsets. Socket messaging is TCP/IP string traffic, robot to PC and PC to robot. A smart camera with an Ethernet port uses the same path, because to the controller it is just another TCP client.

Commit to one architecture per cell. With one camera and one robot, make the robot the server: it listens on a fixed port, the camera connects after power-up and stays connected, and every part result is a single short write. The robot program parses the string and shifts the taught target; the unit and sign arithmetic for that step is covered in applying camera XY offsets in a robot program. For the shape of a complete exchange, read sending camera part coordinates to a robot controller over TCP alongside this guide.

Three ways to build the vision end

The smart-camera path removes the PC entirely. The LS-SMC-0063 is a GigE smart camera: 1.6 MP at 60 fps, a 16 mm lens and built-in polarized white light, so one housing lights the scene, finds the part and reports the result. For finer localization, the MV-SC5060XM-08M-WBN puts 6 MP mono behind an 8 mm focal length on a 1/2.4-inch sensor; trigger it while the part stands still, because its rolling shutter smears motion. Two examples from the smart cameras range:

When several cameras or heavier processing join the cell, run the socket from a vision controller — a single box that holds camera ports, light outputs and the application writing to the robot:

An Intel Core i3 processor, 8 GB of memory, a 128 GB SSD and four built-in light channels run the vision job, the lighting and the robot socket from one address. The parsing side of a complete exchange is shown in smart camera to robot TCP data exchange that parses results, and the client code pattern in sending commands to a smart camera over TCP in C# applies to any socket client. Browse the smart camera finder for the full range.

The message grammar: one line, fixed fields

Write the grammar down once and never change it mid-production. The format that survives commissioning is fixed field order, comma separator, millimetres only, fixed decimals, newline terminator:

OK,13.7,-2.2,0.0 + LF — status token, then x, y and z relative to the taught target, already converted to robot frame.

• One line per result, terminated with LF; the robot reads until it sees the newline.
• Fixed decimal places, so every good line has the same length.
• A failed read sends NG,0.0,0.0,0.0 — the robot always receives a complete, parseable line.
• The z field is sent even when flat parts make it 0.0 every time; empty fields break parsers.
• Prefix a sequence number when two consumers share one stream, so repeats are visible.

Parsing is one split and three conversions. In generic pseudocode, with example values:

string line = "OK,13.7,-2.2,0.0"        // example received line
string[] f = split(line, ",")           // ["OK", "13.7", "-2.2", "0.0"]
string status = f[0]                    // "OK"
float x = to_float(f[1])                // 13.7
float y = to_float(f[2])                // -2.2
float z = to_float(f[3])                // 0.0
// test: status == "OK" and abs(x-13.7)+abs(y+2.2)+abs(z) < 0.001

The variable names above are illustrative; every controller's socket toolkit exposes create, bind, listen, accept, read, write and close — map them to your model's instruction names. The same string flows whether the sender is a camera job or a PC application; sending string data from a vision system to a robot over TCP walks a full exchange end to end.

Setting up the path, step by step

1. Fix the addresses. Put camera and controller on one private subnet with DHCP off — for example camera 192.168.1.10, controller 192.168.1.20, mask 255.255.255.0 — and write both on the cell drawing.
2. Enable socket messaging on the controller and pick a TCP port above 1000 that nothing else on the controller claims. One port per vision device.
3. Write the robot-side listener. Create the socket, bind, listen, accept one connection, then loop: read to the newline, parse, write ACK, update the offset variables.
4. Write the camera-side client. Connect at job start; on failure retry every 2 s in the background rather than stopping the job.
5. Add the handshake. After accept, the robot sends READY, and the camera arms its trigger only then, so no result is written before the listener exists.
6. Wire the trigger. The robot or the PLC starts the exposure, and the camera returns exactly one line per trigger; triggering a USB camera capture from a robot controller over a socket is the same handshake with the roles reversed.
7. Clamp before you move. Reject any offset larger than half the field of view: that is a false detection or the wrong part, not a part that moved.
8. Apply the offsets in the motion program, with the unit and sign conversion done in exactly one place, documented on the cell drawing.
9. Test the failure path. Pull the cable mid-shift: the camera must flag, the robot must stop on timeout, and both must recover when the link returns. Silent bad picks usually trace to a reconnect nobody tested; see robot socket communication: no response, and the send loop that causes it.

Worked example: one-decimal offsets on a 300 mm pick scene

Inputs, each with its source:

• Scene width to cover: 300 mm, the feed-tray window.
• Camera: a 6 MP smart camera, about 3000 pixels across the width.
• Lens and sensor: an 8 mm focal length on a 1/2.4-inch sensor, about 5.3 mm wide.
• Detection: part centre at pixel (2637, 978); taught nominal centre (2500, 1000).

Arithmetic:

• Working distance: 8 mm × 300 mm ÷ 5.3 mm = 452.8, so mount at about 450 mm.
• Scale: 300 mm ÷ 3000 px = 0.1 mm per pixel.
• x offset: (2637 - 2500) px × 0.1 mm/px = 137 × 0.1 = 13.7 mm.
• y offset: (978 - 1000) px × 0.1 mm/px = -22 × 0.1 = -2.2 mm.
• Message: OK,13.7,-2.2,0.0 plus LF is 17 bytes, about 1.4 µs of wire time at 100 Mbit/s. The socket never gates a cycle measured in tens of milliseconds.

Decisions these numbers force:

• One decimal place. 0.1 mm is exactly one pixel here; a second decimal claims ten times finer precision than the image holds.
• Clamp at ±150 mm, half the field of view; anything beyond is rejected as a false detection.
• Convert signs before sending. Image y grows downward and robot y grows upward, so the camera job sends robot-frame values — that is why the line reads -2.2, not +2.2.

Mount the camera on a rigid column, not on a guard door: a column knocked 2 mm sideways adds a constant 2 mm to every offset, and the ±150 mm clamp never catches it. A slide-table stand of this class:

Its 700 mm column height on a 400 × 450 mm base holds the camera stiff at the working distance. At the example's 450 mm there is headroom to focus; see the camera stands range for tilt and slide variants. Recheck the taught nominal after any mechanical change, on the schedule given in camera calibration depth range and accuracy: when to recalibrate.

Checks before the first automatic shift

• Fixed addresses. A DHCP renewal at 06:00 breaks the client quietly; keep both ends static.
• Firewall. Every PC in the path must pass the chosen port both ways — the rule set is the one described for vision controller to PLC TCP ports and firewall rules.
• Terminator match. LF in the sender and LF in the parser; a CR LF pair against an LF parser leaves a stray character in the last field.
• Timeout and fallback. Set the robot's read timeout above the slowest measured trigger-to-string time, doubled for margin; on expiry it holds and faults, never picking on the previous part's numbers.
• Sequence guard. Number the lines and let the robot ignore a repeat, so one retransmit cannot move the arm twice.
• Job switching. If the cell runs several products, switching vision jobs mid-program without losing the camera socket keeps one connection across recipes.

When the usual answer is wrong

• The camera moved. Offsets correct around a taught nominal; after a bump every part reads shifted by the same constant and the clamp stays silent. Re-teach, then verify with a part placed 10 mm off-centre: the reported offset should read 10.0 mm ± 0.3, which is three pixels at this scale.
• The light changed. A detection that drifts near part edges is usually ambient light, not the socket; start with controlling ambient light variation in pick and place vision.
• The part moves under a rolling shutter. Trigger at standstill or move to a global-shutter camera; a smeared edge biases the centre estimate in the direction of travel.
• The cell grew. Two consumers reading one camera need one server and tagged lines; smart camera socket handoff to a robot: straight-down pick shows the single-robot version to start from.

Quick checklist before commissioning

• One subnet, static addresses, one port per device
• Robot as server, camera as client, reconnect every 2 s
• Grammar OK,x,y,z + LF, fixed decimals, NG line on failure
• Offset clamp at half the field of view
• Read timeout, fallback and sequence guard in the robot program
• Cable-pull recovery tested; first moves at reduced speed
• One place, documented, for the unit and sign conversion

Build the camera end in the system configurator, or send your message grammar to our applications team through the contact page for a review before commissioning.