Read X, Y, angle and score from a vision camera over a UR socket

You have a vision camera over the cell, its job finds the part, and the UR cobot must receive four numbers over TCP: pixel X, pixel Y, rotation in degrees, and a match score. Two things fail in almost every first attempt. The read returns an empty string because nothing triggered a fresh result, and the pixel values arrive intact but mean nothing to the robot until they pass through a scale, a rotation and an offset. This guide fixes both: the URScript read cycle that reliably returns four floats, and the calibration arithmetic that turns them into a goal in robot millimeters. How the whole pick cycle then times out is covered in 2D vision guided robot pick and place: how the cycle works.

What the camera should send

Make the job write one line per result: the four values as plain ASCII numbers separated by commas, in a fixed order — X, then Y, then angle, then score — and terminate the line with carriage return and line feed, byte 13 then byte 10. Keep prefixes, units and field names off the line: every character of header is a character the robot must strip before it has numbers.

The trigger command is whatever string the camera's job expects; a single character is common. Some socket servers stay quiet on purpose until you log in. If the connection opens but no result ever arrives, send the account name and password the camera's documentation specifies, each on its own line and terminated with byte 13 and byte 10, before the first trigger.

A smart camera that runs the locate job on the device writes this line straight off its own socket server, with no PC in the cell. The MV-SC5060XM-08M-WBN is that class of part, and a 5 MP color vision sensor such as the MV-SC3050XC-16M-ATN-H does the same job. Rule-based locate or a trained model is a separate choice, walked through in AI vs rule-based smart camera inspection: how to choose. The camera side starts in smart cameras.

The read cycle, step by step

1. Open the socket once, before the program runs.

conn = False
while conn == False:
    conn = socket_open("10.10.10.5", 3000, "socket_0")

Substitute your camera's address and the port its socket server listens on. The socket name, here "socket_0", must be identical in every later call. Keep the connection for the life of the program; reopening per part wastes seconds.

2. Trigger, then read, in the same cycle.

socket_send_string("r", "socket_0")            # the trigger command your job expects
socket_send_byte(13, "socket_0")               # CR
socket_send_byte(10, "socket_0")               # LF

res = socket_read_ascii_float(4, "socket_0")   # returns [count, X, Y, angle, score]

if res[0] == 4:
    x_px  = res[1]                             # part center, pixel column
    y_px  = res[2]                             # part center, pixel row
    ang_c = res[3]                             # part rotation in the image, degrees
    score = res[4]                             # match score from the locate tool

socket_read_ascii_float accepts up to 30 values per call; you need four. The count at index 0 is your guard: anything other than 4 means no part was found, the read timed out, or the line was broken. Send that down the reject path, never into the transform.

3. Gate motion on the score before any transform. Accept at 70 and above, re-trigger once from 60 to 69, reject below 60 and archive the image. A goal computed from a weak match is worse than no goal.

4. Read text only when the camera cannot drop its prefix. Some jobs emit header text around the numbers; read with socket_read_string and convert the fields after. The pattern for turning camera result strings into float arrays covers both forms. After each socket_read_string result, issue one extra read to consume the trailing CR and LF, so stale characters cannot prefix the next line.

5. If nothing at all comes back, fix the send side. A silent server behind an open connection is a different failure — robot socket communication: no response, fix the send loop covers it.

Why the variable goes empty

socket_read_string returns an empty string when the read times out or the reply is invalid. That is the documented behaviour, and it is the only diagnostic you get. The default window is 2 seconds, so a read issued without a fresh trigger gives the camera nothing to send and comes back empty. In the cell this looks like data for the first few cycles, while old results are still buffered, and then nothing at all.

Three fixes, in order:

1. Trigger immediately before the read, in the same cycle, as in step 2 above.
2. If the job cannot answer inside 2 seconds — a long exposure on a dark part, a heavy model — pass the timeout parameter 0 or a negative value; the call then waits until the read completes.
3. Do not put the camera in continuous snapshot mode to paper over it. A continuous stream leaves stale lines in the buffer, and the robot consumes them one per read, always one result behind the part.

Worked example: pixel (1040, 560) at -25° into robot millimeters

Example inputs for an overhead camera on a UR10e-class cell:

• Camera result: X = 1040 px, Y = 560 px, angle = -25.0°, score = 92.3.
• Calibration pin A: a fixed point at robot (300.0, 150.0) mm, imaging at pixel (400, 200).
• Calibration pin B: 100.0 mm from pin A along robot X, at (400.0, 150.0) mm, imaging at pixel (400, 1200).
• Image size 1600 × 1200 px.

The scale comes from the pin pair, the rotation from the pair's direction, and pin A anchors the offset:

The arithmetic, written out:

• Scale: pin B sits 1200 - 200 = 1000 pixel rows from pin A across 100.0 mm, so s = 100.0 mm / 1000 px = 0.1 mm/px.
• Rotation: 100.0 mm along robot X shifts only the pixel row, so image rows point along robot +X and image columns along robot -Y: t = -90°, with cos(t) = 0 and sin(t) = -1. Convert t to radians before your cos and sin calls.
• Camera-frame offsets: x_c = (1040 - 400) × 0.1 = 64.0 mm; y_c = (560 - 200) × 0.1 = 36.0 mm.
• Robot goal: x_r = 300.0 + 64.0 × 0 - 36.0 × (-1) = 336.0 mm; y_r = 150.0 + 64.0 × (-1) + 36.0 × 0 = 86.0 mm.
• Part angle: a_r = -25.0° + (-90.0°) = -115.0°, already inside -180° to 180°. Wrap anything outside with a_r = ((a_r + 180) mod 360) - 180.

The decision: drive the tool to (336.0, 86.0) mm at -115.0° and pick. The score of 92.3 sits 22.3 points above the 70 accept line.

At 0.1 mm/px, one pixel of centroid noise is 0.1 mm of robot error, and ±0.5 px of jitter puts ±0.05 mm on the goal. Pixel coordinates count pixel centers, not corners — half-pixel offset in blob centroid: pixel coordinate origin covers the convention. The pixel budget runs the other way when you choose the camera: pixels needed = field of view ÷ mm per pixel, so a 160 mm field at 0.1 mm/px needs 1600 px across.

This overhead-camera layout is eye-to-hand; the full pose chain including tool offsets is in eye to hand object pose transform for robot picks.

Checks before the first move

Run these with the part static and robot speed limited:

1. Marker test. Command the computed goal, here (336.0, 86.0) mm, and jog a pointer to it. It should land within 1 mm. A miss along one axis is a sign flip; a miss that grows with distance from pin A is a scale error.
2. Rotation test. Turn the part by a known 30.0° and trigger: the reported angle must change by exactly 30.0°.
3. Shift test. Slide the part 50 mm along robot X and trigger: at 0.1 mm/px the pixel row must move by 500 px.
4. Score stability. Ten triggers on the static part: a swing of more than 2 points means glare, focus or exposure is drifting. The causes behind moving match scores are collected in wrist camera score-change troubleshooting.

Keep people out of the cell and the speed low until the marker test passes: a goal from an unproven transform is an unproven motion.

When to move the math off the robot

The pendant pattern is right for one camera and one robot. Move the transform onto a vision controller when you add a second camera, when the locate model outgrows a smart camera, or when the PLC also needs the result. The controller hosts the cameras, runs the transform, and hands the robot one line; the read cycle above stays identical.

The MV-VC5525-256G70-NN is a VC5000 controller with PoE camera ports and 8 built-in channels for lights. The range starts in vision controllers. If the consumer is the PLC rather than the robot, send vision coordinates to a PLC through a register handshake is the pattern; the robot-side TCP variant is send camera part coordinates to a robot controller over TCP. More robot-integration pieces are collected under robotics.

Checklist before you run

• One result line per trigger: X, Y, angle, score as comma-separated ASCII numbers, terminated with byte 13 and byte 10.
• Socket opened once at program start; trigger and read share one cycle on the same connection.
• The res[0] == 4 guard in place, and the empty-reply branch tested by pulling the camera cable.
• Login lines sent first if the server requires them.
• Two-pin calibration recorded, transform verified by marker test, 30.0° rotation and 50 mm shift.
• Score gate set: accept at 70, re-trigger from 60, reject and archive below 60.
• First approaches at reduced speed with the cell clear.

When the camera side is still open, the system configurator chains camera, lens and light in one pass, and the contact page reaches us for cell-specific numbers.