Pan tilt camera control over the internet: latency and interfaces

Pointing a camera on a pan-tilt head from a control room, or from another site, is a latency and wiring problem before it is a software problem. You need two links: a slow command path down to the motors, and a fast video path back to your eyes. This page works the numbers: the delay budget with the arithmetic shown, the choice between RS-232 and RS-485 at the head, what a serial-to-Ethernet bridge does with your TCP bytes, and the rule for when the control loop must stay at the head instead of crossing the internet. Run the budget once and you will know, before buying anything, whether your path supports manual pointing, presets only, or neither.

The two links: commands out, video back

A networked pan-tilt station separates cleanly into two paths, and you design them separately.

The command path carries small, rare messages: absolute angles, speed limits, stored presets. That is bytes per second, not megabits. The video path carries the picture you point by — tens to hundreds of megabits per second, continuous.

At the head sit two servo or stepper axes driven by a motion controller with a serial port, RS-232 or RS-485. A small device-server bridge joins that serial port to the LAN: it accepts a TCP connection, and every byte your application writes to the socket leaves the serial line in order. No special library is needed on the host side — a plain client socket that writes command bytes does the whole job in any language, the standard Java socket classes included. The camera hangs on the same LAN, ideally on the same switch as the bridge.

Where the head should point for a given target is geometry, worked through in Pan a camera to point at a target: yaw angle from coordinates. What follows here is the path the command travels and what it costs in time.

The command latency budget, term by term

Five delays sit between a keypress and the axis starting to move:

• T_serial — clocking the command frame out of the UART.
• T_bridge — buffering and forwarding inside the serial-to-Ethernet bridge, around 1 ms in practice.
• T_lan — the plant-side Ethernet round trip, well under 1 ms on a clean switch fabric.
• T_wan — the internet path, where the speed of light sets a hard floor.
• T_axis — the drive's control period: the axis moves on the next cycle, not instantly.

The T_wan floor is physics. Light travels through fibre at roughly 200,000 km/s, so a path 1,000 km end to end cannot answer in less than 10 ms round trip; real routes measure 10-50 ms regionally once router hops add their share.

The decision rule: hold command-to-motion at 100 ms or less. Professional broadcast remote-head control links are typically specified under 10 ms, so a LAN-only set-up sits far inside that, and a continental internet path lands around 30-60 ms and still feels immediate. Manual pointing degrades noticeably past a few hundred milliseconds — at that point the fix is the path, not the code.

Worked example: a 30 degree pan from 1,000 km away

Set-up: a 6-byte binary command (axis, angle, speed, checksum, framing) sent at 115200 baud, 8N1, across roughly 1,000 km, driving a servo axis that updates on a 20 ms period.

1. Serial frame: 8N1 framing costs 10 bits per byte, so 6 bytes = 60 bits. At 115200 bit/s that is 60 / 115200 = 0.52 ms. Even at 9600 bit/s the same frame costs 60 / 9600 = 6.25 ms.
2. Bridge forwarding: allow 1 ms.
3. Plant LAN round trip: 1 ms.
4. Internet: 2,000 km of fibre at 200,000 km/s gives 10 ms, plus about 2 ms of router hops, so 12 ms.
5. Axis control period: up to 20 ms before the next update window opens.

T_command = 0.52 + 1 + 1 + 12 + 20 ≈ 35 ms from keypress to motion start.

What the operator feels is the picture coming back: exposure 5 ms, encode 40 ms, network 12 ms, decode 30 ms — 87 ms of T_video, so click-to-picture lands around 120 ms. That is usable manual control. A local LAN with a low-latency encoder cuts the same chain to a small fraction, which is why broadcast heads can specify sub-10 ms control.

Which serial interface reaches the head

Two serial standards do this job.

RS-232 is point-to-point over three conductors, and the classic TIA-232 limit is about 15 m at 20 kbit/s — set by allowed load capacitance, so longer runs work at lower rates. Choose it when the bridge and the motion controller share a cabinet, the common case.

RS-485 is a differential pair, multi-drop, and runs to about 1,200 m at 100 kbit/s. Choose it when the controller sits far from the bridge, or when several drives share one line.

Baud rate is barely a factor in the budget: the worked example's 6-byte frame occupies 0.52 ms at 115200 baud and 6.25 ms at 9600 baud, against a 100 ms target. Pick the rate the controller documents and move on.

If the head runs from a microcontroller node inside a robotics middleware stack, the rosserial protocol wraps serialized messages and multiplexes topics and services over the same serial device, so the bridge carries structured messages instead of raw bytes — same wires, different framing.

TCP, not UDP, is the right transport despite retransmit stalls: with 6-byte messages the retry queue is trivial, and losing a motion command is worse than delaying it by one round trip.

Wiring the command line

RS-232 needs three conductors: transmit, receive and signal ground between bridge and controller. RS-485 needs one twisted pair for the A and B data lines plus a ground reference, with roughly 120 Ω termination across the pair at the far end — skip it on short cabinet runs.

For an RS-232 drop to a drive with a terminal block, a DB9 female-to-open lead lands the wires directly:

The 7 m lead comfortably covers a cabinet-mounted bridge to an in-cell controller.

Keep the serial run clear of motor power lines, or cross them at right angles. Stepper and servo drives put fast switching edges on their power wiring, and that is the classic source of corrupted command frames.

The video return path sets the feel

The command link is nearly free; the picture is what makes control feel slow. Glass-to-glass delay stacks exposure, sensor readout, encode, packetization, network and decode. A low-latency H.264 pipeline at 1080p lands in the 100-200 ms class end to end; generic streaming profiles with buffered frame groups sit at 500 ms or more and make the head feel sluggish. Set the encoder for short frame groups and no B-frames before anything else.

Bandwidth comes next. A gigabit link carries about 110 MB/s of payload after overhead, which is the ceiling a GigE Vision camera streams against — why your camera's link bandwidth sets a hard latency floor covers that coupling. Over the WAN hop the encoder dominates instead, so an MJPEG-class stream in the tens of megabits per second often beats a highly compressed profile on perceived delay. If you are still choosing the camera interface, machine vision camera interfaces compared, from Camera Link to USB3 is the map.

Exposure is part of the delay: dropping it from 20 ms to 5 ms takes 15 ms off every frame. The one-frame effects of changing camera settings on the fly are covered in Set exposure every frame: latency and the one-frame delay.

Copper Ethernet is good for 100 m per run; past that, fibre. And keep USB off the moving head — USB cable length limits for 3D cameras: keeping the data link alive explains the short-reach limit that makes Ethernet the cleaner choice for a head that lives metres from the switch.

Keep the vision loop at the head

If the head tracks a moving target from the camera image, the internet cannot sit inside that loop. Jitter of ±30 ms on a 40 ms path means the axis always acts on stale position data, and the loop hunts. Run the detection and the position loop on a host beside the head, and let the WAN carry setpoints downward and status upward while the operator keeps supervisory control. The budget method for a closed image loop is set out in Image based control latency: budget the full feedback loop, and the grab-and-process split that shortens it is in Two camera closed loop latency: grab and process in parallel.

Mounting and cabling at the head

A pan-tilt head is a lever: every newton at the lens front becomes a torque on the pan axis, and a soft base turns commanded motion into wobble the camera then photographs. Give the head a column and base plate with real mass:

The 700 mm column on a 400 x 450 mm base suits a bench or test-cell mount, and the camera stands range covers slide and bracket variants for fixed stations.

Two cables matter at a moving head. At the pan axis, where the lead flexes every cycle, use a drag-chain-rated cable: standard patch leads fatigue in copper long before a chain-rated lead does, and chain grades are rated in the millions of flex cycles. From the head back to the switch, a longer run with connectors that lock takes vibration out of the link:

The 3 m high-flex lead lives at the joint; the 15 m locking-stud lead runs back to the cabinet. Both come from the camera cables range, which the accessories finder filters by length and type.

Decision rules by use case

For presets the WAN delay is nearly irrelevant: the command is a few bytes sent once per move and the head executes locally, so even a 300 ms path gives a usable remote station. For manual pointing, run the worked example's arithmetic against your measured path before committing to the design.

Checks before you build

• Probe the real path for a full day and use the 95th-percentile round trip, not the best ping: a route that averages 40 ms but spikes to 300 ms feels worse than its average.
• Put the encoder in low-latency mode and measure glass-to-glass with a stopwatch against a split screen before wiring the head.
• Confirm the motion controller's frame format and baud from its manual, then write the socket client against that framing.
• Rate every cable at the pan joint for continuous flex.
• Test link-loss behaviour: pull the WAN cable and confirm the head parks rather than drifts.

Once the budget closes, the hardware is ordinary: a stand, a bridge, two cables, and a camera whose interface you chose for bandwidth. Assemble the station from the vision system configurator, and if the head is one axis of a guided cell, send the layout through the quote request for a matched parts list.