Live camera image in an HMI runtime: snapshots vs streams

Your smart camera inspects the part, the PLC runs the machine, and the operator asks the obvious question: what does the camera actually see? Getting that picture onto an HMI runtime is a plumbing decision, not a vision decision. Most panel runtimes cannot decode a camera stream directly, so you either serve periodic snapshots that an image object reloads, or you put a vision controller in the middle and let it publish frames to a video, web or vendor display object. This page gives you the refresh rates, bandwidth arithmetic and latency figures that pick the route for you, the failure that leaves a camera control blank on screen, and the parts that make each route work.

What an HMI runtime can actually display

A panel runtime paints objects. It reads tags, files, URLs and add-in components; it does not run a video pipeline of its own. Camera pictures reach the screen through four object types:

1. Image object bound to a file path — the runtime re-loads a JPEG from a network share or local folder, on a timer or when a tag changes.
2. Image object bound to a URL — the runtime fetches a JPEG from the camera or controller over HTTP at a set interval.
3. Camera or video object — a stream-aware object in newer runtimes that decodes an RTSP or MJPEG URL directly.
4. Vendor display control — an ActiveX/OCX or .NET component supplied by the vision software vendor that pulls frames from a running vision program.

Types 1 and 2 are snapshots; types 3 and 4 are streams. Everything below is the choice between them, with the arithmetic that says how fast each one can go. What a smart camera itself draws on screen inside a PLC HMI is a related but separate question: what a smart camera image in a PLC HMI actually displays.

Route one: the snapshot feed

The vision device or controller saves the current image as a JPEG where the panel can read it. That place is a network share the image object polls, an FTP drop folder, or an HTTP snapshot URL the object fetches. The object reloads every 250-1000 ms, or on the tag that pulses when an inspection finishes. The trigger beats the timer: the operator sees the part that is under the camera now, not a picture half a cycle old. The same registers that send vision coordinates to a PLC through a register handshake can carry the refresh tag.

Worked example, 5 MP snapshots at 2 Hz over Fast Ethernet:

• A 5 MP frame is 2592 × 1944 px = 5.04 MP.
• At moderate JPEG quality, about 0.12 bytes/px on industrial scenes: 5.04 × 0.12 ≈ 0.60 MB per image.
• Wire time: 0.60 × 8 ÷ 100 = 0.048 s, about 50 ms on a 100 Mbit/s link.
• Feed bandwidth at 2 Hz: 0.60 × 8 × 2 = 9.6 Mbit/s, a tenth of that link.
• Panel reload and paint add typically 150-400 ms, so the picture is 0.2-0.45 s old; 2-4 Hz is realistic with full frames.
• Thumbnail route: 640 × 480 at the same quality is about 36 KB, so 3 ms of wire time and 1.4 Mbit/s at 5 Hz.

The two inputs that move these numbers: JPEG quality, where 0.05-0.15 bytes/px is the moderate band and flat, high-contrast scenes sit at the low end, and the panel's own reload cycle, which you can read off the refresh setting.

Two compact picks for snapshot duty:

A 5 MP color vision sensor on Fast Ethernet covers the full-frame case:

The pattern is well worn, so a configured example helps: displaying IP camera snapshots on an HMI panel walks the image object settings end to end.

Route two: the runtime's own video object

Newer HMI runtimes ship a camera or video object that decodes an RTSP or MJPEG URL. This is the only route where the panel itself consumes a stream. Point it at the camera's sub-stream, not the main stream: network cameras serve both. The sub-stream exists for live viewing, typically 640 × 480 to 1280 × 720 at 10-25 fps and 1-4 Mbit/s. The main stream, 1080p H.264 at 2-8 Mbit/s, stays with recording or analytics.

Three limits decide whether this route opens at all:

• Decode count: panel-class runtimes commonly handle 1-4 streams at once, fewer at full panel resolution.
• Codec support: many video objects decode H.264, while MJPEG over HTTP is the lowest common denominator and the one to test first.
• Busy-panel sag: under load, stream frame rates drop before anything errors out; the picture slows down instead of failing.

For URL forms and the practical settings, displaying an RTSP stream in an HMI with a snapshot URL or VLC covers the cases that come up on real panels, and fitting multiple IP camera streams on one HMI screen covers the multi-camera layout. For the same objects configured end to end, integrating webcam and IP camera streams into an HMI runtime is a complete walkthrough.

Route three: the vision controller bridge

When the runtime has no video object and the job needs motion, put a vision controller between camera and panel. The controller grabs full-rate frames from the camera over its own LAN and republishes them in one of three ways:

• a vendor display control in the HMI screen, fed by the controller's vision software;
• an HTTP MJPEG page the controller serves, shown in a camera live stream in an HMI web viewer widget;
• per-frame JPEG writes to a share, for a plain image object reloading at 5-15 Hz.

Grab, encode and republish add roughly 50-150 ms on a controller-class CPU (an estimate; measure it once the feed runs). The camera-to-controller leg is the one to size honestly. The LS-SMC-0063 runs 60 fps at 1.6 MP over GigE with a 16 mm lens. At 8 bits per pixel, that leg carries 1.6 × 60 = 96 MB/s, roughly 770 Mbit/s — three quarters of one GigE link before packet overhead. That traffic belongs on the controller's own switch port or subnet, never on the plant network, and it is why the panel receives republished thumbnails or a sub-stream rather than camera frames.

A 6 MP mono GigE smart camera with an M12 mount fits the camera side:

The same publishing logic scales upward: streaming smart camera live video and metrics to a custom dashboard runs the identical pattern one layer up.

The blank camera control: causes in order

A vendor display control that sits empty in runtime, while the vision software runs fine, has three causes, in order of probability:

1. Never registered on the runtime machine. Component registration is per machine and per bitness; the engineering station where the screen was designed is not the machine that runs it. Register the control on the machine running the HMI runtime, with the Windows regsvr32 command run as administrator, matching the control's 32- or 64-bit build to the runtime process.
2. Bitness mismatch. A 32-bit control will not load into a 64-bit runtime process, or the reverse; registration succeeds and the object still draws nothing. Check the runtime process bitness against the control build.
3. No frames to draw. The control is a client of the vision software, not a camera driver. If the vision program is not running, or its connection to the control is not configured, the rectangle stays empty. Start the program first, then check the control's connection settings.

Where the vendor also offers a file or HTTP interface, prefer it: an image object watching a file fails in visible ways, while an in-process control fails silently. The same idea applies one level down at the device itself: reading live frames from a robot-mounted camera over TCP port 4242 shows a camera serving frames over its own socket.

Snapshots vs streams at a glance

Read the table by the question the operator is really asking. What is in the station right now is a snapshot question. Is the feed moving, is the part centered, is the chute jamming — those are stream questions.

Which route fits which job

• Legacy panel, image object only, operator context. Per-part snapshot at 1-2 Hz, written by the vision device, refreshed by the inspection-complete tag. Decision: snapshot.
• Newer runtime with a camera object, stream-capable devices. Sub-stream MJPEG at 10-25 fps, main stream left to inspection and recording. Decision: native video object.
• Set-up and diagnostics need motion on a legacy panel. Controller bridge with a web viewer widget or a registered vendor control; 50-150 ms of added latency is invisible for diagnostics. Decision: bridge.
• Robot or servo guidance. Never the image. The picture is for people, register values are for the motion system. Decision: fieldbus.
• Four or more cameras on one panel. Staggered snapshot timers first, streams only after the decode-count check passes. Decision: snapshots until proven otherwise.

Checks before you order

• List the object types your HMI runtime offers: file image, URL image, camera object, add-in control.
• Get the exact snapshot mechanism from the camera or controller side: per-cycle JPEG write, snapshot URL, or neither.
• Read the runtime's decode limits: how many streams, which codecs, what frame rate survives at full panel resolution.
• For add-in controls, get the bitness of the runtime process and of the control before installation day.
• Plan the network split: camera-to-controller traffic on its own switch or VLAN, the republished feed on the plant network.
• Set the file policy: who writes the JPEG, where it lives, overwritten in place or kept as a short ring.

When the picture is the wrong answer

• Guidance. A snapshot is 0.2-0.45 s old and a republished stream still adds 50-150 ms; a servo that settles in under 100 ms waits for neither. The image is for people; the motion system gets register values.
• Fine defect judgment. JPEG compression and panel scaling hide the 0.2 mm scratch the inspection is hunting. If operators must judge real pixels, give them the vision sensor browser display for local operator viewing or a camera image on the VGA output from a vision controller.
• Quality records. The evidence copy comes from the inspection path with the verdict and station data attached, not from a screen capture of the HMI.
• Cycle time. Display refresh never touches cycle time; the numbers that do are in smart camera image processing time: what is fast enough.

Where to start

Pick by the panel first and the camera second, because the panel decides which of the three routes exists. Then match the device to the route: a Fast Ethernet vision sensor for snapshots, a GigE smart camera for the bridge, and a controller to do the republishing. Browse the smart cameras category, the vision controllers category, the camera finder, or the systems topic for the neighboring set-up questions. For bigger cells, one controller runs the bridge: it holds the camera links, the vision software and the republished feed. A controller with 3-4 LAN ports, 2 USB 3 ports, 4 built-in light channels and 8 GB of memory holds the camera links plus the plant-side port:

Send the panel type, camera count and cycle time through the contact page, and run the configurator below from the controller step.