Show a camera live stream in an HMI web viewer widget

You want the camera picture on the HMI screen, next to the line controls, and the web viewer widget looks like the natural home for it. It is — once you handle one fact: that widget is a browser engine in a frame, and browser engines do not speak RTSP. So the job has two shapes. If the camera lists an HTTP MJPEG or snapshot URL, you paste it into the widget's address property and you are done. If the camera speaks only RTSP, you put a small converter between camera network and HMI network and point the widget at the converted feed. Both routes follow, with the numbers that decide between them: bandwidth per stream, latency, and what a camera drop looks like on screen.

Why rtsp:// stays blank in the widget

A web viewer widget loads a web page, usually inside an iframe. A page can show JPEG stills, multipart JPEG streams (MJPEG) and, with a player script, HLS video. What it cannot do is open rtsp://. No mainstream browser engine ships an RTSP client, so an RTSP address either fails quietly or makes the operating system offer to launch an external player, which never happens inside a closed HMI panel.

Dedicated camera-viewer components on HMI platforms hit the same wall, and one widely used platform documents it plainly: RTSP limits of the Ignition IP camera viewer.

The question that decides the whole job is therefore not about the HMI. Open the camera manual's streaming chapter and look for an HTTP entry: an MJPEG live-view address or a JPEG snapshot address.

Choose the route before touching the HMI

Prefer the first route whenever the camera offers it: no extra hardware, no extra service to babysit, latency of tens of milliseconds. The converter route exists for RTSP-only cameras, and polling stills is the fallback. More integration patterns for cameras, controllers and HMI software sit on the systems topic page.

Route 1: paste the camera's HTTP stream address

1. Find the address. The camera manual's streaming or network chapter lists it; the format differs by camera family. One industrial family prints a live-view address of the form http://192.168.1.64/ISAPI/Streaming/channels/102/httpPreview; other families ship short paths such as /streaming/mjpeg.jpg after the camera's address for MJPEG, or a plain snapshot path for a still. Write down the camera's IP and the account the address expects.
2. Prove it in a laptop browser on the same network before the HMI is involved. An MJPEG address animates on its own, because the browser renders each multipart JPEG part as it arrives. A snapshot address shows one still, and a page reload fetches a new one; the pattern is the same as grabbing images from an IP camera over an HTTP snapshot address.
3. Add the web viewer widget to the HMI screen and put the proven address into its address or source property; the exact property name changes from one HMI to the next. Size the widget to the stream's aspect ratio, 16:9 for a 1280×720 stream, so nothing is cropped.
4. Decide credentials deliberately. Many cameras accept user and password as parameters on the HTTP stream address, and those parameters then sit in plain text inside the HMI project file.

5. Run a full line cycle, start to stop to fault, and watch the picture through all of it.

Two traps close this route: link the stream address, not the camera's home page, because home pages carry menus, logins and plug-ins and some refuse to render inside a frame at all; and a snapshot address pasted alone never refreshes unless the HMI polls it, so if it cannot, use the converter route and get MJPEG.

The numbers: bandwidth and latency

Worked example, one 1280×720 MJPEG stream at 15 fps from a mid-detail scene at moderate JPEG quality:

• frame size ≈ 60 kB per frame; measure yours once from the switch port counter, because scene detail and the quality setting move it by a factor of two or more
• 60 kB × 15 per second = 900 kB/s
• 900 kB/s × 8 = 7200 kbit/s ≈ 7.2 Mbit/s

A 100 Mbit/s link carries about ten such streams with room for bursts. Switch the widget to a 640×360 substream at the same frame rate and the figure falls to about one quarter, roughly 1.8 Mbit/s, because JPEG size scales with pixel count.

Latency separates the outputs. MJPEG forwards every frame as it decodes, so on a switched LAN the picture trails capture by well under 200 ms. HLS cuts the video into segments and plays them on a delay; with the common defaults of a 2-second segment and a 3-segment playlist the picture is 2 × 3 = 6 seconds behind live. Setup screens get MJPEG; a monitoring wall with many viewers can tolerate HLS.

Snapshot polling sits between the two: a poll every 500 ms adds up to 0.5 s of staleness and runs at 1 ÷ 0.5 = 2 frames per second, enough to confirm what is at the station now but not to judge motion.

Route 2: restream RTSP as HTTP on a line controller

1. Get the RTSP address from the camera manual, in the shape rtsp://192.168.1.10/stream (example address; your path comes from the manual), often with separate paths for the main and the sub stream. Prove it first in any desktop media player that accepts rtsp:// addresses.
2. Place a small x86 controller between the two networks: one Ethernet port on the camera VLAN, one on the plant side. That keeps credential-carrying camera traffic off the plant network and gives the converter a mounted, powered, cooled home on the line.
3. Prove the chain in five minutes with one ffmpeg command that pulls RTSP and serves MJPEG to a single viewer:

ffmpeg -rtsp_transport tcp -i rtsp://192.168.1.10/stream \
       -c:v mjpeg -q:v 5 -f mpjpeg -listen 1 \
       http://192.168.20.5:8080/view.mjpeg

Open http://192.168.20.5:8080/view.mjpeg in a laptop browser; a moving picture means the chain works, at MJPEG latency rather than HLS delay. This bare mode serves one viewer at a time, so treat it as the proof, not the installation.

4. Turn it into a service. For a single widget, wrap the same command in an auto-start service with a restart rule. For several widgets, or when you also want the footage recorded, put a media server behind ffmpeg: ffmpeg pushes the stream as RTMP to nginx with its RTMP module, nginx cuts it into HLS, and the widget loads a small page that plays the playlist through a script such as hls.js. Carry the arithmetic from above: about 6 seconds behind live with the common segment defaults.
5. Point the widget at the production address the same way as in Route 1, then load-test with two viewers open while watching the controller's CPU. One 720p MJPEG encode is light work for a current low-power x86 core; several streams or an added recording duty argue for the step up below.

Which controller hosts the converter

The converter is ordinary software, so the host decision is about ports, mounting and headroom: at least two Ethernet ports, one per network side; an x86 processor with room beside the inspection software; industrial mounting and power. A vision controllers unit already sits on the line for exactly these duties, and the camera finder is where the streaming camera itself comes from.

For a single stream the small unit does it:

Step up when the same box takes on more:

All 3 share: Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

Decision per case. One MJPEG stream in a tight cabinet: the MV-VC2140-128G40-1T, an Intel N-series unit with 3-4 LAN ports and 2 USB 3 ports, is enough. The converter plus the inspection software on one box: the MV-VC3301H-128G66 moves up to an Intel Core i3 with a VM6200Pro license included. Cameras to power over the cable plus headroom for several streams: the MV-VC5525-256G70-NN brings PoE camera ports, 8 built-in light channels and 32 GB of memory.

Prove it on the running line

1. Latency. Hold a phone stopwatch in view of the camera and a second stopwatch beside the HMI screen, then read both. The MJPEG routes should agree within about half a second; a 6-second gap on the HLS route is the segment math, not a fault.
2. Frame rate. Wave a hand through the view. Choppy motion means the stream is saturating something: switch the widget to the substream address and redo the bandwidth arithmetic.
3. Drop behaviour. Unplug the camera. Expect the widget to freeze on its last frame, because it does not time out on its own; confirm the no-video flag on the HMI fires from the controller's watchdog ping.
4. Recovery. Power-cycle the camera and the converter. The service must return on its own and the widget must pick the stream up without a manual reload; if it needs one, add the HMI's periodic refresh or an event that re-sets the widget address.
5. Image sanity. Grey bands or torn frames in an MJPEG widget are truncated JPEG frames in the stream, and the MJPEG corruption fix walks that exact symptom. If exposure or gain must change while the stream runs, apply camera settings while streaming without stalling capture. A blank frame or an error page instead of the picture is usually the address itself — port, credentials, HTTP versus HTTPS — and this walkthrough of IP camera viewer URL and stream errors covers the usual suspects.

Common mistakes

• Typing rtsp:// into the widget address and then faulting the HMI: the widget never had an RTSP stack, so the fix sits upstream of it.
• Linking the camera's home page instead of the stream address, and reading a blank frame as a camera fault when the page simply refuses to render in a frame.
• Using the main stream for every widget: one 1280×720 setup view at 7.2 Mbit/s is fine, ten of them across one busy link are not.
• Pasting read-write credentials into the address: it ends up in the project file and its backups.
• Leaving the converter without auto-start: after the first power outage the widget shows a frozen picture that everyone reads as live.
• Expecting the live picture to be the inspection: it is there for people, not for the pass/fail decision.

When a live stream is the wrong answer

An operator glancing at a live picture builds trust; it does not make the pass/fail decision. Smart cameras run the inspection on the device with no PC, and hand the HMI a result plus a triggered image — and that still, shown with cursors and zoom in your HMI panel, often tells an operator more than continuous video does. What a smart camera puts in a PLC HMI is frequently the last processed frame rather than a stream, and what actually displays from a smart camera in a PLC HMI sorts that out. For a code-reading station, pair the decoded string with its image and archive both for traceability, as in barcode inspection image archiving, instead of relying on anyone watching.

If the line needs inspection rather than a picture, smart cameras do it on the device:

Pre-flight checklist

Camera side:

• Stream address, MJPEG or snapshot, copied from the manual and proven in a laptop browser.
• Read-only stream account created; camera on the segregated VLAN.
• Substream chosen for the widget; main stream left to inspection or recording.

Converter side, Route 2 only:

• RTSP address proven in a desktop media player.
• Converter set up as an auto-start service that restarts on failure.
• Bandwidth arithmetic done with a measured frame size and a link sized with headroom.

HMI side:

• Widget address property holds the proven HTTP address; widget sized to the stream's aspect ratio.
• No-video flag wired from the controller's camera ping or snapshot fetch.
• Latency, frame rate, drop and recovery checks all executed on the running line.