USB camera live stream in an HMI: V4L2 driver support decides

You want a USB camera's picture inside a web-based HMI, an operator view beside the machine controls. The question that decides the project is not which widget to use but whether the camera will appear on the controller at all. That is settled in the Linux driver layer, before any HTML is written. This guide gives you the rule, a two-command bench test, the bandwidth arithmetic that sizes the stream, and the hardware checks that keep the decision from coming back after installation.

Which USB cameras will show a live stream in an HMI?

Every USB camera that Linux exposes as a V4L2 device node — in practice, any UVC-class camera — can feed a live stream to a browser HMI. A proprietary USB sensor without a kernel driver cannot, no matter which streaming example you run.

V4L2 (Video4Linux2) is the kernel's camera interface. Once a driver binds, the camera appears as /dev/video0, and every capture method on the controller — an HMI bridge, a recording script, a PLC visualization example — reads from that node. UVC (USB Video Class) cameras get there without a vendor driver: the built-in uvcvideo driver claims them on plug-in, which is why they are the low-risk choice for an operator view. The UVC USB camera on an embedded board: driver-free support guide shows that plug-in behaviour in detail.

Two families of USB industrial cameras behave differently. Webcam-class and board-level cameras are usually UVC. Cameras built to the USB3 Vision standard use the vendor's host library instead of a plain kernel node, so they stream through that SDK rather than /dev/video0. Both can end up on the same browser page — see show a machine-vision camera image in a web browser HMI — but only the first group works with a generic V4L2 streamer. The data sheet's interface line tells you which group a camera belongs to before you order.

Compact USB3 board-level cameras are the usual pick when one camera must serve both the HMI view and the inspection.

LS-BCM-0003: 3.1 MP at 120 fps over USB3 with a global shutter and C-mount, the version for moving belts. MV-CB060-10UM-B: 6 MP mono on a bare board with a 1/1.8" rolling shutter, for mounting where a housed camera will not fit.

How do you check V4L2 support before building the page?

Plug the camera into the controller and run two commands: v4l2-ctl --list-devices should print the camera's name beside /dev/video0, and --list-formats-ext should list at least one MJPEG or YUYV format at the resolution and frame rate you need.

• v4l2-ctl --list-devices — proves a driver bound, and shows which node belongs to which camera.
• v4l2-ctl -d /dev/video0 --list-formats-ext — proves the node captures, and lists the pixel formats and frame rates it offers.
• dmesg | grep -i uvc — shows the kernel's side of the bind when the first command prints nothing.

A camera can expose more than one node: capture lives on the one that lists formats, while the others carry metadata or controls. If controls seem missing, they may simply sit on the other node — v4l2-ctl list-ctrls returns nothing: controls live on the right node walks through that case. When --list-devices shows nothing at all, start with UVC camera not detected: identify the device node first.

Which pixel format should the live stream use?

Pick MJPEG when the camera offers it: at 1080p a YUYV stream needs about 124 MB/s while the same picture as MJPEG needs roughly 12 MB/s at a 10:1 ratio, and only the second fits comfortably on a shared USB 2.0 port.

YUYV stores every pixel uncompressed at 2 bytes per pixel. MJPEG compresses each frame to JPEG, typically 8:1 to 15:1 depending on scene detail. A camera that outputs MJPEG natively also keeps the controller's CPU low, because the bridge forwards JPEG frames and the browser decodes them. H.264, where a camera supports it, shrinks the stream by another order of magnitude and plays through the browser's video element as fragmented MP4; budget a few hundred milliseconds of extra latency for the encoder.

Format mismatches are the classic black-image cause: the node opens, frames arrive, the widget stays dark, because the bridge negotiated a format it cannot decode. USB camera black image on Linux: fix UVC format negotiation shows the renegotiation fix.

How much USB bandwidth does a live stream consume?

Multiply width × height × bytes per pixel × frame rate: a 1280 × 960 YUYV stream at 30 fps moves 73.7 MB/s, above the roughly 40 MB/s a USB 2.0 port delivers in practice, so that camera drops frames unless you switch to MJPEG.

The arithmetic bites hardest at high resolution. A 6 MP frame around 3000 × 2000 pixels is 12 MB in YUYV; at 30 fps that is 360 MB/s, so uncompressed streaming is a USB 3.0-only proposition, and even MJPEG at 36 MB/s takes most of a shared port. When the HMI view must double as a 20 MP inspection view, USB 3.0 is the floor: the MV-CS200-10UM pairs a 1" sensor with USB3.0 and a C-mount.

What sits between the V4L2 node and the browser widget?

Two small pieces: a server-side bridge that reads frames from the V4L2 node and serves them over plain HTTP, plus one browser element in the HMI page that displays the stream.

Two details decide whether the page works on the first try. The bridge must write its HTML fragment into the web server's actual document root. When the stream plays from a direct browser tab but the HMI widget stays blank, the fragment usually landed in a stale web-root path; USB camera at /dev/video0 but no video in your app UI sorts out that class of symptom.

Second, protect the stream on anything beyond an isolated bench network: camera stream authentication in an HMI browser when URL logins fail covers the login cases. For the page side, embed an MJPEG stream in a browser HMI without a file download and restream an RTSP stream for display in an HMI browser give ready patterns, and scale a camera image to fit an HMI widget the right way keeps the aspect ratio honest.

How many USB cameras can one controller stream at once?

Budget by bandwidth, not by port count: one USB 3.0 host sustains roughly 400 MB/s, so four 1080p MJPEG streams at 12 MB/s each total under 50 MB/s and fit easily, while four uncompressed YUYV streams at the same settings would not.

Rates per stream at 30 fps (MJPEG assumed at 10:1):

On the controller side, count the USB 3 ports left after the inspection cameras: the MV-VC2140-128G40-1T carries 2 USB 3 ports on an Intel N-series processor with 8 GB of memory, and the MV-VC3301H-128G66 moves up to an Intel Core i3 with 3-4 USB 3 ports and 5+ LAN ports.

Decompression happens in the browser, so pass-through MJPEG leaves the controller's CPU nearly idle; converting YUYV on the way in costs real CPU — expect tens of percent of one core at 1080p30 with a software encoder such as the ffmpeg-style bridges.

When does a USB camera fail to stream in an HMI?

Almost every failure falls into one of four buckets: no kernel driver, the wrong device node, a format the bridge cannot decode, or a permissions error; each has a test that takes under two minutes.

1. No node appears. The sensor needs a vendor driver. Test: no entry from v4l2-ctl --list-devices and no bind line in dmesg. Fix: run the vendor SDK on the controller and push its frames into the same HTTP bridge, or fit a UVC camera for the HMI view.
2. The wrong node is opened. Multi-node cameras put capture on one node and metadata on another, so the stream stays black or frozen. Test each node with --list-formats-ext; the one listing MJPEG or YUYV is the capture node.
3. The negotiated format is undecodable. Buffers arrive that the bridge cannot turn into browser pixels. Renegotiate explicitly to MJPEG or YUYV when the bridge opens the node.
4. Open fails with permission denied. The service user sits outside the video group. Linux camera device permissions: video and media node checks gives the node-by-node checks.

The sneaky fifth: nodes renumber when cameras are replugged or added, and a page pointed at /dev/video0 suddenly shows the wrong camera. Point the bridge at the stable /dev/v4l2/by-id/ path instead; multiple USB cameras on Linux: stop device path conflicts has the enumeration discipline.

What should you check before ordering cameras for an HMI stream?

Before you order, confirm four things on the data sheet and one on the bench: interface class, compressed-format support, shutter type for moving parts, and the controller's USB 3 port count — then run the two-command check.

• Interface class. UVC on the data sheet means generic V4L2 streaming works; USB3 Vision means planning the vendor SDK on the controller.
• Compressed formats. MJPEG or H.264 at your resolution and frame rate; uncompressed-only cameras cost ports and CPUs.
• Shutter type. A moving belt viewed through a rolling shutter smears; use a global shutter when parts move through the operator view.
• Controller ports. Count USB 3 ports and keep bandwidth headroom for a second stream; vision controllers list their port counts as filters.
• Cable reach. USB 2.0 is specified to 5 m and USB 3.0 to 3 m; beyond that, move the camera to GigE and keep the browser side unchanged.
• Trigger duty. If the same camera also catches passing parts, set the trigger with V4L2 controls — UVC camera external trigger on Linux: enable it with V4L2 controls.

For a fixed operator view on a short USB 3.0 run, the LS-CMR-0166 covers 12 MP over USB3 with a 1/1.7" rolling-shutter sensor at 30 fps and a 4000 × 3000 resolution.

The decision, per case: an operator view within 3 m on an existing USB 3.0 controller — a UVC-class USB 3.0 camera into an MJPEG bridge; the same view at 5-100 m — a GigE camera into a switch, identical browser side; a proprietary USB sensor already on the bench — keep it and add the SDK-to-HTTP bridge rather than replacing hardware. Shortlist cameras in the camera finder, assemble the whole system in the configurator, or send us the layout through the contact page. More controller-side patterns sit in the systems guides.