Multiple USB cameras drop frames: the MJPEG decode budget

You wired four USB cameras to a hub on an embedded ARM board, set each to 1080p30 MJPEG, and lsusb -t reports every camera at 5000M. Yet the moment all four windows go live, frames vanish. This guide gives you the two budgets that decide whether that set-up survives — the isochronous bandwidth the cameras reserve on the wire, and the decode bill the host pays per pixel — plus a measurement routine that shows which budget you have exceeded. The worked example covers four 1080p30 MJPEG streams; the same arithmetic scales to any frame size, frame rate and format printed on a camera data sheet.

Read lsusb -t first: 5000M is the link, not the budget

lsusb -t prints one line per device with its negotiated link speed. The 10000M root hub is your USB controller; the 10000M hub line is its uplink; each camera reading 5000M holds its own 5 Gbit/s link to that hub. Four cameras therefore ride four separate downstream links that all funnel through one 5 Gbit/s uplink, and the uplink is the shared resource: roughly 4 Gbit/s usable once 8b/10b encoding and protocol overhead are paid.

What the tree does not show is the reservation. UVC cameras stream over isochronous transfers, a mode that reserves a slice of every 125 µs service interval for the camera whether the frames fill it or not. The slice size comes from the camera's descriptors, not from the data it currently sends, so a camera costs its reservation when idle and when busy. Hub topology — which camera hangs off which hub, and what each level carries — is worked through in USB3 hub bandwidth for multiple machine vision cameras, and the same per-link arithmetic applies to USB3 Vision cameras in Reduce USB3 camera bandwidth: the per-camera budget math.

One finding in the tree stops everything: a camera line reading 480M. That camera fell back to high speed, where the whole bus shares about 48 MB/s of periodic bandwidth, and four 1080p30 MJPEG streams do not fit in that. Cable length, a marginal connector and hub power are the usual triggers; the recovery checks are in Camera detected as USB 2: why streams drop and how to fix it.

The bandwidth budget: four MJPEG streams are small

MJPEG sends every frame as JPEG, so wire load follows compression quality, not pixel count alone. At the quality settings UVC cameras ship with, 1080p frames land near 0.4-1 bit per pixel: the low end for monitoring, the high end when you need readable detail in stills.

At 5000M, the wire is not why frames drop. The picture inverts on a 480M link: high speed allows about 48 MB/s of periodic traffic (80% of the 60 MB/s a 480 Mbit/s bus carries), while one 1080p30 MJPEG endpoint can reserve up to 24.6 MB/s (3,072 B per 125 µs) — two cameras at that ceiling already exceed the budget. How much a camera reserves is decided by its bridge chip and descriptors; the build options are laid out in how a USB UVC MJPEG camera is built, from sensor to bridge.

The decode bill is the real limit

Every MJPEG frame must be decoded before it can be shown, and decode cost follows pixels, not bytes. One 1080p30 stream costs 2,073,600 × 30 = 62.2 Mpx/s; four streams cost 248.8 Mpx/s, call it 249. A fast application core running a libjpeg-turbo-class software decoder handles on the order of 100-200 Mpx/s, so plan on 1.5-2.5 cores for decode alone — before your application, the compositor and the window system take their share. On a board where some cores are small efficiency cores, decode alone can pin the whole CPU.

Isochronous transfers have no retry. The uvcvideo driver keeps a queue of buffers per camera; when the decode threads fall behind, the queue empties and the kernel discards whole frames. That is why the symptom appears only under load: lsusb still says 5000M, the link is lightly loaded, and the loss happens at the consumer end. Allocation churn makes it worse — a pipeline that allocates a fresh buffer per frame adds allocator and collector stalls to the same budget, the pattern described in Out of memory in a video loop: fix the per-frame image leak.

Display adds its own tax. Four live 1080p30 windows must be scaled, composited and redrawn in the same process that decodes. A measurement sink skips all of that, and the difference between the two is your diagnostic. If you are building the grid itself, the capture-side set-up is covered in Multi-camera video capture synchronization for four cameras.

Measure one camera at a time

The routine below isolates capture, decode and display so you fix the right stage. Keep the decoder in the pipeline — it is part of the cost you are measuring. Each UVC camera usually shows up as two /dev/video nodes; point the source at the one that offers MJPEG.

1. Re-run lsusb -t while all four cameras stream. Every line must still read 5000M; a camera that drops to 480M under load points at power or cabling, not decode.
2. Start one camera with a null sink and measure: gst-launch-1.0 v4l2src device=/dev/video0 ! image/jpeg,width=1920,height=1080,framerate=30/1 ! jpegdec ! fpsdisplaysink text-overlay=0 video-sink=fakesink sync=0 -v. The console prints current and average fps; expect both at 30.
3. Add a second identical pipeline in a second terminal and note both fps lines.
4. Repeat until four run. Record the source count where a stream first prints below 30 fps.
5. Switch the measurement to a real display: replace video-sink=fakesink with your on-screen sink and run four again. If fakesink held 30 fps and the display stutters, capture and decode are healthy and the compositor is the wall.
6. Watch per-core load in a system monitor while four streams run. Cores pinned at 100% during the drops mark the decode wall; low CPU during drops points back at the USB tree.

Three outcomes, three fixes. A single camera below 30 fps on its own means camera, cable or format. Frame rate that falls when you add source N means the decode budget ran out near N × 62.2 Mpx/s. Capture healthy on fakesink but the display stuttering means the render path.

Fixes in order of payoff

1. Render downscaled, capture full. End capture pipelines in your application rather than a live window, and build the on-screen grid from downscaled copies refreshed at 10-15 fps. You keep full-rate frames for processing and cut compositing cost by half or more.
2. Crop at the sensor. If the inspection only needs a region, a narrower sensor window cuts decode in proportion — the mechanics are in Camera ROI cropping at the source: reduce bandwidth before the host.
3. Drop resolution, keep the frame rate. Four 1280 × 720 streams cost 921,600 × 30 × 4 = 110.6 Mpx/s of decode — 2.25× less than 249 — and about 55 Mb/s of wire at 0.5 bpp. On most boards this alone moves the system back under the wall.
4. Use the hardware JPEG decoder if your SoC's multimedia stack exposes one. Software decode of all four streams is the fallback, not the plan.
5. Grow the queues and stop the churn. Raise the buffer count each source keeps in flight and reuse pre-allocated buffers instead of allocating per frame.
6. Spread the load across interfaces. Two cameras on the USB controller and two on their own GigE ports give each pair a separate budget; that is where a vision controller with more LAN ports than the on-board NIC earns its place.
7. Power the hub from its own supply and re-check 5000M after every change.

When it really is the wire

Most four-camera drop-outs are decode, but the wire and power failures have signatures of their own.

Hub power is the sneaky one: a bus-powered hub with four streaming cameras can renegotiate links mid-run, and the tree changes underneath you — the classic pattern in USB hub drops camera frame rate: budget the bandwidth first. The power-side budget sits in Multiple USB cameras on one host: bandwidth and power budget. Other host-level budgets — GigE machines, trigger timing, capture grids — are collected on the systems topic page.

Parts that keep the budget predictable

Every number above came off data sheets: frame sizes, frame rates, pixel formats. Cameras that print those values make the budget computable before anything is ordered, and for embedded builds the board-level class keeps cabling and mounting simple. Two USB3 modules that suit a multi-camera host:

The LS-BCM-0003 is a 3.1 MP USB3 board-level camera with a 1/1.8-inch global-shutter sensor and a 120 fps top rate. The MV-CB060-10UM-B is a bare-board 6 MP USB3 camera you build into your own housing. Both let you choose the frame format and rate deliberately instead of accepting defaults — exactly the input the bandwidth and decode arithmetic needs.

When four streams beat one embedded board, the answer is a host with more interfaces, not a faster hub. A controller such as the MV-VC3301H-128G66 pairs an Intel Core i3 with 8 GB of memory, 3-4 USB 3 ports and 5+ LAN ports, so two cameras can stay on USB while two move to their own GigE links: separate budgets instead of one shared queue.

To size the chain, filter the camera finder by interface, resolution and shutter. For a host with more GigE ports than a typical board carries, compare the vision controllers by LAN and USB 3 port counts.

Before you add the fifth camera

• Budget each stream from its frame size, not its resolution alone: 31 Mb/s of wire and 62.2 Mpx/s of decode for 1080p30 MJPEG at 0.5 bpp, scaling linearly with pixels and frame rate.
• Hold total decode below about 70% of your cores at peak; the remainder absorbs the OS, your application and the render path.
• Prefer one camera per root controller port where the board offers it; a hub is wiring convenience, and its uplink is the ceiling for everything behind it.
• Re-run lsusb -t after every hardware, cabling or power change; a silent 480M fallback voids every budget above.
• Re-measure after software changes too: a decoder update or a new compositor shifts the same numbers.

Work the budgets first, then make the hardware match them: the camera configurator steps through camera, lens and controller in one pass, and the machine-vision guides hub collects the companion guides on hubs, bandwidth and synchronization.