System designTroubleshooting

Two USB3 cameras at full resolution stream: raise the host USB buffer

Two USB3 cameras stall at full resolution when Linux caps usbfs transfer memory at 16 MB. Size the buffer, split the controllers, and stream both cameras.

LumaSpec Vision10 min read Share on LinkedIn

On this page 11 sections
  1. 01The symptom pattern
  2. 02Run the bandwidth arithmetic first
  3. 03Cause 1: the kernel's usbfs memory cap
  4. 04Raise the host buffer in four steps
  5. 05Why the camera's throughput limit does not fix it
  6. 06Cause 2: both cameras behind one controller
  7. 07Cause 3: the pixel format triples the bytes
  8. 08Prove the fix before the station ships
  9. 09Build the station so it cannot regress
  10. 10When USB3 is the wrong link
  11. 11Questions

Two USB3 cameras on one Linux host, both asked for their native resolution in color: one stream delivers frames and the second never calls back, or freezes after the first few. Halve the resolution, switch the pixel format to Mono8, or unplug one camera, and the survivor runs clean. That pattern indicts the host, not the cameras. This guide walks the failure in order of probability — the kernel's usbfs memory cap first, then controller sharing, then pixel format — with the bandwidth arithmetic, the exact commands, and the checks that prove the fix. It applies to any pair of USB3 Vision cameras streaming full resolution through libusb on Linux.

The symptom pattern

Each camera enumerates, and each streams alone at full resolution and full color. Start both together and three behaviors appear: one camera's frame callback is never entered at all; or it fires a few times and the image stops updating; or frames arrive flagged incomplete while frame IDs skip numbers. A bus meter reading of about 190 MB/s per camera running solo says the cameras, cables and ports are healthy. The silent callback is the decisive clue: the second camera's transfers are rejected before they begin, which is a host-side allocation failure rather than a busy link. When frames arrive late or torn instead of never, read the arithmetic below before touching settings. A single camera hitting the same wall usually shows up as a low frame rate rather than a dead stream; USB3 camera low frame rate and latency: fix usbfs memory and exposure covers that variant.

Run the bandwidth arithmetic first

frame_bytes = width × height × bytes_per_pixel stream_MBps = frame_bytes × fps / 1,000,000 pair_MBps = cameras × stream_MBps

Take a 5 MP color pair at 2592 × 1944 in Bgr8, which is 3 bytes per pixel, running 12.5 fps — that frame rate is what the observed 190 MB/s buys at this frame size:

  • 2592 × 1944 = 5,038,848 pixels
  • × 3 B per pixel = 15,116,544 B ≈ 15.1 MB per frame
  • × 12.5 fps = 188,956,800 B/s ≈ 189 MB/s per camera
  • × 2 cameras = 378 MB/s ≈ 3.0 Gbit/s

A USB 3.0 line clocks 5 Gbit/s; after 8b/10b encoding and protocol overhead, one controller in practice sustains roughly 400 MB/s. The pair fits with about 22 MB/s of headroom — it streams only while the host returns receive buffers back to back, and any gap in that pipeline backs the whole pair up at once. That is why the same configuration sometimes works and sometimes dies.

Cause 1: the kernel's usbfs memory cap

USB3 Vision cameras on Linux stream through libusb, and libusb submits every queued frame buffer to the kernel's usbfs layer as a transfer that consumes kernel memory. All of it is charged against one system-wide cap: usbcore's usbfs_memory_mb, which boots at 16 MB. Both cameras draw from that single cap. With four frames queued per camera in Bgr8, demand is 2 × 4 × 15.1 MB ≈ 121 MB against a 16 MB budget, and the submissions that do not fit fail with an out-of-memory error. That is why the first camera streams while the second camera's callback is never entered: its transfers are refused at the host before the camera sends a pixel. Raising this cap is what fixed this exact case. The companion host settings — the file-descriptor limits that bite at the same time — are collected in USB3 Vision cameras on Linux: usbfs and file limits that need tuning.

Read the live value before changing anything:

cat /sys/module/usbcore/parameters/usbfs_memory_mb
16

Raise the host buffer in four steps

  1. Confirm the cap on the imaging host with the command above; a fresh install prints the boot default.
  2. Raise it now, without rebooting:
echo 1000 | sudo tee /sys/module/usbcore/parameters/usbfs_memory_mb
  1. Make it survive reboots: append usbcore.usbfs_memory_mb=1000 to the kernel command line — in GRUB, the GRUB_CMDLINE_LINUX_DEFAULT entry — rebuild the bootloader configuration (update-grub on Ubuntu) and reboot. The kernel's command-line parameter documentation defines the module.parameter=value form used here.
  2. Restart both streams at full resolution and the native pixel format; the second camera's callback should fire within the first second and keep firing.

Size the number from your own queue: cameras × queued frames × frame bytes. The 5 MP Bgr8 pair above needs about 121 MB; a 12 MP color pair at 4000 × 3000 × 3 = 36 MB per frame needs 2 × 4 × 36 = 288 MB on the same four-frame queue. 1000 MB covers both with headroom for a larger sensor later.

Why the camera's throughput limit does not fix it

DeviceLinkThroughputLimit is a GenICam feature that paces the camera's own transmitter, in bytes per second. It is the right lever when cameras genuinely oversubscribe a shared pipe, and the wrong one here. In the case behind this pattern the limit sat at its floor of 98,375,000 — about 98 MB/s — and one camera streamed at that rate while the pair in Bgr8 still failed, on a machine that carried both streams in Mono8. Pacing a transmitter cannot help a receiver that refuses transfers for lack of memory.

Commit to the split this way: when each camera alone holds its commanded rate and the pair total sits under the controller's roughly 400 MB/s, stop trimming the camera and fix the host buffer. When the pair total exceeds it, trim each camera until the sum of the limits lands under 350 MB/s, or give the cameras separate controllers. For triggered bursts rather than continuous streams, the camera's internal FIFO can ride out a link slower than the sensor: camera burst capture faster than link bandwidth: use the buffer.

Cause 2: both cameras behind one controller

Every port on the same xHCI controller shares that controller's roughly 400 MB/s. Run lsusb -t and read the tree: two cameras under one root hub share one pipe. At 378 MB/s total the pair fits with about 22 MB/s spare — legal, but unforgiving, and a keyboard, disk or capture device on the same controller eats the margin first.

The clean structure is one camera per controller. Use a second onboard controller, or move the pair onto an add-in card such as the MV-GU2204, a quad USB3.0 adaptor with 4 ports. Re-run lsusb -t afterwards and confirm the two cameras no longer sit under the same root hub. Budget power while you are in there: a USB 3 port supplies 900 mA at 5 V, which is 4.5 W, and cameras that draw more need their own supply — multiple USB cameras on one host: bandwidth and power budget works through that arithmetic.

Frame grabbersHikrobot MV-GU2204 MV-GU2200 Series Quad USB3.0 AdaptorMV-GU2204
Interface
USB3
Ports
4 ports
$469.00View partBuild a system with it

Cause 3: the pixel format triples the bytes

Bgr8 is demosaiced inside the camera: three bytes per pixel, computed from one-byte sensor samples. BayerRG8 ships those raw samples at one byte per pixel; demosaic on the host and the color content is identical. In the case behind this pattern, switching the pair from Bgr8 to Mono8 made both streams run — one third of the bytes:

Pixel format at 2592 × 1944, 12.5 fps Per frame Per camera Pair
Mono8 5.0 MB 63 MB/s 126 MB/s
BayerRG8 5.0 MB 63 MB/s 126 MB/s
Bgr8 15.1 MB 189 MB/s 378 MB/s

Halving width and height 'fixed' it for the same reason: half the width and half the height is a quarter of the pixels and a quarter of the bytes.

Prove the fix before the station ships

  • Both callbacks fire within the first second and keep firing through a soak run of several minutes at operating temperature.
  • Frame IDs advance on both cameras with no gaps; a gap is a dropped frame, not a late one.
  • The frame status counters report zero incomplete frames.
  • Each camera holds its commanded frame rate with processing decoupled from acquisition, so host CPU load cannot starve the queue drain.

Build the station so it cannot regress

Cables3 m USB3.0 camera cable, high-flex (drag chain) (LS-CBL-0219)LS-CBL-0219$62.00View partBuild a system with it
  • Bake usbcore.usbfs_memory_mb=1000 into the machine image so a reinstall cannot bring the kernel default back.
  • A pair built from any of these USB3 area scan cameras takes the same host-side fix:
Area scan camerasHikrobot MV-CS200-10UM 20 MP Area Scan Camera, USB3.0, IMX183, 2nd Gen., MonoMV-CS200-10UM
Sensor format
1"
Lens mount
C-mount
$759.00View part
Area scan camerasHikrobot MV-CH120-11UC 12 MP Area Scan Camera, USB3, IMX304, High-speed version, ColorMV-CH120-11UC
Sensor format
1.1"
Lens mount
C-mount
$2,339.00 · quotedView and quote
Area scan cameras12 MP USB3 area scan camera, IMX226, 1/1.7" rolling shutter, 30 fps, 4000 × 3000 resolution (LS-CMR-0166)LS-CMR-0166
Sensor format
1/1.7"
Interface
USB3
$560.00View part

Run the same arithmetic before scaling past two cameras. At 4000 × 3000 in Bgr8 a frame is 36 MB, so 30 fps of it is 1080 MB/s for one camera — that belongs on 10GigE or CoaXPress. The one-byte formats from the same sensor need 360 MB/s, which one USB3 controller carries. Past roughly 400 MB/s per controller, split controllers or move up, and budget the shared link first: multi-camera GigE bandwidth budget: divide the link first shows the method. Plain GigE will not absorb even one of these color streams — a 1 Gbit/s link delivers about 110 MB/s usable, against 189 MB/s here.

Next step: size the whole two-camera station, sensor to cable, in the system configurator. If the pair still stalls with the buffer at 1000 MB and the controllers split, bring the lsusb -t tree and the frame-status log to the contact page and we will work it with you.

Questions

Is DeviceLinkThroughputLimit set in bits or bytes per second?

Bytes per second; that is the GenICam SFNC unit for this feature. A floor of 98,375,000 is about 98 MB/s, not 787 Mbit/s. Use it to pace cameras sharing one controller: keep the sum of all limits on that controller under about 350 MB/s so bursts and protocol overhead have room. It cannot repair a host that rejects transfers for lack of usbfs memory.

What value should usbfs_memory_mb be set to for two cameras?

The boot default is 16 MB and it is one shared, system-wide cap. Budget cameras × queued frames × frame bytes, then add headroom: two 5 MP Bgr8 cameras with four frames queued each need about 121 MB, and a 12 MP color pair on the same queue needs 288 MB, so 1000 MB covers both with room to grow. Set 0 only on a host dedicated to imaging, since 0 removes the limit rather than raising a budget.

Does a USB 3 hub help two full-resolution cameras?

No. A hub multiplies ports on one upstream link, so both cameras still share a single pipe of roughly 400 MB/s, plus the hub's own latency and power limits. Two 189 MB/s color streams through one hub sit at the ceiling with nothing left for retries. A Mono8 pair at 126 MB/s total is comfortable through a hub. To add bandwidth, add a controller with an add-in card, not ports.

How do I tell a host-side stall from a failing camera or cable?

Stream each camera alone at full resolution on the same port: a healthy camera holds its commanded rate with zero incomplete frames — about 189 MB/s in this example. Then swap the two cameras between ports. If the failure follows the pairing and never an individual camera, the host is the culprit; if it follows one camera or one port, replace that cable or camera and retest.

Do I lose color quality by streaming BayerRG8 instead of Bgr8?

No. BayerRG8 carries the sensor's raw samples at one byte per pixel, and Bgr8 is computed from exactly those samples by demosaicing inside the camera. Demosaic on the host and you interpolate the same data with the same outcome, at one third of the link bytes — and you keep control of the algorithm instead of accepting the camera's fixed one.

Why is the second camera's callback never called at all?

Because its transfers are rejected at submission, not delayed. libusb queues each frame buffer as a usbfs transfer charged against usbcore's usbfs_memory_mb; once the first camera's queue consumes the cap, the second camera's submissions fail immediately with an out-of-memory error, so your callback is never entered. Raising the cap lets both queues live side by side on the host.