USB camera timeout at high resolution: fix bandwidth and pacing

Two errors that show up together — a select() timeout from the capture library and 'dropping payload out of sync' in the kernel log — have one physical cause: frames are being delivered late or in pieces, and the driver gives up waiting. It appears at 4K and above because that is where uncompressed YUYV runs into the ceiling of a USB 3 controller. This guide is for engineers streaming from USB video class (UVC) cameras on Linux hosts. You will compute the true data rate of every mode your camera lists, compare it with what the link sustains, and apply the fix that fits: pacing, binning, compression, a controller split, or an industrial camera.

What the two errors actually mean

A UVC camera does not push frames as files; it streams isochronous payloads, each with a small header, into slots the host reserves in every 125 µs service interval. Two things break when the demand exceeds what the controller can grant:

• select() timeout — your capture library (OpenCV's V4L2 backend prints exactly this warning) waited on the device for a complete frame and none arrived inside the wait window.
• dropping payload out of sync — the kernel's uvcvideo driver received a payload whose header no longer lines up with the stream position it expects, usually after a missed or late slot, and discards data until headers line up again.

The messages are downstream symptoms. The chain is: slots missed, payloads dropped, stream resyncs, a wait expires. Fixing delivery fixes both messages; see the resync behaviour in camera image stream out of sync: recover the grab and check the link.

The link-rate arithmetic

Everything follows from one multiplication. YUYV packs 16 bits per pixel, so a frame is width × height × 2 bytes. Multiply by frames per second and by the camera count, then compare with the budget.

A USB 3.0 port runs at 5 Gbps. After 8b/10b encoding that is 500 MB/s of raw capacity, and packet overhead leaves roughly 350-400 MB/s that one host controller sustains for a single isochronous video stream. Plan against 350 MB/s and treat anything above 85% as over budget:

Here is a typical 48 MP UVC module's YUYV mode list, with each mode's rate at its maximum frame rate:

The verdicts reproduce the classic failure pattern: modes up to 3 MP run for hours, 4K-class modes fail intermittently, and the full-resolution mode survives only as the only stream on the bus.

Worked example: two cameras at full resolution

Say the target is synchronous capture from two cameras, 5 fps where available and 3 fps at 8000×6000, all YUYV, both on one host controller.

• One camera at 8000×6000, 3 fps: 96 MB × 3 = 288 MB/s, or 82% of the 350 MB/s budget on a dedicated controller — borderline on its own. A null capture on such a module typically delivers under the advertised rate, about 2.7 fps = 259 MB/s.
• Two such cameras on one controller: 288 × 2 = 576 MB/s, which is 165% of the budget. That fails by arithmetic; no sleep() loop, driver flag or OS setting changes bytes per frame.
• The same pair at 3840×2160, 5 fps: 16.6 MB × 5 = 83 MB/s each, 166 MB/s total, 47% — stable.
• At 4000×3000, 5 fps: 24 MB × 5 = 120 MB/s each, 240 MB/s total, 69% — stable with margin.

Decision: full resolution is a one-camera, one-controller mode at 3 fps or less. If you need two cameras at useful rates, bin or compress — the modes that fit are the ones under 85%.

Causes in order of probability

1. Total YUYV demand over one controller's budget — two cameras sharing a bus, or one camera at a 4K+ mode near its maximum fps. Test: run the table arithmetic for your exact modes; anything above 85% of 350 MB/s explains intermittent failure.
2. Pacing done in software — the camera runs flat out while your loop sleeps between grabs; buffers fill, payloads drop, the stream resyncs. Test: set the frame rate at the device and watch the kernel log; if the payload warnings stop, this was it.
3. Both cameras behind one controller or a hub — ports on the same root hub share one budget, and a hub funnels both streams into a single upstream link. Test: lsusb -t shows the tree and each camera's bus.
4. A USB 2.0 link somewhere in the path — a USB 2 port or cable caps you at about 35-40 MB/s practical; 1080p YUYV at 30 fps needs 1920 × 1080 × 2 × 30 = 124 MB/s and cannot run at all. Test: lsusb -t shows 480M instead of 5000M.
5. Marginal cable or connector — SuperSpeed lanes use separate wire pairs, so a worn consumer cable drops the 5 Gbps link or corrupts payloads. Test: swap in a short certified cable and re-run the soak.
6. Firmware bandwidth quirks — a camera whose firmware computes its isochronous request poorly, or trades rate for stability through an on-board ISP, is the last suspect. Test: lowering fps at the same resolution stabilizes everything else first.

Fix the pacing before anything else

Set the frame rate at the sensor, in a negotiated mode, and let the camera emit — do not sleep() in your loop while the camera runs at maximum. Explicit pacing lowers both the delivered rate and the bandwidth the UVC stack reserves for the mode:

import cv2  # OpenCV 4.x

cap = cv2.VideoCapture(0, cv2.CAP_V4L2)          # int device index 0
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter_fourcc(*'YUYV'))
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 4000)          # px, from the mode list
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 3000)         # px
cap.set(cv2.CAP_PROP_FPS, 5)                     # fps, must be a listed mode

Then grab both cameras first and retrieve afterwards, so the two frames stay close in time — one wait call per frame, as laid out in frame didn't arrive within timeout: one wait call per frame. If the rate still sags mid-run, the causes overlap with camera FPS drops suddenly: exposure, USB link, bandwidth, and two full-rate streams may also need the host-side fix in two USB3 cameras at full resolution stream: raise the host USB buffer.

Cut the rate: binning and MJPEG

Two moves shrink bytes per frame without changing the optics.

Sensor binning. 4-in-1 binning on a 48 MP sensor outputs 12 MP (4000×3000) with the full field of view: 24 MB per frame. At 5 fps that is 120 MB/s per camera, 240 MB/s for two, or 69% of one controller — which is why binned 12 MP streams run steady while the full-resolution mode above them does not. Binning is lossless; prefer it when the inspection survives 12 MP.

MJPEG. Compression typically moves 4-8× less data than YUYV at usable quality on industrial scenes; measure on your own scene. At 8000×6000, 96 MB becomes roughly 12-24 MB per frame, so 3 fps lands between 36 and 72 MB/s per camera — two cameras fit at 72-144 MB/s, or 21-41% of the budget. Set the format with CAP_PROP_FOURCC to MJPG. MJPEG is lossy and usually subsamples colour, so keep it for location, code reading and defect detection, not sub-pixel gauging.

Split the controllers and check the link

lsusb -t prints the USB tree: each device line shows the negotiated speed (5000M or 480M) and each bus is typically one controller. Put each high-rate camera on a port wired to a different controller, and never through a hub — that funnels both streams into one upstream link, the exact mistake worked through in USB hub drops camera frame rate: budget the bandwidth first. The per-camera power and bandwidth accounting for a shared host is the same exercise as in multiple USB cameras on one host: bandwidth and power budget.

For the physical layer, use a cable built for the job: SuperSpeed pairs are separate from the USB 2 pairs, and flexing or a loose connector shows up as intermittent 480M fallback and payload corruption. In a moving machine, use a drag-chain rated cable such as the 3 m USB3.0 camera cable LS-CBL-0219. Passive copper USB 3 is certified to about 3 m; longer runs need active or optical cables, or a different interface.

When firmware really is the cause

If every mode now sits comfortably inside the arithmetic and the stream still drops payloads, the camera's firmware is the remaining suspect. Some modules compute their isochronous bandwidth request badly; others trade rate for stability by routing output through an on-board ISP. The second behaviour is arithmetically visible: a module that caps full-resolution output at 2 fps turns 288 MB/s into 96 × 2 = 192 MB/s, or 55% of budget, and the stream becomes stable for a reason you can compute.

Update only through the camera maker's own flashing tool and package, and ask for the rollback image in the same request before you flash anything. Validate on a spare unit, and flash production units on the bench before they go into a sealed assembly — many flashing tools run only on Windows, and a sealed enclosure turns an update into a disassembly job.

Prove the fix: a throughput and soak test

Measure delivered rate, not requested rate:

import time  # continues from the cap object above

frames, t0 = 0, time.monotonic()            # int frames, float seconds
while frames < 300:
    if cap.grab():
        frames += 1
elapsed = time.monotonic() - t0             # s
print(frames / elapsed, 'fps delivered')    # want >= 0.95 x the fps you set

At 4000×3000 YUYV with 5 fps requested, a pass looks like 300 frames in 61 s: 4.9 fps delivered, and 24 MB × 4.9 ≈ 118 MB/s, which is 34% of the budget. A fail looks like 2.5 fps or payload warnings in dmesg — back to the cause list. Then soak: run the exact production loop for at least 30 minutes with dmesg open in a second window, because streams at the edge of the budget fail minutes in, not on frame one. When you also need to reason about latency, camera buffer timestamp latency: interpret frame timing shows how to read the timing you already capture.

When to move to an industrial camera

If the requirement is uncompressed high resolution at sustained rates, deterministic multi-camera delivery, or connectors that cannot work loose, that is a machine-vision camera job rather than a webcam tuning job.

USB3 Vision cameras use bulk transfers, which sustain about 400 MB/s in practice, and set frame rate through GenICam properties instead of hoping a UVC mode list contains what you need. Sensors in Mono8 or raw Bayer use one byte per pixel, halving the rate against YUYV: 12 MP at 30 fps is 12 × 30 × 1 = 360 MB/s, or 90% of a bulk link — which is why 12 MP is roughly where 30 fps USB3 bodies top out. The LS-CMR-0166 streams 12 MP at 30 fps over USB3. Industrial USB3 bodies in this class:

When runs exceed 3 m or the host sits away from the machine, GigE Vision is the fit: 1 Gbps is 125 MB/s of line rate and about 110 MB/s of usable payload. At one byte per pixel, 12 MP at 9 fps is 108 MB/s — the arithmetic behind why 12 MP GigE bodies specify 9 fps. The 12 MP GigE camera LS-CMR-0053 does exactly that:

What the standard guarantees on such a link — discovery, control, and streaming contracts — is summarized in camera compatibility with vision software: what GigE Vision guarantees, and the rate ceiling itself in GigE Vision for a high speed camera: how much frame rate fits?. For rate beyond that, 10GigE and CoaXPress bodies multiply the budget again. The same cameras exist without housings as board-level cameras for OEM builds.

Decision: UVC when the arithmetic leaves headroom; USB3 Vision for uncompressed 12 MP+ within 3 m of the host; GigE Vision for distance and integration; 10GigE or CoaXPress when the rate math fails again. Browse area scan cameras through the camera finder or configure a camera.

Prevention checklist

Before you order or integrate:

• Run the rate table for every planned mode: width × height × 2 bytes × fps, times the camera count, at or under 0.85 × 350 MB/s per controller.
• Prefer Mono8 or raw Bayer modes: half the bytes of YUYV.
• Specify locking, drag-chain rated USB3 cables of 3 m or less, or plan GigE for longer runs.

During commissioning:

• Set fps at the device in code; delete sleep() pacing.
• Confirm with lsusb -t that every high-rate camera shows 5000M and sits on its own controller, with no hub in the path.
• Soak for at least 30 minutes while watching dmesg; edge-of-budget failures surface minutes in.

In operation:

• Re-run the throughput test after any camera, cable or host change.
• Keep firmware updates to the maker's own tool, rollback image in hand.

If your mode list and the budget never meet, the fix is a different camera, not more tuning. Pick a body whose modes sit under 85% of the link, or talk the application through with us via the contact page.