UVC camera MJPEG streaming on Linux for vision applications

A perception or robotics stack asks a USB camera for full-resolution video, the port cannot carry it uncompressed, and frames arrive torn or the frame rate halves. The answer is usually already on the camera's own format list: stream MJPEG and decode on the host. This guide gives you the arithmetic that decides when compression is required, the numbers to budget for decoding, the cases where MJPEG corrupts a measurement, and the point at which a machine-vision camera is the better purchase than a workaround. Everything here runs on a plain Linux host with the kernel's standard video stack.

What UVC and MJPEG actually are

UVC — the USB Video Class — is a device-class standard from the USB Implementers Forum, and the class documents sit at the USB-IF video class page. A camera that follows it enumerates on any host as a standard video device: Linux binds its built-in UVC driver, the device appears as /dev/video0 and its siblings, and no vendor SDK is involved. The V4L2 API your code talks to defines the interface, and querying it lists the pixel formats, resolutions and frame rates each device really supports.

MJPEG is one of the formats V4L2 will list: the camera compresses every frame as an independent JPEG and sends the frames as one continuous compressed stream. Each frame stands alone, so a lost frame costs one frame, not the rest of the sequence. Your code receives JPEG frames, decodes them back to RGB or grey, and runs its normal processing. That is the entire technique: trade host CPU for link bandwidth. It is also why a depth camera that cannot sustain raw 4K over the wire often sustains 4K MJPEG without saturating the link.

The Linux side is deliberately thin, with no vendor driver and no kernel rebuild. Two nearby guides cover that set-up: UVC cameras on embedded Linux and a UVC camera streaming over USB with no config file. This page is about the step those stop at: choosing the format the link can actually carry.

The bandwidth arithmetic that decides it

One comparison decides the format: the bytes per second a mode demands versus the bytes per second the port carries.

For the link side, use these figures. USB 3.0 advertises 5 Gbit/s of signalling; 8b/10b line coding leaves 4.0 Gbit/s of payload, and an isochronous video pipe sustains about 400-450 MB/s in practice, so plan on 400. USB 2.0 advertises 480 Mbit/s and delivers roughly 35-40 MB/s. Uncompressed UVC modes are usually YUY2 at 2 bytes per pixel or RGB at 3, so compute with the bytes per pixel your mode actually uses.

Common modes at 3 bytes per pixel, compressed at the 15:1 midpoint of the 10:1-20:1 range UVC cameras typically deliver:

The working rule: keep an uncompressed stream below about 2.5 Gbit/s — roughly 310 MB/s, 80 percent of the practical USB 3.0 ceiling — and switch to MJPEG or cut the frame rate above it. The margin is not padding; isochronous scheduling, other devices on the same controller and decode hiccups all take a bite. Compression is unavoidable at 4K30 whatever the pixel format: YUY2 at 2 bytes per pixel still demands 498 MB/s.

Worked example: a 4K RGB camera at 30 fps on USB 3.0

Inputs, each with its origin:

• 3840 × 2160 px RGB — the camera's 4K UHD mode, a standard format a UVC 4K sensor offers.
• 30 fps — the perception loop's target rate.
• 3 bytes per pixel — uncompressed RGB.
• 400 MB/s usable link — one USB 3.0 port at the practical figure above.
• 15:1 MJPEG compression — the midpoint of the typical 10:1-20:1 range.

Rule in words: the stream fits only if frame size × frame rate, converted to bits, stays under the usable link rate, and MJPEG divides that demand by the compression ratio.

Arithmetic:

1. Frame size: 3840 × 2160 px × 3 B/px = 24,883,200 B ≈ 24.9 MB per frame.
2. Uncompressed demand: 24,883,200 B × 30 fps = 746,496,000 B/s ≈ 746 MB/s, and 746 MB/s × 8 = 5.97 Gbit/s.
3. Verdict on the link: 5.97 Gbit/s exceeds even the 4.0 Gbit/s raw payload and is about 1.9 times the 400 MB/s practical ceiling. The camera cannot hold the mode: it negotiates down to about 16 fps (400 MB/s ÷ 24.9 MB) or drops frames.
4. MJPEG demand: 746 MB/s ÷ 15 = 49.8 MB/s ≈ 0.40 Gbit/s, an eight-fold margin under the same ceiling.
5. Decode budget: 3840 × 2160 px × 30 fps ≈ 249 Mpixels/s; fast JPEG decoders manage on the order of 100-150 Mpixels/s per modern x86 core, so 2-3 cores cover it.

Decision: at 4K30 RGB the uncompressed rate computes past the link, so stream MJPEG at 15:1 and budget about 50 MB/s of link and 2-3 decode cores, carrying any depth or measurement data on a separate uncompressed stream. As a general threshold, when width × height × bytes × fps × 8 lands above about 2.5 Gbit/s, compression or a lower frame rate stops being optional; below it, uncompressed is the simpler pipeline and deserves the preference.

Decoding the stream on the host

V4L2 hands each MJPEG buffer over as one complete JPEG frame, so decoding is ordinary JPEG work: a libjpeg-turbo-class decoder, or the capture layer of a vision library such as OpenCV when it opens the device for you. Decode off the capture thread, and if decoding falls behind, keep only the newest frame and discard the backlog. A queue that grows unseen is the classic MJPEG failure: the fps counter still looks healthy while latency climbs.

On ARM boards, check the SoC's multimedia stack for a hardware JPEG or mem-to-mem decoder before budgeting more CPU; many boards carry one. On latency: the camera's JPEG encode plus your decode typically add 20-50 ms end to end at these frame sizes. Perception does not care; a tight visual-servo loop might, and that loop is a reason to move to the machine-vision route at the end of this page rather than push compression harder.

Where MJPEG fits and where it corrupts the measurement

Compressed streaming fits perception work: scene understanding, robot guidance, presence and orientation checks, and barcode reading — decoding barcodes straight from a compressed video stream is routine — plus any multi-camera monitoring job where the bandwidth budget of a four-camera host matters more than pixel-perfect fidelity.

It does not fit measurement. JPEG quantization softens exactly the low-contrast edges a dimension or a subtle defect lives on, and most UVC cameras compress with 4:2:0 chroma subsampling, halving colour resolution in both axes: a 3840 × 2160 MJPEG frame carries colour detail closer to 1920 × 1080. Depth is worse still, because cameras that present depth through UVC pack it into 8-bit luma or vendor formats, and quantization plus subsampling corrupts the depth values themselves. Keep depth, colour matching and dimensional streams uncompressed; use MJPEG for the RGB side only.

One more limit before you commit: UVC exposes a short list of exposure controls, and auto-exposure behaviour differs from device to device. Fixed manual exposure, precise gain and a hardware trigger are machine-vision camera territory, not a format workaround.

Failure modes that look like bandwidth problems

The other three failures have distinct signatures:

• Works alone, fails when the second camera opens. Add the per-camera MB/s and remember two cameras on one controller share its 400 MB/s; the fix is controller layout as much as compression. The arithmetic is in budgeting multiple USB cameras on one host and the depth-and-colour USB bandwidth check.
• Frame rate halves only at high resolution. The camera has no MJPEG mode there, so the pipeline fell back to uncompressed. Re-read the enumerated format list and let the table above tell you which modes survive uncompressed; the FPS-drop checklist covers the wider diagnosis.
• Freezes under sustained load. That is a host controller or decode backlog, not the camera; the USB3 stream bandwidth budget walkthrough applies to RGB devices the same way.

When a machine-vision USB3 camera is the cleaner answer

If the job is inspection — dimensions, surface defects, colour, anything you will measure twice — skip the workaround. Machine-vision cameras on USB3 speak the USB3 Vision protocol rather than UVC: formats and exposure are negotiated explicitly, frames arrive uncompressed, and there is no decode stage to budget. That removes the shared-bandwidth roulette and the chroma-subsampling risk in one move, at the cost of working through the vendor SDK instead of a generic video device.

The MV-CS200-10UM is the fine-detail route: 20 MP, mono, USB3.0, C-mount, rolling shutter — a strong fit for stationary scenes where resolution is the constraint. The MV-CH120-11UC is the motion route: 12 MP colour, global shutter, high-speed USB3, C-mount.

For scale: a 20 MP mono 8-bit frame is 20 MB, so 15 fps costs 300 MB/s — inside USB 3.0 with a real margin, at a resolution where compressed previews stop being attractive. If the run between camera and host outgrows a USB cable (a few metres in practice), the same cameras exist on GigE: the area scan cameras category filters by interface, and the camera finder by sensor, resolution and mount. For robot-perception tasks — depth, volume, pick and place — the 3D sensors category is the shorter path, and cable choice on the longer runs is its own subject in the cables category.

Checks before you commit

• Enumerate formats on the target host. The modes a UVC camera advertises on your machine are the contract; box labels and other people's hosts are not.
• Run the arithmetic per mode. Keep uncompressed × 8 under about 2.5 Gbit/s or switch to MJPEG, and sum every camera's MB/s before adding the next one.
• Measure decode CPU at the target fps. Leave at least 40 percent of the cores for the vision code, or the backlog problem above will find you.
• Judge a captured frame, not the fps counter. Look for 4:2:0 colour softening and blocking at edges before trusting colour or fine defects.
• Keep depth and metrology uncompressed. MJPEG quantization corrupts exactly the values you are measuring.
• Plan the cable. USB 3.0 runs a few metres; beyond that, move to the GigE side of the catalogue.

If the arithmetic says MJPEG, you now have the budget: link MB/s, decode cores, latency. If it says the modes you need do not exist on the camera you have, or the job turns out to be measurement, the camera configurator steps through sensor, resolution and interface in the right order. The system side around this — host bandwidth, device selection, frame sync — is gathered on the systems topic page, and you can reach the team through the contact page for a second opinion on a bill of materials.