Stereo camera module experiments: getting synchronized image pairs

A dual-lens USB camera module looks like a ready-made stereo rig: two sensors, two molded lenses, a fixed baseline and one cable that enumerates as a standard UVC video device on any host. The open questions are what that single USB 2.0 link delivers per view, whether the two frames are truly simultaneous, and what the rigid housing does to your calibration workload. What follows sizes the bandwidth with the arithmetic shown, explains how side-by-side and interleaved pairs arrive, separates true synchronization from two free-running streams, and walks through the one-time calibration the fixed baseline earns you. It closes with the thresholds where two triggered board-level cameras replace the module. Every check here runs from your desk with a frame counter and a checkerboard target.

What a dual-sensor USB module hands you

Two sensor boards sit behind two lenses on one PCB, and the module advertises itself as a standard USB video class (UVC) device, so it opens with the capture stack you already run on Windows, Linux, macOS or Android. Per-side resolutions of 720p, 960p and 1080p are the common offerings. Three properties matter for stereo work:

• Fixed baseline. The lens centers are machined into one housing, so the rotation and translation between the two views are constants. Calibrate once and reuse the result for the life of the module.
• Shared shutter control. Modules sold as synchronized pairs drive both sensors from one shutter signal. That is what makes the two views truly simultaneous.
• One link, two views. Both sensors share the module's single USB 2.0 connection. Every megabyte per second the left view spends is a megabyte the right view cannot use.

That third property sizes the whole experiment, so it comes first.

The USB 2.0 budget, in megabytes per second

A USB 2.0 high-speed link runs at 480 Mbit/s on paper, but a UVC camera streams isochronously, and isochronous traffic is capped by packet size and timing: one 1024-byte packet per 125 µs microframe, 8000 microframes per second. That works out to 1024 x 8000 = 8,192,000 bytes per second, so budget about 8 MB/s for the module's stream — the ceiling controller silicon implements without high-bandwidth endpoints. Now compare demand. A raw 1080p YUYV frame is 1920 x 1080 x 2 bytes = 4.15 MB, so one uncompressed view at 30 fps needs 124 MB/s and two need 249 MB/s. The link carries neither, which is why these modules deliver MJPEG or H.264 rather than raw frames, and why the frame-rate question is answered in compressed kilobytes.

Sizing the frame rate: two 1080p MJPEG streams on one USB 2.0 link

The inputs, with their sources:

• Views: 2 — left and right, each 1920 x 1080 px, the 2 MP-per-side class these modules ship in.
• Raw frame size (computed): 1920 px x 1080 px x 2 B/px (YUYV 4:2:2) = 4.147 MB per view.
• Compressed frame size (assumed): about 250 KB per MJPEG frame at moderate quality, a roughly 16:1 cut against raw; the quality setting moves this up or down.
• Link budget: about 8 MB/s isochronous on USB 2.0, from the packet math in the previous section.

The rule in words: divide the link budget by the total compressed bytes the link must move each second; the quotient is the frame rate each view receives.

• Demand at 30 fps per view: 2 views x 0.25 MB x 30 fps = 15 MB/s, nearly double the 8 MB/s budget.
• Delivered rate: 8 MB/s ÷ (2 views x 0.25 MB) = 16 fps per channel, and with UVC payload headers on top, call it about 15 fps per view.
• Sanity check, raw: 8 MB/s ÷ 4.147 MB = 1.9 fps per view. Raw capture is not a USB 2.0 option at all.
• Sanity check, one view: 8 MB/s ÷ 0.25 MB = 32 fps. A single 1080p MJPEG stream just clears 30 fps — splitting the link across two views is what halves it.
• Escape hatch at 720p: 1280 x 720 x 2 B = 1.84 MB raw, about 110 KB as MJPEG at the same ratio, so 8 MB/s ÷ (2 x 0.110 MB) = 36 fps per channel — 30 fps per view with margin.

The decision: if the experiment tolerates roughly 15 fps per view at 1080p — bench alignment, calibration capture, static-scene work — the module on one USB 2.0 link is fine. If it needs 30 fps per view at 1080p, that link cannot carry it: step the device down to 720p, or move to two cameras on separate USB 3 or GigE links. Tighten or loosen MJPEG quality and every fps figure above scales inversely with the bytes per frame you actually measure, so read one captured frame's size before trusting the 15 fps number.

How the pair arrives: side-by-side, interleaved or two devices

Dual-lens UVC modules ship three delivery formats, and the format decides how you unpack frames:

• Side-by-side, one frame. The device advertises a double-width mode — for two 720p-class sensors, a 2560 x 720 frame. Left and right ride inside one image; split it down the middle and each half is a full view. This is the common format for synchronized stereo and VR-style modules.
• Interleaved rows. The two views alternate on row groups of one frame. Same synchronization guarantee, with a row-group de-interleave instead of a column split.
• Two independent devices. Some boards enumerate as two cameras. Each stream then free-runs, so the pair is no longer synchronized by construction, and you need triggering or timestamp matching; the photogrammetry sync and timestamp guide covers both for board cameras, and the two-USB-camera trigger recipe shows the wiring.

The check is quick: list the frame sizes the device advertises in your capture stack, and a width about double one sensor's active area means side-by-side packing. Then read frames at full rate for a minute and count — that number, not a spec-sheet claim, is your delivered fps.

What synchronization buys you, and the spectrum catch

A synced-shutter module exposes both sensors at the same instant, so the pair is genuine. Two properties still decide whether it works as a depth rig.

Matching spectra. Dense stereo matching assumes both cameras see the scene photometrically alike. Two identical visible sensors — or two mono sensors behind identical IR pass filters — satisfy that. A visible-plus-IR pair does not: surface brightness flips between spectra, and per-pixel block matchers return sparse or empty disparity. The mixed pair still earns its keep for synchronized day-and-night capture, bi-spectral experiments and sparse feature stereo, where you match discrete keypoints rather than every pixel. Test on one textured target: if the fraction of valid disparities collapses against a same-spectrum pair, you are looking at photometric mismatch, not a calibration bug. Fitting IR illumination to a board camera is a known recipe — the IR LED and I2C proximity sensor walkthrough covers the hardware.

Shutter type under motion. Rolling-shutter sensors expose rows sequentially, so with either camera or scene moving, the two views skew differently and matching degrades. Static bench work is unaffected. For moving targets, global shutter on both sides is the requirement, not a nicety.

Calibrating the fixed baseline once

The rigid housing turns calibration into a one-time job. Capture 20-30 pairs of a planar checker target at varied angles and positions covering the full frame, run the standard two-camera calibration — intrinsics per side plus the right-to-left rotation and translation — and log the reprojection error; a rigid module typically lands well under one pixel. Save the result against the module's serial number. From then on, rectified pairs are a fixed remap and any standard stereo pipeline takes over; the fixed-intrinsics calibration walkthrough shows the extrinsics-only shortcut, and feeding rectified pairs to a host-side matcher covers the pipeline after calibration.

Two numbers make the depth metric. Measure the baseline between lens centers with calipers — module baselines are tens of millimeters — then apply:

Here f is focal length in pixels, B the baseline and d the disparity in pixels. The error term grows with the square of range: with B = 50 mm, f ≈ 1000 px and a one-pixel disparity error, the depth error at 2 m is (2000 mm)² ÷ (1000 px x 50 mm) ≈ 80 mm. A module baseline is therefore a near-field tool — solid within a meter or two, coarse beyond — which is why longer-baseline experiments move to separate cameras. Re-run calibration after any drop or lens rotation; one reprojection check tells you whether the old result still holds.

When two triggered board cameras beat one module

Four thresholds move you off the module:

• 30 fps per view at 1080p or faster — one USB 2.0 link caps near 15 fps per view, as worked out above.
• Raw or lightly compressed frames for photogrammetry-grade accuracy — raw 1080p needs about 124 MB/s per view, and only a USB 3 or GigE link carries that.
• A longer baseline — separate cameras let you set tens of centimeters, cutting depth error in proportion to B.
• Identical global-shutter sensors — you choose the sensor pair instead of accepting the module's mix.

The shortcut is a native binocular board camera: one board, two global-shutter sensors and a link that already answers the sync question. The LS-BCM-0043 delivers 1280 x 1024 over USB3 from two on-board global-shutter sensors.

For a build-your-own pair, the LS-BCM-0003 streams 3.1 MP at 120 fps over USB3 from a global-shutter sensor behind a C-mount, and the MV-CB016-10GM-M-W gives a 1.6 MP global-shutter mono sensor on an M12 mount over GigE. Put each camera on its own host controller, trigger both from one line, and synchronization is solved in hardware: the opto-isolated GPIO direction guide sorts out which side sources current, and the per-camera USB 3 bandwidth math sizes each link. Match M12 lenses by thread before ordering with the board camera lens mount guide, and narrow sensor and interface choices with the board level camera selection guide.

Checks before you run the first pair

Five checks separate a working stereo bench from a week of debugging:

1. Enumerate the advertised frame sizes. Double width means side-by-side packing; two entries mean two free-running devices.
2. Count delivered frames for 60 seconds at the working resolution and divide per view; compare against the bandwidth arithmetic, not the spec sheet.
3. Read the real bytes per frame from one captured MJPEG frame, because every fps estimate scales with it.
4. Confirm the spectra match on both sides if you want dense disparity.
5. Calibrate, then log the reprojection error and the caliper-measured baseline against the module's serial number before the first depth run.

The board camera evaluation checklist extends this list into a full selection audit.

The decision in one line: keep the dual-sensor USB module for synchronized pairs at modest frame rates and near-field depth, and step to a binocular board camera or a triggered pair of global-shutter cameras once the experiment demands 30 fps per view at 1080p, raw frames or a longer baseline. The camera finder filters by interface, shutter and resolution, the camera configurator pairs a board camera with lens and cable in one pass, and the 3D sensors category lists ready-built stereo and profiler systems when the module route is outgrown. The rest of the 3D vision topic covers the depth side from calibration to point clouds.