Multi-camera stereo depth: why two cameras beat five for a depth map

You have five identical USB cameras on the bench and one job that needs a depth map. The tempting plan is to point all five at the scene and let the stereo matcher take what it needs. That plan fails at calibration, not at matching: stereo is a strictly two-image computation, and every extra camera multiplies the calibration work. This guide is for anyone building that rig today.

It shows why block matching only ever sees two images, how five cameras turn into fifteen calibration datasets, and how baseline and pixel pitch set your depth error. It also shows where the break-even sits between adding cameras and calibrating the two you already own. The worked example uses an 8 mm lens, 3.45 µm pixels and a 100 mm baseline at 1 m; the closing sections name factory-calibrated 3D sensors that skip the whole problem.

Can five cameras feed one stereo depth map?

No: a block-matching stereo run such as SBM compares exactly two rectified images and returns one disparity image, so five cameras give you no extra depth until you choose which two form each pair and calibrate them as that pair.

The matcher's input is fixed by design — left image, right image, disparity range. What changes between runs is scene content, never camera count. Multi-view stereo, the research field that genuinely consumes N cameras, still computes depth pairwise and then reconciles the solutions, and it carries the same per-pair calibration bill shown below. Turning a disparity image into metres is unchanged by camera count; the disparity-to-3D walkthrough follows that arithmetic pixel by pixel.

How many calibration jobs do five cameras create?

Five cameras create ten stereo pairs, because n cameras form n × (n − 1) ÷ 2 pairs, and every pair needs its own extrinsic calibration on top of one intrinsic calibration per camera — 15 datasets before the first trustworthy depth pixel.

Two cameras need three datasets. The count grows quadratically:

Hold each calibration under about 1 pixel of reprojection error; above that, depth noise climbs faster than any added camera can rescue. A calibration also belongs to the pair, not to the rig: move one camera by 2 mm and every pair containing it is void, which is why a moved camera means recalibrating each stereo pair it forms. When the cameras themselves stay put, the extrinsics-only calibration workflow keeps per-camera intrinsics fixed and redoes only the pair geometry.

How wide should the stereo baseline be?

Size the baseline from your nearest surface: B = disparity window × pixel pitch × nearest distance ÷ focal length, which with a 256-pixel window, 3.45 µm pixels, an 8 mm lens and a 1 m nearest surface gives B ≈ 110 mm.

A block matcher searches a limited disparity window, typically 64 to 256 pixels on workstation tools. The disparity a surface produces is d = f·B/Z, so the nearest surface you must measure caps the baseline; anything wider pushes the near field outside the window. With p = 3.45 µm and f = 8 mm:

The middle row, in full: 256 × 3.45 µm = 0.883 mm of allowable disparity, and B = 0.883 mm × 1000 mm ÷ 8 mm = 110 mm. Sit just inside the limit, because depth precision improves in direct proportion to B. Mount both cameras on one machined bar: a 0.5 mm drift on a 100 mm baseline is a 0.5% depth error at every distance.

Check overlap while you are at it. An 8 mm lens on a 1/1.7-class sensor, about 7.5 mm wide, spans roughly 0.47 m at 0.5 m; the 110 mm baseline consumes about a quarter of that, leaving a generous stereo band.

What depth error does one pixel of matching noise cost?

Depth error scales with distance squared: with an 8 mm lens, 3.45 µm pixels and a 100 mm baseline, one pixel of disparity noise costs about 4 mm at 1 m, 17 mm at 2 m and 39 mm at 3 m.

Worked example, arithmetic shown:

• At 1 m the disparity is d = 8 mm × 100 mm ÷ 1000 mm = 0.8 mm, which is 0.8 ÷ 0.00345 ≈ 232 pixels. One pixel of noise: δZ = (1000 mm)² × 3.45 µm ÷ (8 mm × 100 mm) = 4.3 mm.
• At 2 m the disparity halves to 116 pixels and the error quadruples: 4 × 4.3 mm ≈ 17 mm. At 3 m it is 9 × 4.3 mm ≈ 39 mm.
• Sub-pixel matching at 0.2 pixels, realistic on a well-textured scene, divides these by five: 0.9 mm at 1 m, 3.5 mm at 2 m.

There is a hard floor in these numbers. At the nearest measurable distance the disparity equals the window, so per-pixel resolution there is Z_min × σd ÷ d_win — 1000 mm × 1 ÷ 256 ≈ 3.9 mm — no matter which focal length or baseline you chose. Relative precision is simply σd/d_win: 1 pixel in 256 is 0.4% of range, and 0.2-pixel matching brings it to roughly 0.1%.

Decision: if your tolerance is near 1% of range — 10 mm at 1 m — this pair delivers with margin. If you need ±1 mm at 2 m, that is 0.05% of range, below the 0.1% floor this geometry reaches; no lens or baseline choice fixes it, and a factory-calibrated 3D sensor is the shorter path.

Does fusing ten pairs beat one calibrated pair?

Only against random noise: ten independent pair measurements can cut matching noise by up to √10 ≈ 3.2× where the pairs see the same surface, but calibration error is systematic, repeats in every fused point and never averages out.

Fusion pays only when each pair's calibration is independently tight and the errors uncorrelated. Real rigs fail differently — shared temperature drift moves all cameras on one bar together, so those ten measurements are partly copies of each other. Fusing heterogeneous camera geometries is also an algorithm project, not a setting in a stereo tool. For a first build, spend the effort on one excellent pair, then add a second calibrated unit only when part of the scene is genuinely invisible to the first. The integration recipes exist: two depth cameras combined into one point cloud and multi-camera fusion with depth-camera extrinsics.

Which ready-made 3D sensors already solve this?

For most production depth tasks the honest replacement for five cameras is one factory-calibrated device: a structured-light 3D camera, a stereo camera with pattern projection, a binocular laser profiler or a logistics volume camera — intrinsics, extrinsics and fusion arrive already done.

A 3D sensor collapses the whole calibration chain into one housing. Pattern-projection stereo suits static parts and small-to-medium fields. Speckle projection covers logistics volumes such as parcel dimensioning. A binocular single-line laser profiler triangulates each laser line across two cameras for conveyor profiling; a galvanometer stereo camera steers the beam over larger surfaces. The MV-DP3062-01D V2.0 binocular single-line profiler is the conveyor version of the same two-camera idea, with the geometry set at the factory.

Choose a ready sensor when tolerance is written in absolute millimetres, when the nearest surface sits inside roughly half a metre, or when recalibration downtime is unacceptable. Pick the sensor whose published accuracy at your working distance meets the tolerance from the worked example above.

Which two cameras make a good first stereo pair?

Pick two identical global-shutter cameras with a hardware-trigger input, two lenses of the same model, one rigid baseline bar and a single trigger pulse into both shutters; rolling shutters or free-running cameras tear depth apart at every moving edge.

Global shutter matters the moment anything moves: a rolling shutter exposes rows at different times and stereo reads that skew as geometry. Hardware triggering keeps both exposures within microseconds. Mono sensors match better than colour, though colour works on well-textured scenes. Two GigE cameras also want one network link each — a second NIC or a managed switch — and a host sized for your resolution and frame rate; industrial mini PCs are the usual bench choice. The camera finder filters by global shutter, sensor format and interface, and the area scan cameras page groups the full range. One proven starter:

The MV-CU020-90GC pairs a 1/1.7" global-shutter sensor with GigE on a C-mount, and its 2.5 MP is ample for a first disparity map; a second identical unit keeps the intrinsics interchangeable.

What do you check before ordering the parts?

Freeze the mechanical plan first: identical cameras and lenses, one machined bar holding the baseline to ±0.5 mm, a trigger line reaching both cameras and a host with one network link per camera — then order to that plan.

• Identical camera model on both sides. Identical sensors and pixel pitch make intrinsics interchangeable and spares trivial.
• Global shutter plus hardware trigger. Sync at the shutter, not in software, is what keeps moving scenes solid.
• Two lenses of the same model, focused and locked. Refocusing later changes geometry and intrinsics together.
• Baseline measured and recorded to ±0.5 mm. Half a millimetre on 100 mm is a 0.5% depth scale error.
• Depth of Field covering the working band. Stop both lenses down and check the near and far edges stay sharp with the depth of field calculator.
• A calibration target larger than the shared field. A board that fills one camera's frame but not the overlap band cannot calibrate the pair; for wide fields, build a calibration grid for a 1 m field of view.
• Host bandwidth. Two simultaneous full-rate streams need two links; count NICs and switch ports before the cameras arrive, or start from the camera configurator.

When are five cameras the right answer after all?

Five cameras earn their cost when one pair cannot see the job: a scene wider than the shared field, deep bins with occlusion, or several stations reporting into one world frame — then each pair is calibrated alone and the clouds are fused by extrinsics.

That is a coverage decision, not an accuracy decision. Give each pair its own band of the scene and its own baseline, computed with the rule above. Calibrate every camera directly against one reference camera rather than chaining pair to pair, because chained calibrations compound their errors along the chain. If the fused rig feeds a robot cell, run the 3D vision robot palletizing integration checks before mounting. The rest of the 3D vision guides cover calibration and fusion in depth.

The break-even, in one line: two cameras calibrated under one pixel beat five uncalibrated ones, and if your tolerance sits below 0.1% of range, buy a factory-calibrated 3D sensor instead of building any of it. Send your working distance, nearest surface and tolerance through the contact page and we will run the arithmetic with you.