Calibration of three cameras by chaining two stereo pairs

Three cameras at 90° to each other — top, side and end view of one part — is a standard inspection rig, yet the stereo calibration tools in most vision packages still take two cameras per run. The way through is composition, not a bigger tool: calibrate two overlapping stereo pairs and multiply their transforms, so the camera the pairs share carries the third into the first camera's coordinate frame. This walkthrough covers the intrinsics-first workflow, both pair runs, the composition arithmetic with numbers, and a millimetre-level check that proves the chain. It applies to any area scan cameras you can bolt to a rigid frame, from a bench mock-up to a production cell.

Can a stereo calibration app calibrate three cameras?

No. A two-camera stereo calibrator recovers exactly one relative pose per run, so three cameras need two runs plus a composition step; the composition is exact algebra and adds no error beyond what each pair already carries.

Treat the calibrated pair as one camera and calibrate the third against it? That tempting idea has no clean geometry behind it: a stereo pair has two projection centres, so no single camera model describes it. Chain through the shared camera instead. Camera 2 is calibrated against camera 1, camera 3 against camera 2, and cameras 1 and 3 never need a common view — exactly what a 90° triad needs when the outer cameras face away from each other.

The pattern is a spanning tree of pairwise links rooted on one camera, the same structure used to build robot-style transform trees. Two neighbours of the pattern matter here. When all three cameras see one large board simultaneously, a multi camera extrinsic calibration in one shared coordinate frame is the better solve, because it averages error over all views at once. And the practical traps that block three mono cameras calibrated on one board — corner detection failing on one sensor, sharpness falling off at the board edges — are the traps you will meet inside each pair here.

How do you chain two stereo pairs into one frame?

Calibrate the intrinsics of cameras 1, 2 and 3 separately, calibrate pair 1-2 for R12 and t12, then pair 2-3 for R23 and t23, and compose: R13 = R23 · R12 and t13 = R23 · t12 + t23, which maps points from camera 1's frame into camera 3's frame.

Write the two pair results as mappings between coordinate frames: X2 = R12 · X1 + t12 takes a point from camera 1's frame into camera 2's, and X3 = R23 · X2 + t23 takes it from camera 2's into camera 3's. Substituting one into the other gives the composed map from camera 1 straight to camera 3:

Both pair runs must use the same convention — which frame each R and t maps from and to — and the same length unit. Enter the checkerboard square size in millimetres in both runs and every translation comes out in millimetres; a run entered in centimetres quietly scales half your chain. To invert a transform, use the pair (Rᵀ, −Rᵀ · t); transposing the rotation alone is not an inverse.

What does the composition look like with real numbers?

For two 90° yaws about Z, R12 and R23 are both 90° rotations, so the composed R13 is a 180° rotation; with t12 = [500, 0, 0] mm and t23 = [0, 0, 400] mm the composed baseline is t13 = [0, 500, 400] mm, about 640 mm.

Take a rig where camera 2 is yawed 90° from camera 1 and camera 3 another 90° from camera 2, so cameras 1 and 3 look in opposite directions. Both rotations are about the vertical Z axis:

R23 · R12 = [  0  -1  0 ] [  0  -1  0 ]   [ -1   0  0 ]
            [  1   0  0 ] [  1   0  0 ] = [  0  -1  0 ]
            [  0   0  1 ] [  0   0  1 ]   [  0   0  1 ]

For the translation, the 90° rotation turns the 500 mm x-component of t12 into a 500 mm y-component, so t13 = R23 · t12 + t23 = [0, 500, 0] mm + [0, 0, 400] mm = [0, 500, 400] mm. The composed baseline length is sqrt(500² + 400²) = sqrt(410,000) ≈ 640 mm.

Check the rotation angle against the rig: the angle of any rotation matrix is θ = arccos((trace(R) − 1) / 2). Here trace(R13) = −1 − 1 + 1 = −1, so θ = arccos((−1 − 1) / 2) = arccos(−1) = 180°. Two 90° yaws must compose to a 180° flip about Z. If your numbers give 0°, or the right angle about the wrong axis, you multiplied the pair transforms in the wrong order.

What hardware does a three-camera chain need?

You need three cameras you can lock down, one checkerboard whose squares stay at least 10 pixels wide in every camera, and a rigid frame that does not shift between the two pair runs; a slide-table stand makes the board poses repeatable.

Cameras. Three bodies of the same model keep sensor, gain structure and defaults matched, which makes the image sets comparable across pairs. Match resolution to the smallest feature you must locate, no more. For a bench rig, a common body is the LS-CMR-0166, a 12 MP USB3 area scan camera with the IMX226 sensor, 4000 × 3000 resolution, rolling shutter and 30 fps:

When the part moves while you capture, choose a global-shutter body instead, such as the MV-CH120-11UC, a 12 MP USB3 camera with the IMX304 sensor and a C-mount:

Target. A matte printed checkerboard on a stiff backing, with squares at least 10 pixels wide in every camera at the working distance. Measure the square size once and enter it in millimetres in both pair runs.

Mounting. The cameras must not shift between the two pair runs — a shift becomes a silent error inside R13 that no per-pair error metric will show. Put all three bodies on one beam or stand from the camera stands range; the stillness discipline behind 3D camera mounting and image quality applies unchanged. A slide-table stand such as the LS-MNT-0040, a machine vision camera stand, professional slide-table type, height 700 mm, 400 x 450 mm base:

How do you run the calibration step by step?

Eight steps: lock the optics, calibrate three sets of intrinsics, run the two pair calibrations without touching the mounts, compose the transforms, publish one tree, verify in millimetres, then archive everything so a re-check after a bump takes minutes.

1. Lock the optics. Set focus and iris on every camera, then disable autofocus and tape the rings — the rule of calibrating a camera with autofocus: lock the focus first applies to all three bodies.
2. Calibrate intrinsics per camera. Capture 15 to 20 checkerboard poses per camera, tilts up to about 45°, the pattern reaching all corners and edges of the field of view. Aim for a mean reprojection error of 0.3 px per camera and treat 0.5 px as the ceiling.
3. Calibrate pair 1-2 with intrinsics fixed. Feed both cameras' intrinsics in as constants so the solver returns only R12 and t12. Record the per-pair error.
4. Calibrate pair 2-3 without touching the mounts. Same procedure, same locked optics, intrinsics of cameras 2 and 3 fixed. Record R23 and t23.
5. Compose. R13 = R23 · R12 and t13 = R23 · t12 + t23, in the convention both runs share.
6. Publish one tree. Root the coordinate frame on camera 1 and hang cameras 2 and 3 from it — the same discipline as when you chain pairwise camera extrinsics into one valid TF tree.
7. Verify in millimetres. Run both checks in the next section before the rig sees a part.
8. Archive. Keep the images, the intrinsics, both pair results and the tape-measured baselines, so a re-check after a bump is a ten-minute job, not a recalibration.

Why fix the intrinsics before calibrating the pairs?

Because a pair solver that also refines intrinsics can return two different focal lengths for the middle camera — one from each pair — which makes the chain internally inconsistent; with intrinsics fixed, each pair estimates only R and t from the shared board views.

A stereo solver set to refine intrinsics returns one focal length and distortion set for camera 2 from pair 1-2 and a second from pair 2-3: the same physical camera described two ways. Chain those and camera 2's projection disagrees with itself, which surfaces as a millimetre-level bias in the composed 1-3 transform. Locking the intrinsic parameters to the solo-calibration values leaves each pair solve six extrinsic numbers, so the board poses you already have are enough, and the two pair results are consistent by construction. This is precisely the stereo calibration with fixed intrinsics: extrinsics only workflow. Re-shoot intrinsics whenever a lens, an aperture or a focus ring changes: they describe the locked optical set, not the camera body alone.

How do you verify the chained result in millimetres?

Triangulate one test point through pair 1-2 and again through pair 2-3, express both answers in one frame, and compare: with 0.3 px pairs at a 500 mm working distance they should agree within about 0.2 mm, and a gap above 1 mm means a pair needs redoing.

Geometry check. Compute θ = arccos((trace(R13) − 1) / 2) and the baseline length |t13|, and compare with a tape measure on the rig — in the worked example, 180° and 640 mm. Expect agreement to a few millimetres on the baseline and a degree or two on the angle. A grossly wrong result — baseline on the wrong side, rotation about the wrong axis — is the signature of a convention error, not noise.

Point-consistency check. Put a marked point, such as a printed cross, where cameras 1 and 2 both see it and camera 3 sees it too. Triangulate it from pair 1-2 and map the result into camera 3's frame with R13 and t13. Triangulate it again from pair 2-3 and map that result with R23 and t23. The two estimates of one physical point now differ by exactly the error your chain carries, and every transform in the chain is exercised.

For the acceptance number, run the error budget for your own sensor and lens: with a 3.45 µm pixel behind an 8 mm lens, one pixel subtends 0.00345 / 8 = 0.00043 rad, so a 0.3 px mean error is 0.00013 rad, which at 500 mm of depth is 0.065 mm per camera. Two cameras triangulating place a point within about 0.13 mm, and two independent pairs within about 0.2 mm.

• Below 0.5 mm agreement: the chain is good, ship it.
• Between 0.5 and 1 mm: re-shoot the weaker pair, the one with the higher per-pair error, and recompose.
• Above 1 mm: stop — check conventions and mounts before trusting any of it.

The camera resolution calculator converts your own sensor and field of view into the pixels-per-millimetre scale this budget starts from.

What goes wrong with cameras 90° apart?

Board placement, not the math: at 90° the checkerboard must sit near the bisector of the two views, tilted toward both cameras, and stay sharp in each; opposite-facing pairs need a double-sided board.

The failures, in order of frequency:

• The board shows in only one camera. A board squared up to one camera is edge-on to the other. Work around the bisector, tilt the board roughly 45° to each camera of the active pair, and move it across the shared volume for the 15 to 20 poses.
• All-frontal poses. Poses without tilt leave the rotation weakly constrained; the tilted shots are the ones that pin R down.
• A swinging board on rolling-shutter cameras. Fast handheld motion skews corner positions on a rolling shutter; pause at each pose or use a global-shutter body.
• Focus or aperture touched between runs. The fixed intrinsics no longer describe the optics, and the millimetre check catches it late.
• A bumped mount between the pair runs. The composition silently inherits the shift; the point-consistency check is the only thing that will show it.
• Mixed capture regimes on moving scenes. A static board needs no synchronisation. Once you triangulate moving parts, put all three cameras on one hardware trigger — setting up a USB stereo trigger pair shows the pattern for two, extended to three.

When is chaining pairs the wrong method?

When every camera sees the same board at the same time, calibrate all three in one shared frame, because a joint solve averages error across all views instead of stacking two pairs; and when no two cameras share any view, pairwise chaining cannot even start.

• One shared overlap. Newer MATLAB releases ship estimateMultiCameraParameters, which estimates the extrinsics of several cameras from images of one or more calibration patterns — use that before writing your own composition.
• No overlap at all. Pairwise chaining needs a connected chain of overlapping pairs. Rigs with no shared views need other evidence: a planar target moved between views, or the mirror-based methods from the research literature on non-overlapping multi-camera rigs.
• One dense depth map. If the deliverable is depth rather than three registered views, more cameras can hurt: multi camera stereo depth: why two cameras beat five for a depth map covers the matching side.
• Growing the rig later. A fourth camera is one more pair calibration hung off the anchor, and the observations you already hold can be reused when aligning frames from pairwise calibration observations.

To build the rig: pick the bodies in the camera finder, lock them to one stand from the accessories finder, and take the set through the system configurator to match lenses and controller. For a three-camera cell with awkward overlaps, tell us the geometry and we will suggest the pair order.