Stereo calibration with fixed intrinsics: extrinsics only

Calibrate stereo pair A-B, then pair B-C, and camera B comes back with two different focal lengths and distortion sets, because each solve only sees the checkerboard inside its own pair's overlap. On a rig built from several stereo pairs, every camera that enters two solves ends up with two intrinsic sets, and no triangulated point is fully trustworthy. The fix is to split the problem: solve each camera's intrinsics once, on its own, over its full field of view, then freeze them and let every stereo solve estimate only the rotation and translation between cameras. This guide walks through the procedure for a rig of stereo pairs: the bench set-up, the flags, the image counts, the pose chaining, and the depth-error budget that tells you the result is good enough.

Why camera B gets two intrinsic sets

A joint stereo solve estimates both cameras' intrinsics and their relative pose at once, adjusting everything to drive the total reprojection residual down. It can only learn from corners that both cameras see, and both only see the board where their fields overlap — the same strip where a stereo depth image exists (see why stereo depth coverage is smaller than either image). Solve (A,B) therefore tunes camera B's focal length and distortion against one side of its sensor, and solve (B,C) against the other. Each result minimizes its own residual; nothing forces the two to agree, and on real rigs they do not.

The solver is not broken — the question it was given is underdetermined. Intrinsics describe the camera and its lens alone, so they deserve images that cover the whole sensor, not an overlap strip. Give every camera one full-field intrinsic pass, and the stereo problem collapses to what the mount actually defines: three rotation angles and three translation components, six numbers per pair.

The decision, then: intrinsics once per camera, frozen from then on; extrinsics-only solves for every pair.

What you need on the bench

• A rigid checkerboard. Print or laminate on a flat carrier, and measure the square size with calipers — the true size enters the solve directly, and a warped board biases everything downstream. For fields of view near a metre, build a full-size grid instead of moving a small board around (how to build one for a 1 m field).
• Locked focus and aperture. Intrinsics belong to the camera, lens and focus setting as one unit; a ring that moves later voids them. Lock or tape the rings after Step 1.
• A mount that does not move. The extrinsics are only as stable as the plate the cameras sit on.
• Synchronized capture for the pair passes. Both images of a pair must show the board in the same pose; blur from unsynchronized frames inflates the RMS (fix the trigger before blaming the math).
• Even light on the board. Matte surface, no glare; corners must stay crisp for subpixel detection.

For the cameras themselves, a fixed-focus area scan camera is the standard building block, and if one node of the rig is a factory stereo unit, its 3D sensors ship with intrinsics already loaded — treat that set as the frozen one.

The MV-CU020-90GC is a 2.5 MP global-shutter GigE camera with a C-mount, the MV-CH120-11UC is a 12 MP global-shutter USB3 model, and the MV-CS200-10UM adds a 20 MP mono sensor on a 1-inch format — more pixels under each printed corner steadies the subpixel detection the whole procedure rests on.

Step 1: solve each camera's intrinsics alone

Work one camera at a time. The board now only has to be inside this camera's view, which is the freedom the joint solve never offered.

1. Capture 20-30 views: board in all four corners, along every edge, in the centre, at two or three distances, tilted up to roughly 45 degrees. Let the board reach the frame edges — edge distortion is exactly what an overlap-limited solve never captured.
2. Run a single-camera calibration on each set: cv2.calibrateCamera from OpenCV's calib3d module, or the single-camera mode of your library or desktop tool. The output is a 3×3 camera matrix (fx, fy, cx, cy) plus a distortion vector (k1, k2, p1, p2, k3 for the standard 5-coefficient model).
3. Prune and re-solve. Drop the few views with the largest per-view residual and re-run. A fixed-focus mono lens lands near 0.2-0.5 px RMS; if it stays above 1 px, fix blur, focus or board flatness first.
4. Write the result to a tool-neutral file — OpenCV YAML or JSON — as the camera's single intrinsic record, then stop optimizing it (moving calibration parameters between tools). From here on, the intrinsics are constants.

Step 2: freeze the intrinsics and solve each pair

For every stereo link, capture 15-25 synchronized image pairs of the board inside that pair's overlap, varied in position, distance and tilt. Then solve with the intrinsics fixed:

1. Pass each camera's saved camera matrix and distortion vector into the stereo solve as inputs.
2. Set the fix-intrinsics flag: in OpenCV, cv2.stereoCalibrate with flags=cv2.CALIB_FIX_INTRINSIC. The solver then estimates only the rotation R, the translation T and the essential and fundamental matrices E and F. Tools without a single combined flag expose per-parameter equivalents such as CALIB_FIX_FOCAL_LENGTH and CALIB_FIX_PRINCIPAL_POINT.
3. Read the reported RMS. On clean pair data, a frozen solve lands within roughly 0.1-0.2 px of the solo values, about 0.3-0.5 px overall. If it comes back far higher, the cause is the pair data — blur, too few poses, glare — or a mount that moved between sessions, not the frozen intrinsics.
4. Save R and T for each link. Together with the fixed intrinsics, that is the pair's complete description.

Camera B now keeps exactly one intrinsic set and gains two relative poses, instead of two conflicting sets of intrinsics.

Step 3: chain the poses into one shared frame

Each frozen solve returns one relative pose: a 3×3 rotation R and a 3×1 translation T. Compose them into one coordinate frame by multiplying 4×4 matrices along a spanning tree of the rig.

Ten cameras need nine edges: the five rig-internal pairs plus four bridges connecting the pairs to each other. Choose the bridges where two cameras from different rigs overlap most and see the board at a good angle; any connected tree defines a consistent frame, and the chain is only as good as its weakest bridge. Pick one camera — say the left camera of rig 1 — as the world origin, and every other camera's pose is the matrix product along its path. The same chaining holds when the nodes are depth cameras rather than bare stereo pairs: see calibrating two depth cameras for combined point clouds and multi-camera extrinsic calibration in one shared frame.

Worked example: depth error for a 300 mm baseline at 1.5 m

This is the budget that decides whether the frozen-intrinsics result is good enough for the task.

Inputs, and where each comes from:

• f = 1600 px — calibrated focal length in pixels; an example value typical for a 2-5 MP camera with a normal lens. Your solo pass prints the real number.
• B = 300 mm — baseline between the two cameras of a pair; an example bench scale.
• Z = 1.5 m — distance to the measurement plane; example.
• δd = 0.5 px — reprojection RMS target from Steps 1-2; 1.0 px is the loose case for comparison.

Rule in words: an error of δd pixels in disparity at depth Z maps to a depth error that grows with the square of the distance and shrinks in proportion to focal length and baseline:

Arithmetic at Z = 1.5 m = 1500 mm and δd = 0.5 px:

δZ = (1500 mm)² × 0.5 px / (1600 px × 300 mm) = 1,125,000 / 480,000 mm = 2.3 mm

With δd = 1.0 px the error doubles, to 4.7 mm. The Z² term means 3.0 m quadruples it: about 9.4 mm at 0.5 px. Doubling the baseline to 600 mm halves it again: 1,125,000 / 960,000 mm = 1.2 mm.

Decision: if the task tolerates ±3 mm at 1.5 m, hold every pair's reprojection RMS at or below 0.5 px — exactly what Steps 1-2 target. Measuring nearer 1 px? Widen the baseline or lengthen the focal length before re-solving anything; a 600 mm baseline pulls the 1.5 m error back to about 1.2 mm at 0.5 px.

Verify the rig before you trust it

• Per-pair RMS. Every frozen solve reports it; it should sit inside the band from Step 2.
• Depth check. Triangulate board corners at a measured distance and compare the spread against the budget from the worked example.
• Cross-rig artifact. One known-size artifact, seen simultaneously by cameras from two different rigs: agreement within the budget tests the whole chain, which per-pair RMS cannot.
• Undistorted overlay. Reproject two cameras' images onto one plane using the saved pose; ghost edges at shared features mean the pose is off.
• Mixed resolutions. Cameras of different resolutions fit this scheme by construction, because intrinsics are strictly per camera — see stereo calibration with cameras of different resolutions.

When free intrinsics are still the right choice

A single fixed pair whose board can already cover the whole overlap can be solved with intrinsics floating: the conflict only appears when one camera enters two or more solves. A zoom or motor-focus lens changes its intrinsics with every setting, so freezing a stale set is worse than re-solving per setting. In service, vibration and temperature drift move the extrinsics while the intrinsics hold — online extrinsic-only recalibration from sparse stereo correspondences is established practice on vehicle and robot rigs for exactly that reason. Refresh poses on a schedule; re-solve intrinsics only after a physical change to the imaging path.

Rig calibration checklist

• Board flat, squares measured — size errors become scale errors in T.
• Focus and aperture locked after Step 1; any movement voids the intrinsic sets.
• 20-30 solo views per camera, reaching all four frame edges.
• Intrinsics frozen on every stereo solve, via CALIB_FIX_INTRINSIC or per-parameter flags.
• 15-25 synchronized pairs per link, varied in distance and tilt.
• Nine links for ten cameras: five rig-internal, four bridges.
• Per-pair RMS inside the target band; cross-rig artifact inside the depth budget.
• Extrinsics re-checked after transport or temperature swings; intrinsics only after a lens or focus change.

Specifying cameras for a new rig, or extending an existing one? Start from the configurator, or send the rig sketch — camera count, mounting, fields of view and the accuracy target — through the contact page, and we will check the baseline and focal-length choices against the depth budget before you order.