Stereo calibration with angled cameras: what still works

You angled two cameras toward each other — about 60 degrees between their optical axes — and the calibration you ran on twenty-plus image pairs came back unusable, so the natural suspect is the angle itself. It is the wrong suspect. Stereo calibration estimates the complete relative pose between two cameras, convergence included, and the geometry of a toed-in rig breaks nothing. What breaks is the capture: with converged cameras the checkerboard stops reaching the outer regions of each image, close-range focus goes soft, and oblique boards bias the corners. This explainer covers what the solver actually does on an angled rig, the capture pattern that fixes it, a lens and accuracy worked example, and the checks that pass before you trust the depth map.

Does stereo camera calibration work with angled cameras?

Yes — the solver estimates the full relative rotation and translation between the two cameras, so a convergent rig with 60 degrees between the optical axes calibrates exactly like a parallel one; the convergence lives in the rotation matrix, not in an assumption.

A stereo calibration routine refines each camera's intrinsics — focal length, principal point, distortion — and then the extrinsics: a rotation matrix and a translation vector that place camera two in camera one's frame. Any mounting pose is admissible; perfectly parallel cameras are the special case where that rotation is near identity. Workflows built on checkerboards, like stereo calibration from paired checkerboard images, never assume the axes are parallel, so a rig verged 30 degrees per camera is exactly as solvable as a flat one.

When the output is a depth map of noise and a nonsensical baseline, the cause is capture nine times out of ten, not geometry. The angle changes what the images contain — and the images are everything the solver has.

What does the calibration tell me about the convergence angle?

The rotation matrix between the cameras holds the whole vergence — for a symmetric toe-in, each camera is turned in by half of your 60 degrees — and together with the baseline it fixes where the two axes intersect.

Most toolboxes print the rotation as three angles or a rotation vector; read the yaw term and compare it with what you built. On a symmetric rig the relative yaw should land near 60 degrees, half contributed by each camera. The length of the translation vector should match the center-to-center spacing you measured with a ruler; a mismatch beyond a few percent means the fit is weak or a mount moved mid-session.

The same two numbers give the fixation distance, where the axes cross:

With a 160 mm baseline and 30 degrees per camera, the axes cross at D = 80 / tan(30 degrees) = 80 / 0.577 = 139 mm from the baseline midpoint. If the part lives at 400 mm, the fixation point is not on the part, and every part point then carries the same disparity sign — workable, but worth knowing when you read the disparity map. Instruments that measure a stereo camera's convergence angle from matched image regions fill the patent literature; a healthy calibration hands you the same angle from the rotation matrix anyway.

Why did twenty image pairs calibrate so badly?

Because on a converged rig the board cannot reach the outer thirds of each image, and corners from soft-focus or steeply oblique boards drag the fit: read the per-pair reprojection errors, not just the mean.

The pass mark is a mean reprojection error under about 0.5 pixel, with no pair sticking far above the pack. When the mean is worse, the causes arrive in this order:

1. Coverage holes. Both cameras must see the board at once, and with 60 degrees of vergence the shared volume is a wedge. Poses drift toward the wedge's center, the far left of the left image and the far right of the right image never see a corner, and the distortion model is unconstrained in those columns — depth there comes out as noise. Calibration target field of view and lens coverage margins covers how much margin each edge needs.
2. Soft boards. At part-scale distances the depth of field is tens of millimetres, numbers below; a board pushed toward a camera goes out of focus and its corner centers shift by pixels. If the lens can refocus itself, lock the focus before you calibrate.
3. Oblique poses. Boards tilted far past about 45 degrees compress their far squares and bias sub-pixel corner centers. Keep a spread of mild tilts — flat, then 15 to 30 degrees toward and away — instead of extremes. Enter the true grid geometry too: a wrong square size or pitch breaks the fit.
4. Unsynced shutters. A hand-carried board seen at two different instants is two different scenes. Trigger both cameras from one line and hold still between frames; rolling shutters make any slip worse.

Plot the per-pair reprojection errors, delete the outliers — discard the bad pair or retake the pose — and refit. If most pairs sit high, no pruning saves the set: reshoot with the wedge pattern in the next section. For board handling and stream set-up end to end, the stereo depth camera target capture walkthrough is the companion piece.

How do I cover both fields of view when the cameras toe in?

Work the overlap wedge on purpose: three depth planes — near, fixation, far — crossed with five lateral positions and two tilts gives about thirty poses, and both images fill corner to corner.

Walk the wedge systematically. Shoot at the nearest plane where both cameras still see the whole board, at the fixation plane, and at the far plane; at each depth take five positions across the overlap — hard left, left, center, right, hard right — with the board tilted toward one camera, then the other. Most toolboxes overlay detected corners on the image; the goal is corners inside every quadrant of both images. A quadrant still empty after thirty poses is forbidden by the wedge itself, and the cure is less vergence or a wider lens, not more pictures. When the board cannot be large enough at the working distance, target size for the distance is the real constraint; when a rigid board cannot physically reach, surveyed scene points replace the chessboard.

Two properties of a converged pair shape all of this. Each camera views the part plane about 30 degrees off its normal, so a square board images as a trapezoid — keystone, expected, harmless at moderate angles. And only the overlap produces depth, which is why a depth image's field of view is smaller than either camera's. Match the two exposures as well, so corners are detectable in both eyes: set auto exposure per stereo camera in a depth rig.

What focal length does each angled camera need?

Size each camera for its own working distance, which is longer than the rig standoff: at 60 degrees of convergence and a 160 mm baseline, each camera sits 2 percent farther away at 400 mm and 8 percent at 200 mm.

Worked example for one camera of the pair:

• Part 200 mm wide, 400 mm in front of the rig midpoint; cameras 160 mm apart, toed in 30 degrees each so the axes cross on the part.
• Per-camera distance to the part center: sqrt(80^2 + 400^2) = sqrt(6400 + 160000) = sqrt(166400) = 408 mm.
• Wanted horizontal field of view 250 mm — the part plus a quarter of margin.
• Sensor: a 1/1.7" format, about 7.4 mm wide, with 4000 × 3000 pixels, so the pixel pitch is 7.4 / 4000 = 0.00185 mm = 1.85 µm.

• Magnification m = 7.4 / 250 = 0.0296.
• Focal length f = 408 × 0.0296 / 1.0296 = 11.7 mm — fit a 12 mm C-mount lens.

• Footprint on the pixels: 250 / 4000 = 0.0625 mm per pixel.
• Vertical coverage: 5.5 mm of sensor height / 0.0296 = 186 mm.
• On the tilted part plane the horizontal footprint stretches by 1 / cos(30 degrees) = 1.15, to about 289 mm; the vertical stays 186 mm. The shared window holds a 200 mm part with room to spare.

Focus is the quiet killer at these distances. With a 2-pixel circle of confusion (2 × 1.85 µm = 3.7 µm) at this magnification, total depth of field runs about 70 mm at f/8 and about 24 mm at f/2.8 — stop the lens down for the calibration sweep and leave it there.

Two identical cameras are the clean build. Small and synchronized, the MV-CU020-90GC is a 2.5 MP GigE camera with a global shutter and a 1/1.7" sensor.

For more pixels on the target, the LS-CMR-0166 delivers 4000 × 3000 pixels at 30 fps over USB3, and its rolling shutter makes a shared hard trigger non-negotiable. For a global shutter at that resolution, the MV-CH120-11UC is a 12 MP camera with a 1.1" sensor and a C-mount. Identical models avoid resampling penalties; stereo calibration with cameras of different resolutions covers what changes when they differ.

Does rectification still work with 60 degrees of convergence?

Yes — rectification rotates both images onto a common image plane whatever their starting orientation, but at this vergence you pay in black corners, resampled edges and a smaller valid overlap, so plan to crop to the central wedge.

Rectification computes a remapping rotation for each camera so that epipolar lines become image rows and block matching searches along one axis only; live rectification of a calibrated pair shows the pipeline running. Expect triangular black regions where the remap asks for pixels that were never captured — carry the validity masks through and crop. Interpolation costs a fraction of a pixel of sharpness at the edges; the center, where the part sits, is untouched.

Disparity behaves differently from a parallel rig: the zero-disparity surface sits near the fixation distance, 139 mm here, so points nearer than fixation take one sign and farther points the other. Put the work volume around fixation and disparities stay small, which keeps correlation search windows short.

Depth and its noise follow the standard rectified-pair rules:

With f_px = 12 mm / 1.85 µm = 6486 pixels, B = 160 mm and sigma_d = 0.5 pixel, a point at Z = 400 mm carries sigma_Z = 160000 × 0.5 / (6486 × 160) = 0.077 mm, call it 0.08 mm. Half a pixel of disparity noise therefore resolves a 0.5 mm dent with margin — the focal length and the baseline carry the accuracy, not the vergence.

When should I stop converging the cameras myself?

When the vergence is optional, drop it: most machine-vision stereo rigs run 10 to 30 degrees total, because overlap grows, keystone flattens and matching loosens — save 60 degrees for close, obstructed views where the geometry demands it.

A flat part presented straight ahead wants 10 to 20 degrees: the same depth noise from the formula above, with a far larger shared field and milder keystone. Converge hard only when the view forces it — into a cup, past a flange, around a neighbor on the line. Mount both cameras on one rigid beam; flex appears as reprojection error creeping past 0.5 pixel over weeks, and the cure is a stiffer beam or a recalibration. When bare surfaces refuse to correlate at high vergence, projected texture or a built unit from 3D sensors is the honest exit — the MV-DLS300P-04, a laser galvanometer stereo camera, is that class of instrument.

Which checks must pass before I trust the calibration?

Four numbers gate it: mean reprojection error under about 0.5 pixel, no single pair far above it, target corners in every quadrant of both images, and sub-pixel epipolar residuals on real scene points across the overlap.

• Mean error under about 0.5 pixel, with the per-pair histogram free of a tail above roughly 1 pixel.
• Coverage: detected corners in all four quadrants of both images.
• Baseline: the translation vector's length within a few percent of the measured camera spacing.
• Vergence: the relative yaw from the rotation matrix within a degree or two of the built 60 degrees.
• Epipolar test: 20 textured scene points, each with vertical disparity under 1 pixel after rectification — a check beyond reprojection error that catches extrinsics fitting the boards but not the scene.
• Scale: reconstruct a known gauge; a 150.0 mm block reading within about 0.3 mm matches the 0.08 mm depth noise plus distortion residue.

The decision by case: a flat part with room to stand off — converge 10 to 20 degrees and keep the wide overlap. A close or obstructed view that forces 60 degrees — calibrate as described, capture the wedge, crop to it. A drifting mount or bare surfaces that will not correlate — move to a built 3D sensor or add projected texture. To pick the pair, the camera finder filters machine-vision cameras by sensor, mount and shutter, the system configurator walks camera, lens and light in order, and the 3D topic page gathers the rest of the stereo rig pieces. Bring your capture set to the contact page and we will read the per-pair errors with you.