Stereo system accuracy: three validation tests for 3D reconstruction

A calibrated stereo rig is a measurement instrument, and sooner or later someone asks, in millimetres, how good it is. You do not need a more accurate instrument on the bench to answer. Three tests — certified artifacts, physical constraints and jackknifed views — produce trueness, precision and stability as numbers you can accept or reject against limits you compute first. The depth error budget below fixes those limits for a 1,000 mm bench rig end to end, then each test gets its arithmetic, so raw residuals turn into a pass or a fail. The same loop applies to a rig built from two cameras and to a factory-calibrated 3D sensor alike.

What accuracy means for a stereo rig

Accuracy is two numbers, not one. Trueness is the mean error between your measurement and the true value; precision is the scatter around that mean, quoted as a standard deviation. A rig can be precise and wrong: a baseline set 1% too long scales every depth by 1% and adds no scatter at all. It can also be unbiased but noisy, which averaging will hide and a robot gripper will still feel.

Report both, per axis. Lateral error (X and Y) comes from the pixel ray and grows in proportion to distance; depth error (Z) comes from triangulating disparity and grows with the square of distance. A figure like ±0.5 mm without an axis, a distance and a position in the volume says nothing useful — the corners of the working volume are always worse than the centre, typically by several times, because lenses distort there and the surface is viewed obliquely.

Scale deserves its own check, because it enters every measurement at once. Two identical-looking rigs can disagree on the same scene purely through calibration scale; stereo depth scale differs between identical cameras: verify and fix walks through that failure and its fix.

The depth error budget

Two formulas estimate what the rig can and cannot do before any test runs:

Z is the working distance, B the baseline between cameras, f the focal length expressed in pixels (focal length in millimetres divided by pixel pitch in millimetres) and σd the disparity noise of the matching algorithm in pixels. σZ is the 1σ depth uncertainty, σX the lateral one.

The sensitivity list is short and worth memorising:

• Double the distance and depth noise quadruples; halving distance is the strongest lever you have.
• Double the baseline or the focal length and depth noise halves.
• Matching noise runs from about 0.05 px on a sharp projected pattern to about 0.5 px on weak natural texture; 0.2 px is a fair planning value for speckle or textured surfaces.

Calibration quality enters through the same door: if the calibration that produced f and B is weak at your working distance, the real error exceeds the budget. Stereo calibration accuracy: the measurement volume decides covers how the calibrated volume bounds what you can claim.

Worked example: acceptance numbers for a 240 mm baseline rig at 1,000 mm

Inputs, all typical bench values for this class of rig:

• Working distance Z = 1,000 mm — a common station height for parts up to a few hundred millimetres.
• Baseline B = 240 mm — a wide industrial stereo baseline for that distance.
• Focal length: an 8.6 mm lens on 3.45 µm pixels gives f = 8.6 / 0.00345 ≈ 2,500 px.
• Disparity noise σd = 0.2 px — speckle or textured-surface matching, mid-range of the 0.05–0.5 px spread above.

The rule in words: depth noise grows with the square of distance and in proportion to disparity noise, and shrinks in proportion to baseline and focal length.

Arithmetic:

• σZ = (1,000 mm)² × 0.2 px / (2,500 px × 240 mm) = 200,000 mm²·px / 600,000 px·mm = 0.33 mm
• σX = 1,000 mm × 0.2 px / 2,500 px = 0.08 mm
• At 500 mm instead: σZ = (500 mm)² × 0.2 px / 600,000 px·mm = 0.08 mm — halving the distance buys a four-fold improvement.

Result: the rig carries about ±0.33 mm (1σ) of depth noise and ±0.08 mm laterally at 1,000 mm working distance.

Decision, with thresholds: accept the reconstruction at this distance when the mean error on certified artifacts stays within ±0.33 mm and the measured standard deviation stays under 0.5 mm (the budget plus a 50% margin). Investigate when bias exceeds 0.66 mm — twice the budget — because calibration error, not noise, is then the cause. If the process needs ±0.1 mm at 1,000 mm, this rig cannot deliver it: move to 500 mm, where the same numbers give 0.08 mm, or widen the baseline to 600 mm, which still only reaches 0.13 mm.

Test 1: measure ground-truth artifacts

Certified artifacts give trueness directly, in millimetres, with no second instrument involved:

• Bench scale: a gauge block, a step gauge or a calibrated sphere, measured as if it were a production part.
• Production scale: a laser profiler scan of the same part serves as the reference surface that the stereo measurement is judged against.
• Scene scale: surveyed control points, or dimensions on record for a known structure.

Measure the artifact at nine positions — centre plus eight corners of the working volume — three repeats each. Fit a plane or a sphere to each point set rather than reading single points: fitting a sphere to several thousand surface points averages the random noise down and leaves the bias visible. Report mean error (trueness) and standard deviation (precision) per axis against the acceptance limits from the budget.

Place the truth where your tolerance lives. If the process judges part height, compare fitted plane heights; the arithmetic for that comparison is in measuring surface height difference from a 3D point cloud. For scene-scale work where no chessboard will reach, surveyed points substitute cleanly, as set out in stereo calibration without a chessboard: surveyed scene points.

When the reference should be a profile scan rather than a block, a laser profiler is the standard choice:

Test 2: check physical constraints

Real scenes carry constraints you already know, and each one is a free test:

• Floors are horizontal: fit a plane to each floor and check the angle against a known vertical.
• Walls are perpendicular: the angle between fitted planes should return 90.000° within budget.
• Repeated features repeat: storey heights match each other, window pitch stays constant along a facade.

The arithmetic turns residuals into an error bound. Twelve storeys modelled at 3,500 mm each: if the measured heights span 3,497 to 3,503 mm, the rig holds storey height to ±3 mm at building scale. Squareness scales linearly: an angle off by 0.01° puts a corner 1.7 mm out over a 10 m wall (tan 0.01° × 10,000 mm), so an angle residual of 0.01° certifies corner placement to about ±2 mm.

On the bench the same logic uses a granite surface plate (known flat), a machinist's square (known 90°) or a parallel fixture measured from two orientations — the distance between the faces must come out identical both ways. Physical constraints are the only test that works at scales where no artifact exists, which is why they carry building and ship surveys.

They verify consistency, though, not scale: a reconstruction can satisfy every constraint perfectly and still be uniformly 2% small, because a pure scale error preserves every angle and ratio. Pair the constraints with Test 1 or one known dimension to close the scale question.

Test 3: jackknife the views

Hold one view out of the reconstruction, rebuild the scene from the remaining views, then project the resulting 3D points into the held-out image and compare with the actual matched 2D positions. The residuals are honest: that image never took part in the estimate, so they include model error that a calibration RMS would otherwise absorb.

Read the results two ways. The mean residual should sit at zero; residuals that lean in one direction mark an extrinsic error, usually a weak baseline or a rotated camera. The RMS should stay near twice the matching noise — around 0.5 px for the 0.2 px rig in the worked example.

Convert pixels to millimetres with Z/f: at 1,000 mm and 2,500 px, one pixel of lateral residual is 0.4 mm. Depth error displaces points along the viewing ray, so a depth problem hides from a neighbouring view but shows from a different position — which is why the held-out view must look from a different station, not a slight shift of the same one.

A second, cheaper variant reconstructs the same control points from several different image pairs. The spread of the resulting coordinates is the rig's precision, measured in millimetres, with no truth required.

When the usual answers mislead

Three shortcuts read like validation and are not:

• A laser scanner comparison sets an upper bound only. Terrestrial scanners carry their own field errors at the same order as a calibrated stereo rig, so agreement within two to three standard deviations is the most the comparison can certify. Certified blocks and spheres beat any scanner on the bench.
• The calibration RMS is not system accuracy. A 0.1–0.3 px fit on calibration boards says the camera model matches those images; it bounds nothing about depth trueness across the volume. Millimetre claims come from millimetre tests.
• Averaging kills scatter, never bias. A plane fitted to 10,000 points can sit a full millimetre from truth with very small residuals. Bias needs a known value to compare against, which is what Test 1 provides.

Finally, a published accuracy figure is a starting point, not a verdict: every non-contact measurement system earns its numbers in place, a principle the non-contact level accuracy and system validation article applies to level measurement, and the same discipline applies to a stereo rig.

Sign-off checklist

Run these before the rig signs off for production:

• Budget written down: σZ and σX computed for the actual distance, baseline, focal length and matching noise — the numbers every later test compares against.
• Nine-position artifact run: mean and standard deviation per axis, each inside the acceptance limits.
• Scale check: one certified dimension reconstructed end to end within budget.
• Constraint residuals: squareness and repeated-feature spread recorded for the fixture or scene.
• Jackknife run: mean residual at zero, RMS near twice the matching noise, from held-out views at different stations.
• Repeatability: the artifact re-measured after a power cycle and a fixture re-clamp; drift beyond half the budget points to mounting, not math.
• Conditions recorded: temperature and exposure settings, because sensor warming quietly shifts calibration — 3D camera range drops when hot: thermal derating fixes covers the mechanism and the remedies.

Once the rig passes, measurement code built on its cloud can be trusted; measure distance and bend angle from depth point clouds in Python shows the downstream step.

Choosing the sensor these tests protect

The catalog splits its 3D sensors into three families, and the validation loop is the same for each: laser profilers for high-accuracy profiles and reference scans, structured-light stereo cameras for dense surfaces, and ToF cameras for volume and logistics work where speed matters more than fine resolution. Each data sheet quotes working range and accuracy figures; the tests above are how those figures get confirmed in your cell, at your distance, on your parts.

If you are building the rig from components instead, two area scan cameras on a rigid bar with a known baseline behave exactly like the bench rig in the worked example — the budget and the three tests transfer unchanged. Browse the 3D sensors category for factory-calibrated alternatives, start the camera configurator to match sensor and mount, or reach the application team through the contact page with your working distance and tolerance, and the budget comes back as numbers. The rest of the 3D material sits in 3D vision guides.