SSD block matching for stereo depth: how the disparity map is computed

Two stereo depth modules with the same resolution can return very different depth maps. The cause is usually the matching engine rather than the pixel count. Most industrial stereo depth devices compute disparity with SSD — sum of squared differences — block matching, wrapped in vendor post-filters that are rarely published. This guide explains what that algorithm actually does: the correlation window, the disparity search range, the sub-pixel step, and the formula that turns disparity into depth. You will be able to estimate the depth error of any stereo pair from three numbers — focal length in pixels, baseline and working distance — and decide when passive matching is enough, when the scene needs a projected pattern, and when a ToF camera or laser profiler is the better engine.

What SSD block matching does, step by step

The input is a rectified stereo pair: two images of the same scene, resampled so that a feature sitting on row y in the left image also sits on row y in the right one. Rectification is what makes the search one-dimensional. The engine only has to look along the row for the offset that lines the pair up, and that offset is the disparity.

For every pixel the engine cuts a small window out of the left image — commonly 5×5 to 15×15 pixels — and slides the matching window along the right image's row through every candidate offset d in the search range, scoring each position:

Squaring is the point. A large grey-value mismatch is penalized quadratically, so the winning minimum is sharp and the match distinctive. Sibling costs do the same job with other arithmetic. SAD sums absolute differences and costs less silicon. Census transform compares bit patterns and tolerates gain differences between the two sensors. Normalized variants survive exposure mismatch. Production engines are an SSD or SAD core plus post-filters — uniqueness ratio, left–right consistency, hole filling — and the exact recipe is rarely published, so ask for the cost function, the default window size, the search range and the sub-pixel setting whenever you compare two modules.

The output is a disparity map: one value per pixel, an integer before refinement. That one-value-per-pixel design is why stereo depth arrives in quantized steps rather than a smooth continuum.

From disparity to depth, and why error grows with z²

With f in pixels and b and z in the same length unit, pinhole geometry ties the three together:

The third line is the derivative of the first. Doubling the working distance quadruples the quantization step; doubling the focal length or the baseline halves it. Sub-millimetre ambitions at metre-scale ranges fail on this line, which is why vendors print focal length and baseline on the first page of the data sheet.

Getting f_px right is where integrations go wrong, because data sheets quote focal length in millimetres:

The conversion mechanics are walked through in stereo disparity to depth conversion: set the depth window, disparity map to depth conversion: bands are not fixed mm shows why the quantization bands move with range, and stereo disparity scale and calibration for depth estimates covers builds whose millimetres come out uniformly scaled.

Worked example: depth step at 1 m for f_px = 1280 and b = 50 mm

The inputs are illustrative of a compact industrial stereo module: f_px = 1280 px (a 2560-pixel-wide sensor processed at half resolution), b = 50 mm (the fixed sensor-to-sensor baseline) and z = 1000 mm (the mounted working distance). First the disparity at the target, then the quantization step:

One disparity step at 1 m moves the reported depth by 15.6 mm. Sub-pixel refinement fits a parabola through the three lowest cost samples; on clean industrial imagery it typically lands between 1/10 and 1/2 px, and 1/4 px is a fair planning value, which shrinks the step to roughly 3.9 mm at this range. The same arithmetic across the working band:

The decision: treat the refined step as the depth repeatability you can hold. At 1 m this module delivers roughly ±4 mm repeatability — pose, presence and bin-picking territory — while a ±1 mm dimensional check needs a change. Doubling the baseline to 100 mm halves the raw step to 7.8 mm (about 2.0 mm refined); doubling f_px halves it again; past that the engine type has to change. Quantization also biases the measured height of thin features, not just its noise: stereo depth height overestimation from disparity quantization works that case through.

Window size: the trade between smooth depth and sharp edges

The window side sets what the matcher can separate. A 5×5 to 7×7 window tracks fine detail and keeps depth edges crisp, but its cost curve is shallow, so disparity is noisier. An 11×11 to 15×15 window — the common factory default — averages over more texture, smooths the noise and bridges sparse detail, but it bleeds depth across edges. A thin rib sits inside windows that also see the background, and the two depths average into a wrong middle value. Windows above 21×21 are for nearly featureless surfaces judged on gross shape alone.

Two working rules follow. Keep the window side clearly smaller than the smallest feature you must measure in depth — a 15×15 window cannot resolve a 10-pixel rib. And remember compute scales with window area: doubling the side quadruples the cost terms per pixel.

Search range: set it from the nearest surface

The engine only finds disparities inside its configured range. The nearest measurable distance follows from the same geometry: d_max = f_px × b / z_min. For the module in the worked example, a surface at 500 mm demands disparities up to 128 px; a range capped at 64 leaves every close pixel unmatched, arriving as holes in the depth map.

Each extra disparity level also adds compute linearly and one more chance of a false minimum in repetitive texture, so size the range to the real scene rather than to the maximum. SDKs expose this as a disparity shift or minimum-range setting: setting the minimum range from your script shows a working example.

Where the cost curve breaks: texture, occlusion and gain

SSD needs signal in the pixels. On an unpatterned matte surface every candidate offset scores about the same, the minimum is chosen by noise alone, and depth on that patch is arbitrary. The production fix is active stereo: project a speckle or stripe pattern so every surface patch carries texture the matcher can lock onto. Diffuse side lighting that pulls out surface grain, plus temporal averaging, helps on lightly textured parts — stereo depth map noise: smooth raw depth into usable data covers the filtering side.

Structured-light 3D sensors ship exactly that projector, and a galvanometer can instead steer a laser pattern to wherever the part actually sits:

Occlusion is the second failure mode. Near a depth edge, part of the left window has no partner in the right image, the match slides, and foreground objects fatten in the depth map by a few pixels. Smaller windows and narrower baselines shrink the effect; left–right consistency checks reject the worst pixels.

Gain or exposure mismatch between the two sensors biases SSD toward the brighter image. Normalized or census-based costs tolerate it; otherwise lock both cameras to the same exposure. Repeating texture — thread pitches, vent grids — produces several equal minima, which uniqueness and left–right filters are meant to reject. Whatever the cause, the engine reports which pixels it trusts, so decode those flags instead of trusting the raw map: 3D camera confidence image values: decode bitflags and mask bad pixels. Fuzzy sheets and translucent films fail for a different reason again — the surface itself scatters the light — and close range depth dropout on fuzzy surfaces and thin parts walks that case.

Some modules add a learned refinement stage on top of block matching. When its depth output arrives speckled, the mode settings are the first suspect, and this walkthrough on noisy neural stereo disparity traces them in order. Flicker that survives the matcher itself usually lives in the rig — mount shake, timing, IMU fusion — which this stereo camera depth jitter troubleshooting guide covers.

What the math costs: why the engine lives in silicon

Count the work for a 1280 × 1024 image, a 128-level search range and a 9×9 window: 1,310,720 pixels × 128 candidates × 81 window terms ≈ 1.4 × 10^10 cost terms per frame. At 30 fps that is roughly 4 × 10^11 terms per second — orders of magnitude beyond a CPU core in real time. This is why depth cameras build the SSD engine into an FPGA or ASIC, and why PC-side software matching trades resolution or search range for speed.

When block matching is not the engine in the box

Block matching is the default stereo engine, not the only one. A time-of-flight camera returns depth per pixel from the phase of modulated reflected light; its depth noise stays roughly constant with range — until the return signal degrades — instead of growing as z². On bare metal or black plastic at a fixed mounting height, that profile can win outright:

A single-camera structured-light engine projects fringes and decodes phase, buying close-range accuracy without a second camera's occlusion geometry, at the cost of a few captured frames per scan. A laser profiler triangulates a swept laser line and sets the accuracy benchmark at short range, paying with scan time.

The shop's 3D sensor range spans all four engine types:

Checks before you commit to a module

• Ask the vendor for four numbers: cost function, default window size, disparity search range, and whether sub-pixel refinement is enabled. Without them, two modules cannot be compared on depth quality.
• Compute Δz = z²/(f·b) at your worst-case range with the module's f_px and b, and read the refined step as the repeatability you can hold. If it exceeds a quarter of your tolerance, change optics or engine now, not after installation.
• Test on the real part, not a chart: matte black, polished, fuzzy and translucent surfaces each break the cost curve differently.
• Building the pair from area scan cameras? Match mono sensors with global shutter, trigger both exposures from one hardware line, hold the baseline rigid, then calibrate — stereo calibration with cameras of different resolutions covers the mixed-resolution case.
• Check rectification before trusting any disparity: rows must line up first, per stereo disparity map troubleshooting after rectification and how to inspect stereo rectification map values across toolkits.

The decision, per case

• Volume, pose and bin picking at 0.5–2 m on textured or pattern-projected parts: an SSD stereo module is the right engine. Budget Δz ≈ z²/(f·b) with the module's real f_px and b and it will not surprise you.
• Dimensional checks around ±1 mm at 1 m: passive stereo at compact optics does not reach there; move to structured light or a laser profiler.
• Bare, dark or shiny parts at a fixed range: weigh a ToF camera's flat error profile against a profiler before defaulting to stereo.
• If the underlying question is whether the task needs depth at all, start from 3D vs 2D machine vision system: when to choose 3D.

Shortlist hardware in the camera finder, start a build in the configurator, or keep the depth-math walkthroughs handy on the 3D vision topic page.