Noisy stereo point cloud fixes: thresholds, sensing mode, depth range

A single-frame stereo point cloud almost always arrives speckled: floating points in mid-air, fuzz along flat surfaces, junk clusters across the far half of the scene. That is not a broken camera. It is per-frame disparity noise plus filters set too loose to catch it. The fixes below are ordered by how much each one buys: cap the depth range, tighten the confidence thresholds, drop fill mode for the standard mode, feed the matcher texture and light, then merge frames. A worked example shows why the far field dominates the noise, so you can set the range cap from arithmetic instead of trial and error.

What the noise actually is

Passive stereo computes depth by matching pixels between two views, and every failed match lands in the cloud as a bad XYZ point. The usual causes, ranked by how often they matter:

1. Range. Depth error grows with the square of distance, so the far half of a wide-open scene carries several times the noise of the near half.
2. Weak texture. A plain wall or unmarked plastic gives the matcher nothing to lock onto, and it returns plausible but wrong disparities that pass straight into the cloud.
3. Loose filters. A confidence_threshold of 50 with texture_confidence_threshold at 100 — the values most example code starts from — admits nearly everything the matcher produces.
4. Fill mode. A sensing mode built to paint over failed matches manufactures surface where nothing was measured.
5. Light and exposure. A dark scene starves the matcher; motion blur does the same to a moving part.

Downstream stacks hide this: tracking and mapping fuse many frames, so their output looks clean while any single raw frame does not. Noise concentrated at the image border has different causes and its own write-up: troubleshooting stereo depth corner noise in point clouds.

Fix 1: cap the depth range first

depth_maximum_distance is the cheapest fix because it deletes noise instead of fighting it. Set the cap just past the farthest surface you actually measure, not at the sensor's maximum. A real capture of a cell that mattered out to roughly 2.5 m set the cap to 3 m and the large far-field outlier clusters disappeared with that single change; with the cap wide, the same scene collected junk from walls and racks it never needed.

Set the unit system first so the bounds mean what you think: a script that sets coordinate_units to meters and writes depth_minimum_distance = 0.2 gets a 200 mm near bound, which also cuts anything placed too close to the head.

Two limits of this fix. The cap removes points; it does not make the survivors more accurate, because their error is set by range and texture — the worked example below shows how much. And a cap far beyond your working range does almost nothing, which is why the 3D camera working distance and positioning guide belongs to the same decision: mounting distance and range cap are one choice.

Fix 2: tighten the confidence thresholds

Every depth pixel carries a confidence score next to its XYZ. confidence_threshold sets how much of that scale you keep: lower the value to keep only high-confidence pixels, raise it to admit junk. With the range cap already in place, step it down 50, 40, 30 and stop at the lowest value that keeps part edges intact. Edges fail first: one tested cloud lost its edges a step too strict, and artifacts came back a step too loose — the working point was the stricter threshold combined with the 3 m cap, not either change alone.

Then lower texture_confidence_threshold from 100, which admits every texture score. Expect fewer points overall: filtering trades completeness for cleanliness, and that is the correct trade for measurement.

Confidence encodings differ between depth engines — plain scores on one, bit flags on another — and they are not interchangeable. Confidence image values for 3D cameras: decode bitflags and mask bad pixels shows how to read one properly before filtering on it. A learned (neural) disparity engine lowers the noise floor further on textured scenes, at GPU cost, but its failure modes differ from a block matcher's; the noisy DNN stereo disparity troubleshooting page covers that stack.

Fix 3: run the standard sensing mode, not fill

sensing_mode = SENSING_MODE::FILL exists for game engines and augmented reality, which cannot tolerate holes in a mesh, so it interpolates depth across failed matches. The result looks complete and is partly fiction: unmeasured regions come back as smooth surface, and speckle gets painted over instead of filtered. Fill is the wrong default for anyone computing with the cloud.

Run the standard sensing mode in every measurement, filtering and robot-guidance path, and accept the holes. Filter the points that were really measured; render a solid-looking surface afterwards, from the filtered data, if you need it for display.

Fix 4: give the matcher something to match

Texture is the fuel of passive stereo, so the scene side has three levers:

• Add oblique light. A plain surface under angled white light gains shadows and gradients the matcher can lock to. Darkness is not cosmetic here; brightness is signal, which is why a dim cell clouds up as the lights go down.
• Kill glare. Specular highlights blind the matcher and return blobs at wrong depths. The moves that fix shiny metal 3D point cloud capture — polarized or diffuse light, angled heads, exposure discipline — apply to any reflective part.
• Set exposure before gain. Raise exposure while parts stand still; moving parts need a short exposure, because a blurred image pair cannot match at any threshold.

When the scene itself has no texture to light up — unmarked plastic, plain cardboard — project the pattern instead. The MV-DPS200P-02 projects structured light onto the scene, so matching holds on plain surfaces with no scene-side lighting at all.

Fix 5: stabilize, then merge frames

For a static scene, enable depth_stabilization before concluding the depth engine is as raw as it gets; temporal smoothing removes per-frame jitter that no single-frame filter can. Whole-cloud wobble between frames — as opposed to per-pixel speckle — is jitter with its own causes, and the stereo depth jitter and IMU drift guide walks that path.

Merging several frames then beats any single-frame filter: random noise falls with the square root of the frame count, so 16 merged frames carry 4 times less noise and 25 frames 5 times less. Ten to thirty frames collapse single-frame fuzz into one coherent surface for anything that stands still. The alignment and accumulation mechanics are in merge multiple depth frames into one gap-free point cloud. Parts moving during capture smear instead of averaging, so either freeze motion with exposure and trigger, or stay single-frame and lean on the filters from Fix 2.

Judge the result from the exported file, not the render: why exported point clouds differ from the viewer render explains the gap.

Worked example: depth noise versus range at a 3 m cap

Inputs. Scene distances of 1.5 m, 2 m and 3 m — the 3 m figure is the cap that cleaned the reference capture, and 1.5 m a typical part position in such a cell. Baseline B = 100 mm and a 1280-pixel-wide image with a 60° horizontal field of view — a typical compact stereo head, taken as a working assumption so the arithmetic is concrete. Disparity error Δd = 0.2 px on a textured scene, rising toward 0.5 px as texture fades — the usual span of subpixel stereo matching accuracy.

Rule. Stereo depth noise grows with the square of distance and falls with baseline times focal length:

Z is the range in meters, B the baseline in meters, f the focal length in pixels and Δd the disparity error in pixels; the result is in meters. This is the standard small-error estimate for stereo geometry, not a data-sheet bound.

Arithmetic. Focal length from the field of view: f = (1280 / 2) / tan(30°) = 640 / 0.5774 ≈ 1108 px, so B × f = 0.100 m × 1108 px = 110.8 px·m. Then:

• Z = 1.5 m: 1.5² × 0.2 / 110.8 = 2.25 × 0.2 / 110.8 = 0.45 / 110.8 ≈ 0.004 m — about 4 mm of depth noise.
• Z = 2 m: 2² × 0.2 / 110.8 = 0.80 / 110.8 ≈ 0.007 m — about 7 mm.
• Z = 3 m: 3² × 0.2 / 110.8 = 1.80 / 110.8 ≈ 0.016 m — about 16 mm.

Tripling the range from 1.5 m to 3 m multiplies noise by (3.0 / 1.5)² = 4. That is why the far half of an open cloud supplies most of the speckle, and why one range cap cleans more than any threshold sweep.

Decision. If the task needs depth better than ±10 mm, solve Z² = noise_z × B × f / Δd. At Δd = 0.2 px the limit sits at sqrt(0.010 m × 110.8 / 0.2) = sqrt(5.54) ≈ 2.4 m: cap depth_maximum_distance near 2.5 m and let the thresholds trim the last band. As texture fades toward Δd = 0.5 px, the same limit falls to sqrt(0.010 × 110.8 / 0.5) = sqrt(2.216) ≈ 1.5 m — texture loss halves the usable range, which is why the scene-side fix in Fix 4 buys more than any filter. Edges degrade faster than flat surfaces at distance: stereo depth edge accuracy: why distance deforms edges quantifies that, and a wider baseline stretches every limit here outward — long range stereo depth accuracy with a wide baseline.

Decision order and prevention checklist

Run the fixes in this order — each one makes the next sweep shorter:

1. Set depth_maximum_distance just past the farthest real surface and depth_minimum_distance at the nearest one.
2. Sweep confidence_threshold down from 50 until part edges thin, then step back one.
3. Lower texture_confidence_threshold from 100 the same way.
4. Run the standard sensing mode in every path that computes.
5. Add oblique light, kill glare, set exposure — then re-grab before touching thresholds again.
6. For static scenes, enable depth_stabilization and merge 10–30 frames.

And the habits that keep the next cloud clean:

• Record the thresholds with each recipe. A cloud cleaned at one confidence value is not cleaned after someone nudges it in another script.
• Mount the head rigidly. Relative motion between the two views during exposure reads as disparity error, and no filter tells it apart from scene noise.
• Mask the background once. For a fixed cell, a static mask removes the residual far-field points that survive every threshold.
• Separate noise from scale error. A uniformly stretched cloud is calibration, not noise — depth map to real distance calibration for XYZ point clouds fixes that kind of error.

When the scene, not the settings, is the problem

Thresholds and caps can only cut a cloud; they cannot create matches the sensor never made. When parts are dark, curved, moving or featureless, change what measures range:

• Time of flight returns range per pixel without needing scene texture, so dark conveyor parts and volume checks keep working where stereo returns holes. The MV-DT01WMU is a ToF sensing camera built for exactly those scenes.

• A scanned laser line covers small, shiny or profile-critical parts. The MV-DLS300P-04 is a laser galvanometer stereo camera: a scanned laser line profiles the surface directly, without relying on passive texture.

Both trade coverage or speed for the robustness stereo lacks on hostile surfaces. For the wider decision between 2D and 3D in a cell, 2D or 3D camera for conveyor pick and place: choosing the system walks the trade-offs, and the rest of the 3D capture guides sit on the 3D topic page.

To size a sensor against your own part, start from the 3D sensors category or the 3D camera finder, and the configurator steps through camera, lens and mount in order.

If a cloud still resists after these six steps, send us a frame and we will read it with you.