A stereo depth rig that produces a clean point cloud indoors and a speckled, wobbly one in bright light usually has nothing wrong with its sensor. The symptom is a rectified IR image full of salt-and-pepper noise, often worst in the lower rows, and a point cloud where smooth surfaces ripple by centimetres in z.
The cause is frequently the autoexposure loop: in a bright scene it shortens exposure until it reaches the shortest exposure the camera tuning allows, cannot shorten further, and pushes gain up instead. Gain lifts the noise floor across the image, and the depth engine loses stereo confidence first where readout artefacts are strongest. This guide shows how to confirm that diagnosis, fix it with controlled exposure, and guard the fix so a lighting change does not silently degrade the cloud again. It is written for anyone integrating stereo or structured-light depth cameras into inspection, agriculture or robotics systems.
Symptom: noise at the bottom of the IR image and curvy surfaces in the cloud
Three signs travel together:
- The rectified IR images from one or both stereo sensors show bright speckle, concentrated toward the bottom of the frame.
- The point cloud has dropout holes where the IR noise is worst, and smooth leaves or panels bend by 5-20 mm in z between adjacent points.
- The problem appears at certain times of day or under strong lamps, and disappears under dimmer light or manual exposure settings.
Post-processing filters reduce but do not remove it. Speckle, temporal and spatial filters all operate after stereo matching, so they delete noisy pixels instead of matching them. A smooth leaf that matched badly stays unmatched, and the fill around the hole interpolates across it, which is exactly the impossible fluctuation you see.
For background on how stereo coverage and image quality interact, see depth image field of view: why stereo coverage is smaller.
Likely causes, in order of probability
- Autoexposure at its floor. The AE controller shortens exposure in bright light until it reaches the minimum the tuning permits, often around 10-20 microseconds, then raises gain. High gain plus very short exposure is where readout artefacts and shot noise become visible. This is the most common cause of the bottom-row pattern.
- Scene illumination outside the AE range. Direct sun or a bright lamp drives the loop past its range entirely, so the image is both over-driven and noisy.
- Firmware or SDK version. Readout behaviour has improved across releases on several depth platforms, so an outdated package can make the same hardware noisier.
- A genuine sensor fault. Real sensor defects exist, but they produce noise that is constant over time, independent of lighting, and not reproducible on a second unit in the same scene.
Work down the list in that order; each test below rules one cause out.
Test 1: log exposure and gain over time
Print the exposure time and sensitivity from the IR stream metadata every few frames for several minutes in the failing scene. Two mistakes are common here. First, extracting the exposure with a seconds-only accessor can truncate sub-second values to zero, making manual and AE look identical; use a total-duration readout. Second, reading one frame is not enough: the loop responds continuously, so capture a series.
What the log tells you:
- AE output pinned at a constant minimum value with gain climbing: the exposure floor is your problem.
- Exposure varying but noise still high: suspect over-driven illumination or a version issue.
- Noise present even at moderate exposure and gain: test another camera before blaming the scene.
Test 2: switch to manual exposure and compare
Set a manual exposure at or below the AE floor you measured, with the lowest gain that still gives a usable image, and capture the same scene. If the IR image cleans up and the z-wobble shrinks, you have proven the cause in one minute. Toggle AE on again and confirm the noise returns, so you know it is the control mode and not something else that changed.
A projected structured-light camera such as the MV-DPS200P-02 projects its own IR pattern, which keeps the effective illumination constant and reduces how hard the AE loop has to work on dark or low-texture surfaces.
Test 3: rule the sensor out (or in)
Run a second camera of the same model in the same scene, at the same time, with the same settings. If both show the pattern and both clean up under manual exposure, the cause is systemic, not a defective unit. A sensor that stays noisy at moderate exposure and gain, in a dim scene, across firmware versions, is the case that justifies a warranty return; contact us with the exposure and gain logs attached.
Also update the camera firmware and SDK to the current release before concluding anything. Readout timing changes have measurably reduced this artefact on some stereo sensors.
Fix: controlled manual exposure with an AE watchdog
If you control the lighting, reduce it: shade the work area, dim the lamps, or attenuate the optics. AE then stays inside its range and the problem disappears with no software changes.
If you cannot, as in plant imaging where altering daylight is not an option, run manual exposure and guard it:
- Set exposure to the shortest value the camera accepts and gain to the minimum that gives enough IR contrast for matching.
- Periodically switch to AE for a short evaluation window, compare depth confidence and fill rate against the manual baseline, and switch back.
- Warn when AE reports its minimum exposure, because that is the moment noise will climb; log it so you can correlate point-cloud quality later.
That factor of eight is why the AE image looks speckled while the manual image looks clean in the same light: the AE loop traded exposure for gain and lost.
For a related failure mode where IR illumination itself is the problem, see stereo depth lighting optimization for dark surfaces.
Verify with depth confidence and fill rate
Do not judge the fix on the point cloud render alone. Two numeric checks separate input noise from genuine geometry:
- Confidence map. Display the per-pixel depth confidence. Under the AE floor the noisy regions score poorly; under manual exposure the same regions score well. This is the fastest objective check.
- Fill rate. Count valid depth pixels as a fraction of the region of interest on a flat target at fixed range. Track it over the day; a drop that follows the sun angle points back at exposure control, not at the part.
If z-wobble persists on well-matched pixels after both checks pass, the residual error is geometric rather than exposure-related; calibration is then the suspect, starting with the checks in why a laser triangulation point cloud is tilted: calibration checks.
Choosing hardware that avoids the problem
If your scene lives permanently in bright or variable light, pick the sensing method before fighting exposure settings:
| Approach | Behaviour in bright light | Best fit |
|---|---|---|
| Passive stereo | Needs scene texture and light; AE range matters most | Indoor, controllable light |
| Projected structured light | Own IR pattern; AE sees a stable target | Dark, low-texture parts |
| Laser profiling | Its own illumination line; tolerant of ambient light | High-accuracy surfaces, outdoor-capable |
A laser profiler such as the MV-DLS300P-04 generates its own illumination geometry and is far less sensitive to ambient light than a passive stereo pair, which is why many outdoor and high-precision applications move to profiling rather than tune stereo. Browse the 3D sensors category to compare the approaches, or the infrared cameras range if your application needs a dedicated IR or thermal channel alongside depth.
Before ordering, run the geometry numbers for your working distance with the camera resolution calculator so the exposure fix does not leave you short of pixels across the field of view.
Prevention checklist
- Log exposure and gain continuously in production, not just during commissioning, so a lighting change is visible in your data.
- Alarm when AE reports its minimum exposure for more than a few consecutive frames.
- Track fill rate on a known target at start of shift; alert on a drop beyond a few percent.
- Pin firmware and SDK versions, and re-run the bright-light test after any update.
- Where possible, shade the scene or use active IR illumination so the AE loop never reaches its floor.
When the usual answer is wrong
Filters are the usual first response to a noisy cloud, and they are the wrong tool here: they edit the output after matching has already failed. If confidence is low because the input images are noise-dominated, no filter recovers the measurement. Fix the exposure path first, keep the filters tuned for genuine outliers, and your smooth surfaces will stay smooth as the light changes through the day.
To specify a complete depth rig with the right sensor type, optics and exposure control, start at the system configurator or ask us through contact with your scene, working distance and lighting conditions.
Questions
Why does the noise only appear in the lower part of my IR depth image?
The lower rows are read out last on many stereo sensors, so readout timing interacts with a very short exposure. When autoexposure sits at its minimum exposure and the scene is still bright, per-row gain and readout artefacts become visible first at the bottom. A uniform sensor fault would not come and go with lighting, so test that first before assuming hardware.
How do I prove the problem is autoexposure and not a faulty sensor?
Log the reported exposure time and gain over several minutes in the same bright scene, then switch to manual exposure with a short time and low gain and capture again. If the noise follows the autoexposure setting and disappears under manual control, and the fault repeats on a second camera in the same scene, the cause is the exposure control path, not the sensor.
What is a depth confidence map and how does it help?
Most stereo and structured-light depth engines score every pixel with a confidence or error estimate. Pixels formed from noisy IR image data score poorly. Displaying the confidence map shows immediately whether a hole in the point cloud comes from noisy IR input or from geometry the stereo matching cannot resolve, which points to very different fixes.
Should I use ND filters or shade the work area instead of manual exposure?
If you control the environment, reducing the IR illumination or adding a shutter and ND filtering is a robust fix, because the autoexposure loop then stays inside its range. If you measure living plants or cannot block daylight, a guarded manual exposure with a periodic AE check is the better path, since filtering adds a maintenance step and changes every capture.
Can I just rely on speckle, temporal and spatial filters to clean the point cloud?
Filters remove or smooth bad pixels after the depth engine has already failed to match them, so they shrink holes rather than fill them. A 2 cm z-wobble across a smooth surface comes from noisy input images, and no post-filter recreates the lost detail. Fix the exposure first, then tune filters for the remaining outliers.