ND filter sizing for stereo depth cameras in direct sunlight

Direct sunlight returns an empty depth map from a projector-based stereo camera: exposure and gain sit at their minimum, the sunlit texture still clips, and the disparity search finds nothing left to match. The fix is optical, not a setting. You will measure how many stops of attenuation your scene needs, choose between a matched ND filter pair and an 850 nm band-pass, fit the filters without wrecking the stereo calibration, and re-verify depth on a plane target with a number instead of an eyeball. The arithmetic works for any structured-light or speckle stereo module with bare silicon imagers, outdoors or under bright artificial light.

Why the depth map dies in direct sunlight

A stereo imager without an IR-cut filter is bare silicon: it responds across the visible band out to roughly 1000 nm, and the sun is bright over that entire range. The structured-light projector, in contrast, emits a narrow pattern near 850 nm; some designs use 940 nm. Outdoors the broadband solar component swamps the narrowband pattern. Pixels saturate, a saturated pixel carries no texture and no dots, and the correlation stage has nothing to lock onto, so the depth map drops out wholesale.

Exposure and gain cannot rescue you once both sit at their floor; the image-quality settings that fix indoor overexposure are exactly the ones that run out of travel here. The camera-side measures for outdoor work are in outdoor machine vision image quality; this page is the optics half of that problem. If you are still choosing the sensor itself, start with industrial 3D camera selection and the 3D sensors category, or go straight to the camera finder.

ND filter or 850 nm band-pass: the decision

A plain ND filter multiplies the sun and the projector dots by the same factor, so it cannot improve dot-to-sun contrast. It restores exposure headroom once the imagers are pinned at their minimum, which is exactly what you need when depth is healthy in shade and dies only when sunlight clips the imagers. A band-pass centred on the projector wavelength does more: it rejects the solar continuum by a large factor while passing the dots almost untouched, so it improves contrast itself. The cost is scene texture: with visible light removed, a matcher that also uses ambient texture works only within the projector's reach, a few metres for typical pattern projectors.

For the density mathematics of a single filter on a lens path, see neutral density filter optical density. Filter hardware lives in the lens accessories range, found through the accessories finder.

Size the ND pair from your own histogram

Lock both imagers first: same manual exposure, same gain, at the values that showed the failure. Per-camera auto exposure drifts independently and hides the effect you are measuring; the rig-side fix is in set auto exposure per stereo camera in a depth rig.

Read the raw IR histogram over a sunlit patch of the actual target surface, and set your mid-grey target with headroom, 140 on an 8-bit imager:

Worked example, with illustrative numbers: the histogram mean over the sunlit region is 250 at minimum exposure, target 140. A = 250 / 140 = 1.79. stops = log2(1.79) = 0.84. With the margin, 1.84 stops. The next standard filter up is ND4: 2 stops, OD 0.6, 25% transmission. On a 10-bit imager the same ratios apply against a clip value of 1023.

If the imagers still clip through ND16, the bright patch is a specular sun glint rather than ambient light: re-aim the camera or fit a shade. Density beyond that point pulls the dots below the correlation floor, and depth dies with the sun solved.

When the projector must win: size an 850 nm band-pass

Size the band-pass from how much sun it removes against how much projector light it keeps. Across the 400 to 1000 nm band that bare silicon responds to, clear-sky ground irradiance is of the order of 500 W/m². Near 850 nm the ground-level solar spectrum delivers about 1 W/m² per nanometre, so a passband of width Δλ admits roughly Δλ nanometres' worth of sun:

A 10 nm class band-pass therefore cuts the sun-weighted ambient by about 50, which is OD 1.7. A typical hard-coated interference filter passes 60 to 80% at its centre; at 70% the projector loses log2(1/0.7) ≈ 0.5 stops. Dot-to-sun contrast improves by about 50/1.43 ≈ 35, where any plain ND leaves it unchanged. Widen to 20 nm and rejection halves to about 25. Narrow to 3 nm and rejection rises, but an interference filter's centre shifts to shorter wavelength with tilt, so alignment tolerances get tight.

Solar irradiance also dips near 940 nm, the atmospheric water-vapour absorption band, so a 940 nm projector with a matched band-pass buys a little extra margin.

The trap runs the other way: a visible-cut long-pass at 940 nm passes almost nothing at 850 nm and blanks an 850 nm projector entirely. The mirror-image failure, an IR-cut filter blocking a projector you planned to use, is covered in IR projector and depth camera.

In the shop's filter range, the LS-LXR-0225 is a band-pass filter at 850 nm with an M30.5 x 0.5 thread, and it fits this job.

Adapter rings bridge any other barrel thread to it, and the extras step of the configurator builds the whole stack.

Fit the filters to a stereo pair

What you need: access to the raw IR stream and its histogram, a matched filter pair, a planar check target, and a fixed shooting distance with a tape or a fixture.

Work in this order:

1. Fit a matched pair — same density, same glass thickness, same coating, one filter per imager. Different attenuation between the two images biases the disparity search even when each image looks fine alone.
2. Mount both filters square to the optical axes and at the same tilt as each other. A plane-parallel plate of thickness t and index n shifts focus by about t(n−1)/n and, tilted by θ radians, shifts the image laterally by about tθ(n−1)/n. For 1 mm glass at n = 1.52 that is a 0.34 mm focus shift and a 30 μm lateral shift at 5°; make any residual shift common to both imagers.
3. Refocus if the module allows it, then lock the focus. If focus is fixed at the factory, prefer a thin filter around 0.5 mm and re-check corner sharpness in the IR stream. Mount and flange arithmetic is covered in camera lens adapter focus.
4. Choose filters coated on both faces. Uncoated glass in the projector path reflects a ghost of the dot pattern back into the imager, which correlates as false structure.
5. Re-shoot the histogram: mean near 140, no clipped patches, dots clearly visible on a mid-grey surface.
6. Verify depth on the plane target at your fixed distance and record the result.
7. If the filtered result is worse, re-check focus before touching calibration; a focus shift, not the filter, is the usual cause.

For the before/after comparison, a planar target at a fixed measured distance is the reference: the LS-LXR-0029 is a 76 x 76 mm dot-grid calibration board with 0.5 mm dots, small enough to sit on a bench fixture and return a clean plane fit.

Target size and shooting distance interact — checkerboard calibration at distance covers choosing a target that fills enough of the frame — and when you need a formal acceptance figure for the filtered rig, see ambient light rejection specification.

Common mistakes

• One filter on one imager. The pair sees different contrast and the matcher pays for it.
• Expecting plain ND to improve dot contrast. It scales sun and dots equally; only a band-pass shifts the ratio.
• Chasing a sun glint with more density. Past ND16 the dots reach the correlation floor before the glint gives up; re-aim or shade instead.
• Uncoated glass facing the projector. Back-reflections create ghost dots that correlate as false structure.
• Blaming calibration for a focus shift. Check sharpness across the IR frame before recalibrating anything.
• Tilting a band-pass for mechanical clearance. The passband centre moves to shorter wavelength with tilt and the dots dim unevenly across the field.
• Choosing bandwidth on rejection alone. A 3 nm filter demands squareness you cannot hold in the field; 10 to 20 nm is the workable compromise.

The decision, and the parts

Read the histogram. If 2 stops of attenuation clear the clip, a matched ND4 pair is the whole fix. If depth is marginal even unclipped, or bright visible sources compete with the projector, the 850 nm band-pass filter buys about 35 times the dot-to-sun contrast for half a stop of projector loss. Either way: matched pair, square, coated on both faces, refocus for the glass, verify on the plane target before the rig goes on the line.

More depth-camera optics and integration guides sit on the 3D imaging topic page, and the contact page is the route for matching a filter to a specific module's barrel thread.