Black depth image on a textureless surface: stereo needs IR texture

Your depth camera looks straight down at a plain white table and the depth image comes back black. Put a hand or a box on the table and depth appears exactly there. That reads like a fault, but on a stereo depth camera it is the matcher reporting that it found nothing to match: black means no valid disparity, not out of range. This page is for anyone commissioning a stereo or structured-light depth camera on a fixture. In a few minutes you will be able to confirm the cause in the IR streams, fix the scene with texture you can remove later, and decide when the surface itself calls for a different sensor.

What black really means on a stereo depth camera

A stereo depth camera does not time light. It watches the scene through two infrared cameras separated by a baseline, and for every pixel a matching algorithm searches the second image for the same texture, shifted sideways. That shift, in pixels, is the disparity, and triangle geometry turns it into distance: the larger the shift, the closer the point.

When the matcher finds no confident shift, it declares the pixel invalid. Most drivers write zero for invalid pixels, some write another sentinel, and the viewer paints the result as a black hole. Black is therefore a verdict of no answer found, and it covers two different situations: the surface is genuinely beyond the range, or the matcher had nothing to match. The pixel value cannot tell you which. The stand-off distance and the IR images can.

If your pipeline deprojects depth into a point cloud, those invalid pixels are the points that vanish. The depth pixel deprojection zero guide handles the downstream side of that.

Why a blank surface starves the matcher

Passive stereo needs the scene to carry texture: edges, grain, print, dirt. A depth camera adds an infrared projector that throws a fixed dot or speckle pattern over the scene, so even a fresh matte surface carries something matchable. That is active stereo, and it works only while the pattern arrives with contrast. Three things take the contrast away:

• The pattern never arrives. The projector has its own optics and its own field, narrower than the camera's field of view near the edges. It fades toward its boundary, so the corners of the frame lose the pattern first; non-present texture in the upper corners is the classic signature.
• The pattern clips. A short stand-off, a glossy patch or a projector power set high saturates the infrared sensor, and a clipped pattern is as flat as a missing one.
• Ambient infrared washes it out. Direct sunlight and incandescent or halogen lamps carry strong near-infrared, which lifts the floor under the dots until contrast is gone.

In all three cases the IR images go locally flat, no unique shift exists, and the region prints black. Wave a hand through the scene and you add edges, creases and random texture, so depth appears exactly over the hand. Nothing in the camera moved; the scene gained the input the matcher was missing.

The five-minute test: look at the IR images, not the depth

The IR pair settles the question faster than any setting change. Run it at the real mounting distance, under the lighting you actually run.

1. Open both raw IR streams and view them as plain grayscale. A driver that publishes each IR camera over ROS shows them as topics such as /camera/ir/image_raw and /camera/ir2/image_raw; on other SDKs, look for the infrared or left-camera and right-camera image streams.
2. Read the black region in both IR images. Visible dots or speckle mean texture exists there, so matching should work and the suspicion moves to exposure, the projector enable or your processing. Flat gray with no dots means the pattern is absent. Clipped white means the sensor is saturated.
3. Sweep a textured object through the region. A scrap of corrugated cardboard is the standard bench tool. Depth that follows the cardboard while the bare surface behind it stays black proves matching is healthy and the surface is the cause.
4. Aim at something truly beyond range. Far from the camera the pattern fades, matching degrades, and depth goes black as well. The flat look is the same as a texture failure, which is exactly why the stand-off distance, not the pixel color, has to arbitrate.

The decision rule: a black region, at a stand-off inside the camera's range, whose IR images are flat, is a texture failure, and the fixes below apply. Dots visible in the same region point at settings instead, starting with exposure and the projector enable. If both IR images are black everywhere, that is a different fault entirely: see why a stereo infrared camera shows a black image.

Worked example: depth from disparity on a table at 700 mm

One number drives the whole test: the stand-off. Take a tabletop cell where the camera hangs 700 mm above the surface, measured with a tape. To make the arithmetic concrete, use illustrative optical values typical of bench stereo modules: baseline B = 50 mm, focal length f = 1.2 mm, pixel pitch p = 3 µm. Assume the module's printed depth range includes 700 mm.

The rule in words: stereo converts the matched shift d into distance through Z = B × f_px / d, where f_px is the focal length expressed in pixels.

The arithmetic:

• Focal length in pixels: f_px = 1.2 mm ÷ 0.003 mm = 400 px
• Scale term: B × f_px = 50 mm × 400 px = 20,000 mm·px
• Shift the matcher must find at the table: d = 20,000 mm·px ÷ 700 mm ≈ 28.6 px
• Scale cross-check: d = 40 px gives Z = 500 mm, d = 8 px gives Z = 2,500 mm, and d = 0 leaves the equation with no finite answer

The scale term carries the sensitivity. Double the baseline to 100 mm and the term doubles to 40,000 mm·px, so the same table demands a 57 px shift; bigger shifts are easier to match at range, at the cost of near-field overlap between the two fields of view.

On a blank tabletop, every candidate shift returns the same flat patch, so nothing marks the true 29 px shift as better than its neighbors. The matcher reports no valid d, the depth equation never runs, and the pixel prints black even though the table sits well inside the range.

The decision and its threshold: black at 700 mm, with flat IR images, inside range, is a texture failure. Spend the next hour on the scene — texture, exposure, projector aim — or move to a sensor that brings its own signal, as below.

Fixes, in the order to try them

1. Add matte, random texture where you need depth. Matte gaffer tape, printed random-speckle targets and removable sticker patches all work. On glossy parts use chalk-based powder or a removable matte spray. Random beats periodic: with a regular grid the matcher can confuse the true shift with a shift by one whole period, and depth comes back at multiples of that spacing.
2. Aim the pattern at the work area. The projector's field and the camera's field of view are separate optics. Place the target where the IR image shows dots, crop the region of interest to that zone and keep the stand-off inside the band where the pattern still carries contrast. The 3D camera working distance and positioning guide covers that placement geometry.
3. Tune exposure for pattern contrast. Watch the IR pair while you adjust. Shorten exposure until clipped white patches recover; lengthen it if the dots go faint; shade sunlight or halogen lamps that lift the infrared floor.
4. Check the emitter before blaming the algorithm. No dots anywhere means the projector is off, blocked or failed: confirm the projector enable in the driver, clean the window, then look further. An external IR illuminator can restore coverage, but its wavelength has to clear the IR-cut filter on the camera's color module — the IR projector and IR-cut filter guide walks through that trap.

5. When the part must stay clean, change the sensing principle. A surface you cannot mark needs a sensor that carries its own signal:

A projected structured-light camera floods its field with a coded pattern, so an unmarked panel or casting still returns depth across the full area.

A time-of-flight camera measures the return time of its own light in every pixel, needs no matching, and reads depth on any surface that reflects enough light back.

For a profile across a blank part, a laser profiler triangulates from its own scanned line, so the surface's texture never enters the measurement.

For blank cartons and totes in volume and logistics work, a multi-DOE speckle structured-light 3D camera is built for exactly those featureless box surfaces; see the multi-DOE speckle 3D camera for that variant.

When black is the right answer

• Beyond the far limit. Pattern strength falls with the square of the distance, matching degrades before any hard cutoff, and black is the honest verdict there. Shorten the stand-off or step up to a longer-range sensor; the long-range stereo depth accuracy guide shows what a wider baseline buys at distance.
• Along the frame borders. Stereo leaves an invalid band at the edges by geometry, whatever the surface. That black is expected; the stereo depth invalid band guide explains the band and how much to crop.
• On a mirror-like patch. A specular highlight clips the infrared locally even at a healthy stand-off. Angle the camera, matte the spot, or accept a hole there.
• Around depth edges. Mixed-depth boundaries produce flying pixels, points that belong to neither surface, rather than clean black. The flying pixels guide covers that signature.

Black regions are one artifact family. Output that jitters frame to frame is another, with its own causes, taken apart in this stereo depth jitter and drift troubleshooting guide.

The decision, per case

Two of the five rows fix themselves in the scene, one is a settings problem, and two are correct behavior. The IR pair puts you in the right row in minutes.

A short checklist for the next station

• Pull up both IR images before judging any depth result; they are the ground truth for matching.
• Lay a matte, mid-gray, randomly speckled mat on the fixture bed if the cell ever needs depth on the background.
• Keep glossy paint and periodic patterns off every surface the cameras must read.
• Aim the projector's field onto the work volume and crop the region of interest to where dots are visible.
• Log the black-pixel fraction per frame; a slow rise usually means a dirty projector window or an emitter fading with heat.
• Commission with the cardboard test: depth that follows the scrap and nothing else means the scene lacks texture, not the camera.

If the scene decides the sensor, start in the 3D sensors category or the camera finder, and the configurator's camera step narrows by interface, mount and range. More failure modes in this family sit on the 3D topic page. If you are staring at a surface you can neither mark nor replace, contact us with the material and the stand-off — those two facts usually settle the sensor type.