Occlusion bands in depth to color alignment at close range

You align depth to the color camera and the result looks wrong: the fingertip is drawn twice, a strip of the part is pasted onto the conveyor, black bands run along every edge. At a working distance around 0.3 m this is almost never a calibration fault. It is occlusion: the depth and color sensors sit tens of millimeters apart, so they see slightly different scenes, and no software mapping can invent the pixels one sensor cannot see. This guide gives you the geometry to predict the band width, three tests that separate occlusion from a true extrinsic error, and the fixes that work at close range — in that order, so you recalibrate only when the evidence says so.

What the artifact looks like

At a working distance of a few hundred millimeters, depth-to-color alignment produces a recognizable family of artifacts. Strips of foreground color or depth repeat across the background on one side of every edge: a fingertip printed onto the table, a part rim copied onto the conveyor. Black bands appear where one sensor had a view and the other did not. The artifacts hug depth edges rather than image edges, and each edge carries them on one side only — a hole where the mapped stream has no data, a ghost strip where it duplicates foreground onto background. The sides swap when you map in the opposite direction.

One more signature matters: the artifacts move when the part moves. They belong to the geometry of the scene, not to the camera, which is why they rotate with the object while a calibration offset does not.

The geometry behind the band

The depth sensor and the color sensor sit apart inside the housing. Short-range stereo modules carry baselines of roughly 50–95 mm, and the color camera sits between or beside the two mono sensors, so the depth-to-color offset d is typically 25–50 mm. Each sensor therefore sees a slightly different scene. Behind every foreground edge lies a wedge that one sensor can see and the other cannot, and no mapping can fill that wedge with true data, because the data was never captured.

The wedge has a width you can compute. On the background plane, a sensor offset d hides a band of width w:

Here Z_fg is the distance to the foreground edge, Z_bg the distance to the surface behind it, and f_px the color camera's focal length in pixels. The band grows in proportion to the depth step behind the edge and shrinks in proportion to the working distance: double Z_fg at the same step and the band halves. When the background sits twice as far as the edge, w reaches d itself — a strip as wide as the sensor offset, tens of millimeters at 0.3 m.

Stereo depth misbehaves at edges in other ways too; the guide stereo depth edge accuracy: why distance deforms edges covers the wider effect.

Worked example: fingertip over a table at 0.3 m

Assume d = 25 mm and a color stream of 1280 pixels across a 70° field of view, which gives f_px = (1280 / 2) / tan(35°) = 640 / 0.70 ≈ 914. The fingertip sits at Z_fg = 300 mm with the table 30 mm behind it, so Z_bg = 330 mm.

Band on the table: w = 25 × (330 − 300) / 300 = 2.5 mm. Band in the aligned image: 2.5 × 914 / 330 ≈ 7 pixels — a visible smear exactly where the tip overlaps the table.

Now let the finger hang over a tote whose far wall sits at 600 mm: w = 25 × (600 − 300) / 300 = 25 mm, which maps to 25 × 914 / 600 ≈ 38 pixels — a ghost strip 25 mm wide that writes fingertip depth onto tote texture.

Finally, move the same part out to 1.5 m with the table still 30 mm behind the edge: w = 25 × 30 / 1500 = 0.5 mm, or 0.5 × 914 / 1530 ≈ 0.3 pixels — invisible.

The decision follows directly. At 0.3 m, any depth step behind an edge produces a band of several to tens of pixels. At 0.6–1.0 m the same 30 mm step costs 1.3–0.8 mm, roughly one pixel. Past 1.5 m the artifact disappears for small steps, and only deep backgrounds keep it alive.

Likely causes, in order of probability

1. Occlusion geometry. The default suspect at close range with depth steps in the scene. Signature: bands hug edges, sit on one side, rotate with the part.
2. Zero-depth holes. A fixed, short depth exposure at 0.3 m — often chosen to stabilize color values for marker detection — starves the IR image and punches holes in the depth map. The mapping skips every hole, and the aligned image tears around each one. Signature: bands coincide with zero-depth patches, and agreement improves when exposure recovers.
3. Stream resolution mismatch. Mapping expects a listed pairing of depth and color formats; an unusual pairing resamples one stream and smears the overlay. Signature: misalignment appears only after a format change. The guide depth to color alignment breaks up: match stream resolution first covers that failure.
4. Extrinsic calibration residual. A small uniform shift of the whole depth frame relative to color. These modules ship with extrinsics optimized around working distances of about 1–3 m, so at 0.3 m a residual of a few millimeters is plausible. Signature: the same offset everywhere on a flat target.

If you have shifted the disparity offset to pull the minimum range closer, treat all four causes as magnified: the shifted geometry is even more specific to one distance than the factory setup.

Three tests before you touch calibration

Each test takes minutes and needs no more than the camera and a printed target.

1. Rotate the part 90°. Occlusion bands belong to the sensor axis. Turn the part and the bands turn with it, staying on the far side of each edge. A calibration offset stays fixed in the image no matter how the scene rotates. If the artifact followed the part, it is occlusion.
2. Substitute a flat target. Aim at a flat printed target or a plain wall at the same distance. An offset that is uniform across the whole image is a calibration residual. If the overlay lines up on the flat target, your extrinsics are fine at this distance and the bands at the real part are data holes.
3. Restore depth coverage. Switch the depth stream back to auto exposure, lengthen the exposure, or add machine-vision IR illumination mounted out of the operator's line of sight. Watch the zero-depth holes. If the bands shrink as coverage returns, the cause was missing data, not geometry.

Fixes that work at close range

1. Fix coverage before geometry. Restore auto exposure on the depth stream or add IR illumination until the depth image holds the whole part with no holes. Coverage faults masquerade as alignment faults; the guide depth coverage changes after startup: check IR exposure shows the check.
2. Map in the frame your code consumes. If your algorithm works in depth coordinates — robot picking, volume checks — align color to depth: the depth values stay native and the gaps land in the color image, where a mask is easy to apply. The guide on depth to color alignment without losing depth precision compares the cost of each direction.
3. Mask the computed band. You already know w. Mask the band plus a 50 percent margin around every depth step — about 4 mm around the fingertip edge in the worked example, where the band is 2.5 mm — and convert to pixels with band_px = w × f_px / Z_bg so the mask matches the image. Downstream matching and measurement never read the corrupted strip.
4. Move back or shrink the step. At 0.6 m the 30 mm step costs 1.3 mm; at 0.9 m it costs 0.8 mm. If the task tolerates 0.6–1.0 m, the camera works inside its calibrated envelope and the bands fall to about a pixel. If the task truly forces 0.3 m, choose hardware built for near range; the guide close range depth camera for 1mm resolution at 5-15 cm shows what that demands.
5. Recalibrate last, only on evidence. If the flat-target test shows a uniform offset beyond tolerance, run your own extrinsic refinement at the working distance. A flat printed checkerboard target and a hand-eye style refinement at the actual distance works: capture several poses of the target across the frame, solve for the depth-to-color transform, and verify on a fresh target pose before trusting it. Publish nothing from a fit made against a band that rotates with the part.

When it really is calibration

Two conditions must hold before you edit extrinsics. First, the flat-target test shows the same shift everywhere in the image. Second, that shift exceeds your tolerance — one or two pixels at 0.3 m is within the residual to expect, because factory calibration on this class of camera is optimized for working distances of about 1–3 m and close range sits outside that envelope.

A few millimeters of uniform offset at 0.3 m is a genuine calibration problem, and your own checkerboard refinement at the actual distance fixes it. Occlusion beside it is not fixable: after a successful recalibration the uniform shift disappears and the edge-hugging bands remain, unchanged. If your SDK's viewer offers a point-cloud occlusion filter, use it to confirm the diagnosis in preview; it hides occluded points on screen and changes nothing in the frames your code receives.

Next step

Run the three tests, restore coverage, and recompute w for your own d, Z_fg and Z_bg before changing any hardware. If your working distance is genuinely too close for the offset of your housing, compare housings with a shorter minimum range in the 3D sensors category or the camera finder. The system configurator starts at the camera step and checks the rest of the chain against the same numbers, and the 3D topic page collects the other depth-camera diagnostics on this site.