Depth map change detection without speckle false alarms

Speckle in a depth map can look like continuous motion, even when every object remains stationary. For a fixed stereo camera, start with a valid-depth mask and a measured depth reference rather than treating depth as an ordinary grayscale image. This procedure shows you how to separate geometric changes from invalid measurements, set thresholds from stationary recordings, exclude unwanted surfaces, and confirm events over time. It applies to scene monitoring and inspection, not personnel protection or other safety functions.

Choose a depth-specific change detector

The useful question is not whether a pixel value changed. It is whether a valid geometric measurement changed enough, for long enough, to matter to the application.

Use separate states for unchanged geometry, candidate change, confirmed change, and unknown measurement. Invalid depth belongs in the unknown state; it is not evidence that an object appeared or disappeared.

A fixed camera is a fundamental assumption. Camera movement changes the projected background even when the scene remains unchanged. If the mounting position moves during operation, use the approach described in moving-object detection with a moving camera using depth instead.

1. Stabilize acquisition and decode valid depth

1. Fix the camera mounting, acquisition settings, calibration, and processing configuration before recording a background.

Check for vibration, autofocus changes where applicable, exposure instability, and changes in the stereo matching configuration. A reference captured under one configuration may not remain valid after another configuration starts.

Record the original depth or disparity data, timestamps, and available confidence information. Do not build the detector from an automatically rescaled display image. Display normalization changes the relationship between pixel value and physical distance.

Decode the documented units, invalid markers, and any disparity scaling before performing arithmetic. Reject nonfinite values, out-of-range measurements, and samples below the selected confidence threshold. Keep that validity mask alongside the measurements throughout processing.

For a rectified stereo pair with matching principal-point coordinates, the basic relationship is:

Use the calibrated offset when the rectified projection model requires one. Do not divide an encoded disparity value directly without applying its documented scale.

If static surfaces already show widespread mismatches, follow stereo disparity map troubleshooting after rectification first. A change detector cannot recover geometry that was never measured reliably. Custom stereo pipelines also need consistent live stereo rectification.

2. Build a reference without filling holes

2. Capture a stationary background sequence and calculate a temporal median from valid samples at each pixel.

As a starting trial, capture 60 frames and require at least 40 valid observations before accepting a reference pixel. These are commissioning settings, not universal requirements. Longer recordings may be necessary to represent normal lighting or vibration conditions.

Store the reference depth, the valid observation count, and a reference-valid mask. A pixel with insufficient observations remains unknown. Do not replace its missing reference with zero or a distant artificial surface.

Keep the background reference fixed during initial validation. Automatic adaptation can absorb a slowly arriving object or make an object disappear from the detector after it stops moving. If adaptation is later necessary, update only eligible, confidently unchanged pixels under a documented policy.

Reference coverage also needs checking near image borders and occlusions. The depth image field-of-view guide explains why an intensity image can cover regions without usable stereo depth.

3. Set thresholds from stationary residuals

3. Record a separate stationary sequence and measure the residual between each valid current sample and the saved reference.

Use a held-out sequence rather than evaluating only the frames that created the reference. Otherwise, threshold selection can appear better than its performance on subsequent acquisitions.

A practical starting model uses a robust spread estimate and a minimum physical threshold:

The spread estimate is calculated over time for each pixel, using only valid paired samples. Because it measures residuals against the actual reference, do not add reference uncertainty again without a separate uncertainty model.

Try a multiplier between 3 and 5 during commissioning, then evaluate false alarms on held-out recordings. This multiplier does not guarantee a false-alarm probability: stereo residuals can contain outliers, quantization, and correlated errors. An empirical percentile of absolute stationary residuals is another option when enough representative data are available.

Check the residual median as well as its spread. A persistent offset may indicate camera movement, calibration drift, or changed acquisition settings. Broadening the threshold can hide that fault.

Stereo depth uncertainty generally increases with distance. First-order propagation gives this approximation:

This assumes small disparity errors and the same rectified geometry used above. Prefer measured residuals for final thresholds. Use local surface regions or depth bands when individual pixels have too few observations, but avoid pooling across depth discontinuities.

4. Exclude irrelevant geometry without hiding changes

4. Define the monitored volume and unwanted surfaces in calibrated coordinates before accepting candidate events.

A floor, wall, or fixture can produce many irrelevant residuals. Back-project valid depth into camera coordinates, then apply volume limits or distance-to-plane tests. Confirm the depth convention and use the matching intrinsics; the camera intrinsic parameter conversion guide covers common coordinate mismatches.

For a normalized plane model, signed point-to-plane distance is:

Choose the plane exclusion band from measured plane-fit residuals and the smallest relevant object height. A wide band suppresses more noise but can remove thin objects.

For two-sided change detection, suppress a pixel as plane noise only when both its current point and reference point lie inside the excluded band. If one lies outside, retain it for evaluation. Otherwise, an object disappearing to reveal the floor could be discarded before detection.

Distinguish this from an occupancy detector, which may intentionally consider only objects above a support plane. Inspect the reconstructed geometry using the checks in stereo point-cloud visualization before trusting plane exclusions.

5. Require spatial support and persistence

5. Group candidate pixels into connected regions, then require repeated evidence before confirming a change.

An isolated disparity error should not have the same weight as a coherent object surface. Start with a small connected-region threshold and validate it against the smallest relevant target. For example, a trial threshold of 25 connected pixels rejects smaller components, but it does not establish a constant physical object size across different distances.

Use morphological filtering cautiously. Opening can remove narrow real features, while closing can merge separate objects or bridge invalid regions. Do not fill depth holes merely to make the resulting mask look cleaner.

For temporal confirmation, a practical trial is evidence in at least 3 of the last 4 frames. Require the current observation to be valid and a candidate before issuing a new confirmation. Invalid observations contribute no positive evidence; they are not automatically unchanged observations either.

Apply persistence to spatially corresponding regions or overlapping masks. A strict per-pixel rule can miss moving objects because each surface occupies different pixels between frames.

Use a separate release rule to avoid output chatter. For example, require 3 valid unchanged observations before clearing an existing event. If measurements become unknown, report that condition and apply an explicit timeout rather than silently retaining an old result forever.

Worked example: threshold and confirmation delay

Suppose a monitored surface has a measured residual spread of 4 mm. The minimum relevant depth change is 15 mm, and the trial multiplier is 4.

A valid residual of 11 mm remains below threshold. A valid residual of 28 mm becomes a candidate. A missing measurement becomes unknown, regardless of the previous residual.

Assume the candidate occupies 80 connected pixels, exceeding the trial region threshold of 25 pixels. It then appears in three successive acquisitions at 20 frames per second.

Confirmation therefore occurs 0.10 s after the first observed candidate, excluding processing and output delays. An object arriving between acquisitions also waits for the next sample. Under these assumptions, arrival-to-confirmation can approach 0.15 s before additional system delays.

An intermittent sequence can take longer or fail confirmation. Measure actual timestamp intervals and end-to-end latency rather than assuming the requested acquisition rate is always achieved.

Verify behavior with a controlled test sequence

6. Validate the complete detector with a fixed reference and settings before enabling background adaptation.

Use a test sequence that separates different failure mechanisms:

• Empty scene: record normal lighting changes and machine vibration. Count confirmed false events, not just noisy pixels.
• Smallest relevant target: place it at near, middle, and far operating distances. Record detection rate and delay.
• Stationary target: leave it in place to verify that the reference policy does not absorb it unexpectedly.
• Appearance and removal: check both signs of depth change if both matter to the task.
• Poor-return surfaces: test dark, reflective, thin, and partly occluded features represented in production.
• Depth loss: obstruct the view and verify an unknown or fault output rather than a fabricated change.
• Restart: confirm that calibration, acquisition settings, and reference selection remain consistent.

Log valid-depth coverage, candidate area, confirmed event count, and latency. Inspect residual images together with raw measurements when an event occurs.

Do not judge success from a smooth visualization alone. Spatial smoothing can make the image attractive while erasing a relevant edge. Acceptance should depend on missed events, false events, unknown coverage, and response time under representative conditions.

Choose acquisition hardware by measured repeatability

7. Compare candidate hardware using the same scene and acceptance tests, not only nominal image resolution.

For integrated depth acquisition, review 3D sensors. The MV-DPS200P-02 is identified in the catalog as a projected structured-light camera. Its suitability for your surfaces and timing requirements still needs confirmation.

A different acquisition principle may be worth evaluating when stereo correspondence is unreliable. The MV-DT01WMU is identified as a ToF sensing camera. That classification alone does not establish its repeatability, range, or suitability for this detector.

For a custom stereo system, area scan cameras provide another integration route. Verify synchronization capability, calibration stability, exposure behavior, and the effect of object motion before selecting a pair. Browse the camera finder to identify candidates, then request the missing acquisition details.

A different depth technology is not automatically a cure for false changes. Ask for representative stationary sequences, documented invalid-value handling, timestamps, confidence information where available, and measurements on your actual materials.

The usual answer is also wrong when camera vibration dominates the residual, the target is smaller than reliable depth support, or the relevant surface rarely returns valid depth. Correct the mounting or reconsider the sensing arrangement before adding stronger filtering.

Checks before deployment

Confirm these items before turning detections into process decisions:

• Depth units, coordinate conventions, and invalid markers are documented.
• The reference has adequate valid coverage throughout the monitored region.
• Thresholds come from representative stationary residuals.
• Plane exclusions preserve the smallest relevant geometric change.
• Spatial filtering retains narrow and partially visible targets.
• Confirmation and release delays meet the process requirement.
• Unknown measurements and stale results have explicit outputs.
• Restart and recalibration behavior have been verified.

Retain the raw test sequences and configuration alongside acceptance results. They provide a repeatable baseline after mounting, lighting, software, or acquisition changes.

Use the camera configurator to outline the acquisition hardware, or contact the applications team with representative depth recordings, target dimensions, working distances, and acceptable false-event and latency limits.