Depth filter camera intrinsics: diagnose wavy point clouds

A wavy point cloud can come from noisy depth measurements, an unsuitable filter, or a mismatch between an image and its calibration. These faults need different fixes. Stronger smoothing may make the display look better while leaving the measurements wrong. This troubleshooting workflow helps you separate acquisition noise from processing errors, verify the geometry of each filtered frame, and decide whether the remaining surface variation is acceptable for your inspection.

Start with an unfiltered, repeatable baseline

Use a stationary, matte, rigid target with independently verified flatness. Keep the camera and target fixed, and choose a distance inside the sensor's documented measurement range. Avoid using a glossy panel as your first reference: reflections can introduce errors unrelated to the processing pipeline.

Save the original depth frames before filtering or point-cloud generation. Record the active acquisition mode, exposure settings, depth scale, frame dimensions and calibration metadata. Preserve invalid-depth flags rather than converting missing measurements into valid zero-distance points.

Capture approximately 100 frames as a practical starting sample. This is an observation window, not a universal acceptance requirement. It lets you distinguish variation between frames from a repeatable spatial pattern.

Compare three outputs from the same recorded input:

• The original depth image, displayed with a fixed depth scale.
• A point cloud generated through the manufacturer's documented conversion function, with post-processing disabled.
• Your application's point cloud, also without post-processing.

If the reference conversion looks correct but your application does not, investigate your conversion and metadata handling first. If both show the same variation, examine the depth measurements and acquisition conditions.

For noise concentrated near the image boundaries, the checks in troubleshooting stereo depth corner noise address a more specific spatial pattern.

Separate the likely causes before changing settings

Use the following order to isolate processing faults before attempting calibration changes. It is a diagnostic sequence, not a claim that every sensor fails in the same order.

A stale pinhole calibration often causes incorrect lateral scale or position. By itself, a uniform focal-length or principal-point error does not necessarily turn an ideal plane into waves. Local distortion, mixed coordinate conventions, depth errors and processing artifacts require separate investigation.

Before using a generic coordinate conversion, confirm the optical-axis conventions described in transforming point clouds between optical and camera frames. A rigid transform cannot repair incorrect camera intrinsics.

Pair every processed frame with its own intrinsics

For a rectified pinhole depth image, the basic conversion is:

Here, u and v are image coordinates. The focal lengths fx and fy, and principal-point coordinates cx and cy, are expressed in pixels. All output coordinates share the physical unit selected by depth_scale.

This formula assumes the stored measurement is axial depth along the optical axis. Some sensors provide radial range instead. Distorted images also require the documented distortion model or an appropriate rectification step. Prefer the manufacturer's deprojection function when those assumptions are unclear.

A processing stage may change more than pixel values. Decimation can reduce dimensions, cropping moves the image origin, and alignment can change the camera model associated with the output grid. Some implementations also introduce padding or retain invalid border regions.

After every geometry-changing stage, retrieve or calculate the output calibration using the documented processing behavior. Do not keep a calibration object captured once at application startup.

Treat these items as one data packet:

• Depth buffer, pixel format, width, height and row stride.
• Depth scale and invalid-value convention.
• Intrinsics, distortion model and valid image region.
• Coordinate-frame identity and acquisition timestamp.
• Processing history and any associated frame identifier.

The closest related diagnostic is depth image resolution smaller than expected: check decimation first. It is particularly useful when the displayed image size differs from the requested acquisition mode.

Work through a resize example

Consider a hypothetical rectified depth image measuring 1280 × 720 pixels. Its calibration is fx = fy = 900 pixels, with principal point cx = 639.5 pixels and cy = 359.5 pixels.

Suppose an ordinary resize produces a 640 × 360 image using a half-pixel sampling convention, without cropping or padding. The scale factors are:

These principal-point equations apply to that particular sampling convention. A decimator selecting input pixels, a block-based filter, or an output containing padding can use different geometry. Read the output metadata rather than assuming every reduction follows this example.

At output pixel u = 410, v = 180, suppose the axial depth is 1000 mm. Correct deprojection gives:

Using the original calibration instead gives:

The stored depth has not changed, but its reconstructed lateral position is wrong. This is a geometry problem, not evidence that stronger smoothing is needed.

For a crop without resizing, subtract the crop's horizontal and vertical offsets from the original principal point. If cropping and resizing are combined, apply the transformations in their actual processing order.

Add filters individually and preserve their meaning

Replay the same saved input through each candidate pipeline. Comparing different live scenes makes it harder to identify which stage changed the geometry.

1. Enable only decimation or resizing. Check dimensions, valid borders, output calibration and point count before judging surface smoothness.
2. Add spatial filtering. Inspect both flat regions and boundaries. Averaging across foreground and background can create points where no physical surface exists.
3. Add temporal filtering. Test a stationary target, then representative motion. Reduced stationary noise may come with lag or trails.
4. Evaluate hole filling separately. Track which measurements are observed and which are estimated. Do not count filled points as independent evidence of surface shape.
5. Verify the final representation. Confirm that the output contains depth rather than disparity, and that its scale and camera model are still correctly attached.

Use the manufacturer's documented stage order where one is specified. A stereo-specific disparity-domain filter is not automatically appropriate for a different sensing technology.

Ordinary image resizing also needs care at depth discontinuities. Weighted interpolation can blend two different surfaces into a fictitious intermediate depth. The general sampling principles in antialiasing when downsampling or warping images explain why resampling changes data, but depth additionally requires explicit treatment of invalid pixels and surface boundaries.

If a separate color image supplies texture, do not substitute its intrinsics merely because the displayed images have equal dimensions. Follow the registration checks in full-resolution color with a depth point cloud.

Measure residuals instead of judging the viewer

Fit a plane to a defined region containing only the reference target. For a plane with a unit-length normal, each point's signed residual is:

Record the fitted plane's orientation, RMS residual, valid-point percentage and a map of signed residuals. Divide the target region into a 3 × 3 grid to check whether apparently acceptable overall statistics hide a localized problem.

Measure two different properties:

• Temporal repeatability: how a fixed pixel or stationary feature changes between captures.
• Spatial surface error: how measurements depart from the known reference surface within a capture.

A plane fit removes overall tilt, so it cannot independently prove correct orientation or absolute distance. Use a reference distance and known lateral dimensions as additional checks. A globally mis-scaled cloud can still fit a plane extremely well.

For example, suppose an inspection must detect a 1 mm step. A filter that removes the step while reducing plane residuals is unsuitable, even if the surface looks smoother. Test representative steps, edges and shallow defects alongside the flat target.

For a stereo system, the approximate relationship Z = f × B / disparity explains why a fixed disparity uncertainty produces increasing depth uncertainty with distance. Do not transfer that behavior directly to every sensor technology. Use the working-distance and positioning checks when noise changes strongly across the intended measurement range.

Change calibration or hardware only after geometry checks

If raw measurements remain systematically distorted, first verify the active acquisition mode and documented software compatibility. The workflow for firmware and SDK alignment helps separate software-version problems from measurement problems.

Use only manufacturer-documented calibration verification or recalibration procedures. Randomly adjusting calibration coefficients until one target looks flat can damage measurements elsewhere in the field.

When considering replacement hardware, begin with the sensing method, required field of view, surface behavior and uncertainty budget. The 3D sensors category and camera finder provide starting points, but a product listing alone does not establish suitability for a particular filter pipeline.

The MV-DPS200P-02 is identified in the catalog as a projected structured-light camera.

The MV-DT01WMU is identified as a ToF sensing camera. Its sensing method should not be treated as interchangeable with stereo processing.

Neither listing establishes support for the example processing stages. Confirm output representation, calibration access, supported modes and measurement performance before selection. Separate area scan cameras may support a color inspection alongside depth, but they do not replace depth calibration or registration.

Use a release checklist before accepting the fix

• Frame geometry: verify dimensions, stride, crop, padding and valid borders after every geometry-changing stage.
• Calibration ownership: attach the correct intrinsics and distortion model to each processed frame.
• Units and representation: verify depth scale, axial depth versus range, and depth versus disparity.
• Coordinate consistency: check axis directions and distinguish camera coordinates from any transformed output frame.
• Filter behavior: compare identical input data and test motion as well as stationary surfaces.
• Measurement acceptance: retain small defects, verify reference distances, and record residual maps rather than screenshots alone.
• Restart behavior: repeat the checks after changing modes, reconnecting the camera or restarting acquisition.

Accept the pipeline only when geometry and measurement checks pass together. If you need help selecting a sensor, use the camera configurator or send the required field of view, working distance, smallest feature and saved diagnostic observations through application support.