Depth image to point cloud: apply per-axis scale factors

A depth image is a grid of raw counts, not a shape you can measure. To get a point cloud, every pixel needs an X, a Y and a Z in length units. That takes three scale factors, one per axis. This guide shows what each factor does, where the correct numbers live — in the camera's calibration, never in a guessed constant — when a constant per-axis scale is valid, and when it quietly stretches your part. A worked example converts a 16-bit height map to millimetres with the arithmetic shown at every step.

What the three scale factors do

A depth image stores one value per pixel: how far the surface sits from the sensor's reference, in counts. A point cloud stores a triplet (X, Y, Z) per point, in millimetres. The conversion is three multiplications and at most three offsets:

DN is the stored depth value. On calibrated hardware the offsets are zero, so each axis reduces to index times scale.

The loader pattern that does this is short. It reads the frame and converts it to floating point with convert_image_type, generates two ramp images that grow by one per pixel in the X and Y directions (gen_image_surface_first_order), then applies the three factors with multiply-and-add calls in the style of scale_image — value times factor, plus an offset of zero. Stacked together and passed to xyz_to_object_model_3d, the three planes become a 3-D object model. If you load the frames in Python instead, sort out strides and dtypes first; the NumPy depth-image-to-point-cloud fix lists the traps.

Where each factor comes from

All three are calibration values, and they are not interchangeable:

• sz — length per depth count. This is the sensor's Z calibration. It converts counts to millimetres; it is not the sensor's accuracy.
• sx — length per pixel across the profile. The lateral calibration at the reference condition, shipped in the sensor's calibration file. Square pixels let one factor serve both lateral axes, which is why loaders type the same number twice.
• sy — length per profile line. Set by motion: scan speed divided by profile rate. At 120 mm/s and 6,000 profiles per second, sy = 120 / 6,000 = 0.02 mm per line. On a line-scan 2.5D system an encoder clock takes over, and the line scan line rate calculator does this division for you.

In production, read all three from the camera's calibration data: the factory file, or the parameters its SDK exposes after capture. Constants typed into demo code are remembered values from one set-up — a worked example, not a default.

Constant scales or deprojection: which case you are in

Height maps from profile sensors: constant factors hold

Laser profilers and line-scan 2.5D systems image a calibrated plane. The sensor's calibration already maps every pixel to a length across the profile and every count to a height, so the same pitch applies everywhere in the frame and the three multiplications are exact within the calibration's own error. This is the case the classic loader was written for.

The MIS-DP3850-01P is a 3D laser profile sensor of the DP3000 series, and the MV-DP4180-03P is its DP4000-series counterpart; both deliver the height map and the calibration that the three multiplications consume. Line-scan systems convert the same way, one profile at a time: the MV-HLVS-16K100-L60 pairs 16K resolution with a 600 mm emitting area and a 120 kHz sampling rate, and its sy comes from the encoder that clocks those lines.

Perspective depth cameras: per-pixel deprojection

Stereo, projected structured-light and ToF cameras return a Z per pixel of a perspective image. Lateral spacing is then not constant: millimetres per pixel grows linearly with distance, which is what the pinhole model expresses:

fx and fy are focal lengths in pixels; cx and cy locate the principal point; all four come from the intrinsics the SDK publishes. Example: with fx = 550 px, a pixel 200 px right of centre sits at X = 200 × 1,000 / 550 = 364 mm when Z = 1,000 mm, and at X = 200 × 2,000 / 550 = 727 mm when Z = 2,000 mm. Loading that frame with constant factors squashes the near field and stretches the far field. The step-by-step deprojection, including the Z-value-to-camera-distance correction, is in Depth map to point cloud: the deprojection calculation explained; Depth value vs true distance covers that correction on its own.

For these cameras take the cloud from the SDK's own conversion, or deproject with the published intrinsics — do not hand-scale. A projected structured-light 3D camera such as the MV-DPS200P-02 and a ToF sensing camera such as the MV-DT01WMU are built for exactly that path.

Worked example: a height map to millimetres

Take the calibration behind the classic loader — one set-up's values, not defaults:

• 1,280 × 1,024 pixels, 16-bit counts (0 to 65,535)
• sx = sy = 0.019 mm per pixel (19 µm)
• sz = 0.0016 mm per count (1.6 µm)
• Point of interest: pixel (500, 300) with depth value DN = 20,000

The surface point sits at (9.5, 5.7, 32.0) mm in the sensor frame. The same arithmetic sizes the whole capture:

• Field: 1,280 × 0.019 = 24.32 mm wide by 1,024 × 0.019 = 19.46 mm tall.
• Z span: 65,535 × 0.0016 = 104.86 mm, with 1.6 µm between adjacent counts.

A 10.00 mm gauge block should therefore span 10.00 / 0.0016 = 6,250 counts. That number is the anchor for the checks below.

When a constant scale gives wrong geometry

Three situations distort a cloud even though the code runs clean:

1. A perspective depth map under constant factors. The error grows with distance. At fx = 550 px, treating the Z = 2,000 mm point as if it sat at 1,000 mm turns its true 727 mm X into 364 mm — the far half of the scene collapses toward the centre.
2. A line pitch that no longer matches the motion. sy = speed / rate, so a conveyor running 5 % slow while the loader keeps 0.019 mm per line stretches a strip that should span 1,024 × 0.019 = 19.46 mm to 19.46 × 1.05 = 20.43 mm — a millimetre of phantom part. Clock the pitch from an encoder whenever speed is not fixed.
3. Tilt between sensor and part. A part off the calibrated plane mixes X, Y and Z, and no scale repairs that. Level after conversion: the classic pipeline takes the principal axes with moments_object_model_3d, flips them with pose_invert, and applies the result with rigid_trans_object_model_3d. If a triangulation cloud still leans, the checks in Why a laser triangulation point cloud is tilted trace the cause to calibration.

Checks before you trust the scaled cloud

Run these once per sensor and after any mechanical change:

1. Z: measure a 10.00 mm gauge block. Expected: 10.00 / 0.0016 = 6,250 counts of difference; measured counts × 0.0016 must return 10.00 mm.
2. X: image two features a known 8.00 mm apart across the profile. Expected: 8.00 / 0.019 = 421 px; measured px × 0.019 must return 8.00 mm.
3. Y: scan a part of known length and divide length by profile count. At a true 0.02 mm per line, a 50.0 mm part spans 2,500 lines.
4. Noise floor: the 1.6 µm count size is quantization, not accuracy. Repeat a flat-spot measurement ten times; the spread is the repeatability you can quote, and it sits well above one count.

To watch one pixel's depth drift over a shift, the method in Center pixel depth measurement reads that fixed point straight from the frame.

Matching the method to the sensor

The decision follows the first column. Calibrated height map in hand: multiply column by sx, row by sy, count by sz, and the cloud is in millimetres. Perspective depth map in hand: let the SDK or the intrinsics do the lateral mapping, because no constant reproduces it. Downstream, crop the XY region of interest before conversion to keep clouds small, and set point cloud x, y and z resolution once the geometry is honest.

If a converted cloud ever looks stretched, suspect a factor from the wrong camera before suspecting the part: 0.019 and 0.0016 belong to the set-up they were measured on, and every sensor carries its own pair. To pick hardware whose calibration hands you these numbers directly, start in the 3D sensor finder or lay out the whole station in the configurator; the 3D topic page collects the surrounding workflow, and contact is open when a live scan needs a second pair of eyes.