Convert a 3D point cloud to a depth image for 2D processing

You have a merged or exported 3D point cloud and a 2D toolchain you already trust: rectangle ROIs, thresholding, blob counting. This guide shows how to rasterize that cloud into a gray depth image - one pixel per XY grid cell, height as the gray value - so those tools run unchanged. One decision drives everything: the height window you map onto 0-255, because the window fixes how much height one gray level carries. Six steps below take a cloud to a finished height map, a worked 10 mm example sets the window and pitch, and the verification tests catch a wrong window or a holed grid before it costs you a missed defect.

What you need

• The cloud as three coordinate arrays - X, Y and Z, one value per point - or as an XYZ file from a scan tool. A cloud merged from several depth frames works the same way once it lives in one coordinate frame; the multi-frame merging guide covers that step.
• A measured point spacing: the median distance from each point to its nearest neighbour. Everything about the grid follows from this one number.
• A height datum. Fit a plane to the fixture or conveyor surface and subtract it, so z reads height above the part seat instead of raw sensor distance.
• An output bit depth, chosen from the smallest height step you must see. The worked example gives the threshold.

The conversion, step by step

Before writing a rasterizer, check whether your vision library round-trips this conversion. If your pipeline built a 3D model from X, Y and Z images with xyz_to_object_model_3d, the inverse operator object_model_3d_to_xyz hands those planes back in one call. The steps below cover what the round trip does not: merged clouds, XYZ files and grids of your own choosing.

1. Plane the cloud. Subtract the fitted datum so z = 0 at the part seat. On a 200 mm wide part, a leftover 0.5° tilt lifts one edge by 200 mm × tan(0.5°) ≈ 1.7 mm - a sixth of a 10 mm window spent on nothing.
2. Choose the pitch. Set the grid cell size equal to the measured point spacing. Finer grids fill with holes; coarser grids throw away lateral detail you paid for.
3. Bin points to cells. Column = round((x - x_min) ÷ pitch), row = round((y - y_min) ÷ pitch). No camera model survives this projection, which is the point: the result is a metric image with a uniform millimetre-per-pixel scale.
4. Fill each cell with height. Where points collide in one cell, average them or keep the last. Where no point lands, leave the cell empty and record it in a separate validity mask - never as z = 0.
5. Scale height to gray. Map the window linearly onto the gray range:

6. Save 16-bit, derive 8-bit. Keep a float or uint16 master for measurement and write an 8-bit copy only for display or for tools that demand one.

A minimal rasterizer in NumPy, with illustrative arrays where your SDK would hand over real ones:

import numpy as np

# Illustrative inputs: float32 arrays, one value per point, millimetres
x = np.array([12.0, 12.2, 12.4], dtype=np.float32)   # X per point, mm
y = np.array([30.0, 30.0, 30.0], dtype=np.float32)   # Y per point, mm
z = np.array([5.0, 10.0, 0.0],  dtype=np.float32)    # height above seat, mm

pitch = 0.2               # float, grid cell size, mm
z_lo, z_hi = 0.0, 10.0    # height window mapped to 0-255, mm

rows = np.round((y - y.min()) / pitch).astype(np.int32)
cols = np.round((x - x.min()) / pitch).astype(np.int32)
height = np.full((int(rows.max()) + 1, int(cols.max()) + 1), np.nan, dtype=np.float32)
height[rows, cols] = z

gray = np.zeros(height.shape, dtype=np.uint8)
valid = ~np.isnan(height)
gray[valid] = np.round(np.clip((height[valid] - z_lo) / (z_hi - z_lo), 0.0, 1.0)
                       * 255.0).astype(np.uint8)
mask = valid.astype(np.uint8) * 255   # 255 where a point landed, 0 where empty

The image must read 128, 255, 0 across its single row: (5.00 - 0.00) ÷ 10.00 × 255 = 127.5, which rounds to 128; the 10.00 mm point clips to 255; the seat point sits at 0. If scrambled values come out instead, dtypes and strides are the usual culprits - the NumPy depth image to point cloud guide works through them in detail.

Worked example: a 10 mm boss on a bracket, scanned at 0.2 mm spacing

Inputs. The cloud covers 240 mm × 180 mm of fixture and comes from a merged scan whose median nearest-neighbour spacing measures 0.20 mm. The datum sits at the part seat (z_lo = 0.00 mm), and the tallest feature, a 10 mm boss, sets z_hi = 10.00 mm. The defect you must catch is a 0.16 mm dent; a 50 µm scratch would be a bonus.

Rule. Gray resolution is the window divided by 255. Image size is the scene span divided by the pitch. The pitch stays valid only while the cloud carries roughly one point per cell.

Arithmetic.

• Height scale: 10.00 mm = 10,000 µm, and 10,000 ÷ 255 = 39.2 µm per gray level.
• Grid size: 240 ÷ 0.20 = 1200 columns and 180 ÷ 0.20 = 900 rows, so the map is 1200 × 900 ≈ 1.08 million cells at 200 µm per pixel laterally.
• Occupancy: filling that grid takes about 1.08 million points. A 500,000-point cloud covers 500,000 ÷ 1,080,000 = 46% of the cells; at pitch = 0.30 mm the grid shrinks to 800 × 600 = 480,000 cells and the same cloud covers it.
• Dent contrast: 160 µm ÷ 39.2 µm ≈ 4 gray levels - thresholdable but thin. The 50 µm scratch spans 50 ÷ 39.2 = 1.3 levels, invisible in 8-bit.
• Window that gives the dent 8 levels: 0.16 × 255 ÷ 8 = 5.1 mm, which resolves 5,100 ÷ 255 = 20 µm per level.
• 16-bit alternative: 10,000 ÷ 65,535 = 0.153 µm per level, so the scratch spans 50 ÷ 0.153 ≈ 328 levels.

Result. A 1200 × 900 height map with 200 µm per pixel laterally and 39.2 µm of height per gray level across the full 10 mm span.

Decision. Measure on the 16-bit master, where the scratch survives at 328 levels. When the dent must threshold cleanly inside an 8-bit tool, derive that copy from a narrowed 5.1 mm slice around the seat, not from the full-window image. Keep the 0.20 mm pitch while occupancy stays above 90%; at 500,000 points, re-run at 0.30 mm.

How to verify the height map

• Gauge test. Seat a 5.00 mm gauge block inside the window and read its plateau. With z_lo = 0.00 mm and z_hi = 10.00 mm it must read (5.00 - 0.00) ÷ 10.00 × 255 = 127.5 → gray 128. A plateau of 255 means the block sits at the window top; a plateau of 64 means the effective window is 5.00 × 255 ÷ 64 ≈ 20 mm, so the datum or the window never made it into the export.
• Occupancy. Mask pixels ÷ total cells. A single organized frame should sit above 99%; a merged cloud below 90% means the pitch is too fine or the frames are misregistered.
• Round trip. Deproject the map back to points with the same pitch and window, then compare 1,000 sampled points against the original cloud: X and Y should agree within one pitch (0.20 mm here) and Z within one gray step (39.2 µm here). The per-pixel math is the deprojection in the depth map to point cloud guide, and the per-axis scale guide handles grids whose pitch differs per axis.

Six mistakes that flatten height detail

1. Window wider than the part. Leaving z_hi at the sensor's full range instead of the part's coarsens the scale from 39.2 µm per level to 50,000 ÷ 255 = 196 µm per level on a 50 mm window - five times worse for no benefit. Set the window from the feature.
2. Pitch finer than the spacing. Halving the pitch to 0.10 mm quadruples the cell count to about 4.3 million while the cloud still holds 1.08 million points, so occupancy falls near 25% and the map turns to lace.
3. Thresholding without the mask. Empty cells hold 0, the same value as a seat-level surface. A threshold on dark surfaces eats the holes; threshold the validity mask first, then the gray image.
4. Measuring on the display copy. A contrast stretch changes gray values, so any height computed from it changes too. Measure on the master and stretch only the copy.

5. Gray read as camera distance. The gray value is height above your datum, not range along the optical axis, so never treat a gray level as a distance reading without a proper conversion from height to range.
6. Frames mixed before binning. If X and Y arrive in the sensor frame while Z arrives in a robot or camera frame, the map shears. Move all three into one frame first - the optical-frame transform guide shows the conversion.

Skip the conversion when the sensor already gives a depth frame

Data that never left the sensor's grid is already a depth image. A laser profiler returns one height profile per trigger, and structured-light or ToF cameras return full-frame depth, so the only work left is step 5's scaling plus a validity mask. Line-scan 2.5D inspection systems are the same case in hardware: the scan delivers height directly, so the conversion reduces to gray scaling. The MV-HLVS-16K100-L60 pairs 16K resolution with a 600 mm emitting light line and a 120 kHz sampling rate.

The rasterization earns its keep once data has left that grid: clouds merged from several frames, points moved between frames, or XYZ exported from a scan tool. And if the inspection is one known region, crop the cloud instead of rasterizing the whole scene - the XY-region cropping guide for 3D point clouds shows that path with no resampling error at all.

Parts and the next step

Because the height map is an ordinary grayscale image, the inspection guides under area scan cameras - thresholds, blobs, measurements - apply to it unchanged. The workflow still starts from organized 3D data, and both sensor families below become the cloud you rasterize once you merge, transform or export it.

Compare depth types in the 3D sensors category, or start the system configurator at the camera step:

If you want the point spacing of your own cloud checked against a candidate sensor, send a sample through the contact page. More depth-image and point-cloud workflows are collected on the 3D topic page.