Export a structured point cloud as xyz and PNG from one capture

You have the HDR settings tuned, and one capture has to become four files: the structured point cloud, an RGB png, a depth png and a plain-text xyz list. The capture interface hands you every array you need from a single frame. But the first xyz export prints nan nan nan across whole blocks of lines, and the obvious np.nan_to_num fix plants phantom points at the sensor origin. This guide gives the working pipeline: which array feeds which file, the one-channel NaN test that sidesteps the ambiguous-truth-value error, and the metres-to-millimetres cast that keeps a 16-bit depth png honest. Three verification checks close it out, and the same steps run once or inside a loop over several HDR settings, so the point clouds you compare stay shot for shot identical.

Why does the xyz file show nan nan nan on some lines?

The xyz array is a structured point cloud: it stores one xyz triple for every camera pixel. Each pixel the projector cannot reach — inside its own shadow, too dark, or too shiny to triangulate — keeps NaN in all three of its channels.

The fill is deliberate. An array has to hold something in every cell, and the capture cannot invent a coordinate for a pixel whose projected stripe the sensor never saw. NaN stands in as an explicit no-return marker. The same NaN sits in z, because z is exactly the third slice of xyz: filter one channel and you have filtered both.

An unstructured cloud has no such cells. It is a flat list of points, so it simply omits the failed pixels and comes out smaller — that is the file you build when you write the xyz list by hand. The failed pixels cluster behind edges and inside recesses. When a scene has too many of them, the cure is lighting and fixture geometry, covered in missing points in a 3D camera point cloud.

A projected structured-light camera returns depth on exactly this per-pixel grid. That is why the export starts from a height-by-width-by-three array instead of a point list.

Which files can one capture give you?

One capture yields four registered files: the structured pcd, an 8-bit RGB png, a 16-bit depth png in millimetres, and an unstructured xyz text list of the valid points. All are written from the same frame, so nothing moves between exports.

These are not four captures. Pull the arrays once, and every file inherits the same exposure set, the same pose and the same transform, which is what makes later comparisons fair. If a deliverable turns out to need a depth image for 2D processing, it can be regenerated from the same frame — convert a 3D point cloud to a depth image shows that direction.

How do you export RGB, depth and xyz from one capture?

Capture once inside the with-block, pull rgb, xyz and z off the frame's point cloud, and write every file from those arrays. copy_data takes the field names rgb, xyz and z, and frame.save writes the pcd while the frame is still open.

1. Load the settings file and connect to the camera, as in your working capture script.
2. Wrap the capture in a with-block so the frame lives exactly as long as the export.
3. Pull rgb, xyz and z from the frame's point cloud with copy_data while the block is open.
4. Call frame.save for Frame.pcd — the structured cloud, every pixel, NaN fill included.
5. Scale z to millimetres, cast to 16-bit unsigned, and write depth.png.
6. Write the colour png with the colour axis reversed first, because the png writer reads arrays as BGR.
7. Build the valid mask from one NaN test on x, and write frame.xyz from the kept rows.
8. Run the three verification checks below before the folder leaves the bench.

from pathlib import Path
import numpy as np
import cv2                                # the png writer used below

out = Path("captures/001")                # example variable: one folder per run
out.mkdir(parents=True, exist_ok=True)

with camera.capture(settings) as frame:   # one HDR capture -> one frame
    p = frame.point_cloud()
    rgb = p.copy_data("rgb")              # colour, shape (H, W, 3), values 0-255
    xyz = p.copy_data("xyz")              # coordinates, (H, W, 3), metres, NaN = no return
    z = p.copy_data("z")                  # depth, identical numbers to xyz[:, :, 2]

frame.save(str(out / "Frame.pcd"))        # structured cloud, NaN pixels kept

The capture call, point_cloud, copy_data and save names come from the camera's frame interface; camera, settings, out and the array names are this example's own variables. For the colour file, either reverse the last axis of rgb before writing or request the bgra layout and write that straight — otherwise viewers show red and blue swapped.

Several HDR settings change nothing structurally: one folder per capture, arrays pulled and written per folder. When those captures later need to become one cloud, merging multiple depth frames into one gap-free point cloud picks up where this export stops.

How do you filter the NaN rows out of the xyz list?

Test one channel per pixel with np.isnan and keep only the rows that pass. A dead pixel carries NaN in all three channels together, so a single isnan check on x removes it, and a 12 percent shadow rate simply drops that share of rows.

valid = ~np.isnan(xyz[:, :, 0])           # (H, W) booleans: True where the pixel returned
points = xyz[valid]                       # (N, 3): real points only
np.savetxt(out / "frame.xyz", points, fmt="%.6f")

The check lands on x alone because the fill marks the whole triple: if x is NaN, so are y and z. One channel decides the pixel, no more.

A pixel-by-pixel loop does the same job and leaves room for per-pixel logic:

with open(out / "frame.xyz", "w") as f:
    for row in range(xyz.shape[0]):
        for col in range(xyz.shape[1]):
            point = xyz[row, col, :]          # one pixel: a three-element vector
            if not np.isnan(point[0]):        # scalar test, so `not` is allowed
                f.write(f"{point[0]} {point[1]} {point[2]}\n")

The failure mode of the careless version — ValueError: the truth value of an array with more than one element is ambiguous — comes from testing a whole matrix row instead of one scalar. np.isnan on a row returns an array, and not cannot decide an array. Index down to a scalar, as above, and the check behaves.

Take a 2448 × 2048 capture — 5,013,504 pixels — with 12 percent of pixels in projector shadow:

• nan_pixels = 0.12 × 5,013,504 ≈ 601,620
• xyz_lines = 5,013,504 − 601,620 = 4,411,884
• frame.xyz ≈ 4,411,884 × 27 bytes ≈ 119 MB
• depth.png payload = 5,013,504 × 2 bytes ≈ 10 MB before compression
• the same points as float32 binary: 4,411,884 × 12 bytes ≈ 53 MB

ASCII is readable by everything and costs the most; a binary float32 export cuts the size by more than half whenever the downstream reader accepts one. When that xyz list feeds a profile-tolerance comparison rather than a visual check, point count decides whether the verdict stands. The CMM point cloud density guide for profile tolerance works through how many points each feature needs.

How do you write a depth png that keeps real millimetres?

Multiply the z array, which arrives in metres, by 1,000 and cast to 16-bit unsigned, so 0.812 m stores as 812 and the file spans 0 to 65,535 mm — an 8-bit png stops at 255 mm.

z_mm = np.where(np.isnan(z), 0, z * 1000.0)   # metres -> mm; NaN -> 0 means "no data"
cv2.imwrite(str(out / "depth.png"), z_mm.astype(np.uint16))

The png format stores integers, so the scale factor is where your resolution lives: 1 mm per count here. A cell under 6.55 m can keep 0.1 mm steps instead — scale by 10,000 and 0.812 m stores as 8,120, still inside the 65,535 ceiling. Save the metres float unconverted and the writer chooses a rounding you did not ask for. Declare 0 as no data and keep that convention on the way back in: applying per-axis scale factors to a depth image shows the matching arithmetic.

Why should you never zero-fill the xyz export?

np.nan_to_num rewrites every dead pixel as a clean [0, 0, 0] triple, and every tool downstream takes that literally: a physical point at the sensor origin that registration has to absorb, bounding boxes stretch to include, and volume sums count.

With 601,620 dead pixels, zero-filling adds over half a million phantom points at one exact location. Fine matching pulls the pose to explain them, and the defect you were hunting can hide inside the error they create. That invalid-zero pattern has its own diagnosis path in depth pixel deprojection zero.

Zero-filling belongs in exactly one place: the depth png, where 0 is a declared no-data code that no measuring tool mistakes for a metre. An xyz text list carries no such convention — every line is a claim — so filter there, and fill nowhere else.

How do you verify the export before it moves downstream?

Read the numbers back and assert three facts: the xyz file holds exactly the valid-pixel count, no kept point contains NaN, and a spot pixel times 1,000 matches the depth png. Those three checks catch every silent failure this pipeline can produce.

points = xyz[valid]
assert points.shape[0] == int(valid.sum())    # one line written per valid pixel
assert not np.isnan(points).any()             # no NaN escaped the filter
probe = xyz[1200, 1224]                       # any pixel that sits on the part
print(int(round(float(probe[2]) * 1000)))     # depth.png at row 1200, col 1224 must equal this

Open depth.png with the reader's unchanged-bit-depth mode; a default 8-bit read collapses the file silently and the comparison lies. Expect the pcd and the xyz to disagree on count by design: the pcd holds all 5,013,504 pixels, the xyz holds 4,411,884. Viewers render or hide the NaN set differently — why exported point clouds differ from the viewer render walks that gap down.

When does the structured pcd still beat the xyz list?

Keep the structured cloud whenever the next step needs pixel adjacency — normal estimation, height maps, ROI masks, or overlaying the 2D image on depth — because the flat list has forgotten which pixel each point came from.

Normals, curvature and hole-filling all read neighbours straight off the grid; from a list they need a spatial search that runs slower and breaks at the removed pixels. The grid is also what 2.5D height map to 3D point cloud conversions travel along in both directions.

For sizing and volume work the list is usually the better deliverable: it carries only real points and drops straight into downstream tooling. A speckle structured-light camera aimed at volume measurement produces exactly the per-pixel cloud this pipeline starts from, so the same eight steps serve both deliverables.

To size the sensor behind these files, browse the 3D sensors range, start the camera configurator, or contact the applications team with your part and cell — we will help you pick the capture and the export that fit.