Capture and save a depth camera image to disk in the preview callback

Depth frames belong on disk: golden images for a new algorithm, the failed part for the quality report, a tuning set built in the office. Every 3D camera SDK exposes the same hook for this — a preview or frame callback it calls once per depth image — and the same two traps. The buffer it hands you is yours only until the handler returns, and the pixel values are 16-bit millimetre distances that an ordinary image save crushes to 8 bits. This how-to gives you the save path that keeps the data: copy first, encode as a 16-bit grayscale PNG, write on a worker thread, then prove the file with a read-back test. The pattern is identical in C++ and C#; only the callback signature differs.

What you need

• A depth camera that streams Z values through an SDK — ToF or structured light alike.
• The SDK's frame-callback registration. It lives in the capture or streaming section of the manual, next to the buffer format.
• A PNG writer that supports 16-bit grayscale. The PNG format defines 16-bit grayscale natively (PNG specification), so mainstream image libraries all write it.
• About 2 MB of disk per 1,280 × 720 frame, before compression; the worked example gives exact figures.
• One decision made up front: the saved file stays 16-bit. Everything else follows from that.

Rule 1: copy the depth buffer inside the callback

The SDK hands your handler a pointer or array into its own streaming memory. When the handler returns, the SDK refills that memory with the next frame — the data is not yours. Keep writing to it afterwards and you get torn frames, zeroed rows or an access violation that looks like a sensor fault. The fix is one copy, inside the callback, into memory you own. Encoding and disk write happen later, elsewhere.

Depth adds one wrinkle: cameras also deliver null or empty buffers during start-up and teardown, and multi-camera setups deliver them routinely, so guard before copying — null frames in a multi-camera callback shows the queue-side handling. The same copy-first rule governs ordinary frame grabbers too: torn frames in a camera callback. Nothing here is specific to one SDK.

// Illustrative handler: take the real parameter types and the registration
// call from your SDK's frame-callback documentation.
void OnPreviewFrame(const uint16_t* depthData, int width, int height)
{
    if (depthData == nullptr || width <= 0 || height <= 0)
        return;                                   // guard: empty buffers on start/stop

    static std::atomic<int> frameIndex{0};        // own counter, initial value 0
    std::vector<uint16_t> snapshot(depthData,
                                   depthData + static_cast<size_t>(width) * height);
    int index = frameIndex.fetch_add(1);
    // saveQueue.Push({index, std::move(snapshot), width, height});
    // -> hand the copy to a bounded queue; a worker thread encodes and writes.
}

// Illustrative handler: use your SDK's delegate signature for the preview event.
void OnPreviewFrame(ushort[] depthBuffer, int width, int height)
{
    if (depthBuffer == null || depthBuffer.Length < width * height)
        return;                                   // guard: empty buffers on start/stop

    ushort[] snapshot = new ushort[width * height];
    Array.Copy(depthBuffer, snapshot, depthBuffer.Length);   // the copy, inside the callback
    int index = Interlocked.Increment(ref _savedCount);      // own field, initial value 0
    Task.Run(() => SaveDepthPng(snapshot, width, height, $"depth_{index:D6}.png"));
    // SaveDepthPng is your own writer: single-channel 16-bit grayscale PNG.
}

The hand-off queue is bounded. If the writer falls behind, drop the newest frame and increment a drop counter — a visible gap in the sequence beats a stalled stream or an out-of-memory kill. The queue also keeps the callback fast, which matters because PNG encoding plus disk write takes milliseconds; run it in the handler and your frame rate falls to encode speed. That arithmetic — encode time against disk throughput — is worked through in fixing image-save lag and disk write bottlenecks; the mechanism is language-independent.

Rule 2: write a 16-bit grayscale PNG, not 8-bit or JPEG

Depth Z arrives as unsigned 16-bit values, one per pixel, in millimetres on most ToF and structured-light cameras. The SDK's frame-format page states the unit and any scale — some cameras deliver 0.1 mm units or shifted values — and the analogous container case for smaller data is covered in why a Mono16 buffer carries 12-bit sensor data.

Encode the copied buffer as a single-channel 16-bit grayscale PNG and the millimetre values survive byte for byte: PNG is lossless and its grayscale channel supports 16 bits. Convert to 8-bit "so it displays" and the distance steps collapse; save as JPEG and you add a lossy, 8-bit-only codec on top, which smears depth edges. The worked example below puts numbers on the loss. The encoder settings themselves — channels, bit depth, compression level — are the same ones covered in how to save 16-bit grayscale camera images reliably; depth adds only the unit question.

The save procedure, step by step

1. Register the handler. Subscribe to the SDK's preview or depth frame event before you start the stream, then confirm frames arrive at the advertised rate.
2. Read the buffer format once. Pixel type (unsigned 16-bit), unit — millimetres on most depth cameras — and row order, from the frame-format documentation.
3. Guard, then copy. Skip null or zero-size buffers, copy all width × height values into your own array, return. Nothing slow happens in the handler.
4. Stamp the frame. Frame index in the filename; monotonic timestamp and unit in a sidecar or log line.
5. Encode off the callback thread. A worker takes copies from a bounded queue and writes one-channel 16-bit grayscale PNG files.
6. Verify before you trust it. Run the read-back test below on the first saved file, then let the run proceed.

Worked example: one 1,280 × 720 frame

Take a depth frame of 1,280 × 720 pixels from a camera with a 4,000 mm working range — a common indoor figure — and cost out both save paths:

The 8-bit file saves about 0.9 MB of disk and costs a factor of roughly 16 in depth resolution: over a 4,000 mm range, every 15.7 mm of distance shares one gray value, so a measurement good to ±1 mm is destroyed. The 16-bit file keeps 1 mm per level and cannot clip, because the 65,535 mm ceiling sits an order of magnitude above the camera's range.

Throughput at 30 frames per second: 30 × 1.84 MB ≈ 55 MB/s raw. Depth images compress well — flat surfaces produce long runs of identical values — so expect PNGs roughly 2–5× smaller, about 11–27 MB/s on disk. A laptop SSD sustains that without effort; a callback thread asked to run 30 encodes per second does not, which is why step 5 exists.

Verify the file with a read-back test

Two quick checks, each costing seconds.

First, prove the bit depth and value spread by reading the file back:

# Test: prove depth_000001.png still holds 16-bit millimetre values.
import numpy as np
import imageio.v3 as iio

d = iio.imread("depth_000001.png")          # ndarray of dtype np.uint16 for a 16-bit PNG
assert d.dtype == np.uint16, f"saved as 8-bit: {d.dtype}"
valid = d[d > 0]                             # 0 marks invalid pixels on many cameras
print(int(valid.min()), int(valid.max()))    # e.g. 312 3846 — a plausible millimetre spread

If the array comes back 8-bit, the writer converted on the way out — fix the encoder flags and re-save. If values sit pinned at 255 or 65,535, something in the chain clipped or scaled them.

Second, prove the values mean what the unit claims. Point the camera at a wall, measure the wall with a tape — say 2,000 mm — then read the centre pixel of the saved file. Agreement within 1–2% at 2 m is normal depth-camera precision; a reading near 0 or 65,535 is invalid or clipped data, not a bad wall. The reading methods are in center pixel depth measurement and reading distance from a depth image pixel without an SDK.

Common mistakes that ruin depth saves

• Saving JPEG "so the viewer can open it": lossy and 8-bit at once. Export a display copy from the 16-bit master instead.
• Using the buffer after the handler returns: the next frame overwrites it mid-write, producing torn rows that look like sensor faults.
• Encoding on the callback thread: frame rate drops to PNG encode speed, and trigger-synced captures miss frames.
• An unbounded queue: a slow disk builds minutes of backlog, then the process dies of memory. Bound the queue and log the drop counter.
• Trusting the picture: viewers stretch contrast, so a file that looks black can hold perfect data and a pretty one can be 8-bit. Read values, as the test above does.
• Skipping the unit check once: a 0.1 mm-unit stream saved as millimetres is wrong by a factor of 10, in every file, forever.

When a session recording beats one PNG per frame

Single PNGs suit goldens, failure captures and short tuning sets. For an hour of full-rate frames they waste effort: use the SDK's session recording if it offers one. Some 3D SDKs write H5 stacks you replay frame by frame offline — recording a 3D camera to H5 files — and log-style captures replay without drops when read sequentially — bag file playback without dropped frames. The copy-first discipline does not go away: whatever the recording call, it still runs inside the callback and must return fast. For the folder layout and naming that keep saved frames usable weeks later, saving camera frames to disk for offline testing picks up where this procedure stops.

Depth cameras for this workflow

Any depth camera that hands 16-bit Z buffers to an SDK callback fits — ToF, projected structured light or stereo module alike. The camera finder filters them by sensing type and interface. These cover two of the families — a ToF sensing camera and a projected structured-light camera:

Start with one file and the read-back test; once the assertions pass, let the run go. If you want a depth camera matched to your working range and mounting geometry, tell us about the application, or browse more depth-frame work under the 3D camera topic.