Preview shows stacked aligned depth frames: fix the shared filter

Your Android capture app works until you press the snapshot button: every save adds one more picture to the preview surface, and after twenty saves twenty images sit stacked on the screen. The saved files are correct — aligned depth, color frame, raw depth array — which makes it look like a display bug. It is not. This guide is for engineers writing capture code against a stereo depth camera SDK on Android in Java, where a streaming loop feeds a GLSurfaceView and a background task writes the files. You will see why the layers accumulate, the two-path code fix, and how to prove it in five snapshots.

Symptom: one more image every time you save

The streaming loop pulls a frameset, pushes it through a colorizer, and uploads it to the view; on its own, the preview is stable. The snapshot function then pulls its own frameset on a background task, writes the raw depth array to disk, and applies align-to-color plus a colorizer to produce the saved images. Those files come out right.

The instant the align call enters the snapshot path, every button press paints one more image onto the view: twenty snapshots, twenty stacked images. Nothing in the snapshot code ever calls the view's upload. The frames reach the screen through the filter object both paths share — that is the whole failure.

Cause: the same filter object on two paths

Align and colorize blocks are stateful processing blocks, not pure functions. They carry state between calls, and this app calls applyFilter on the same instances from two places: the streaming runnable on its handler thread, and the snapshot task on its executor. The two paths are coupled at that shared object. The surface view paints every frame it is handed and keeps a layer for each distinct stream and resolution it has seen, so the snapshot's processed output lands on screen as new content and stays there.

Confirm it in this order:

1. Comment out every applyFilter in the snapshot and save only the raw array. The stacking stops and returns when you restore the calls: shared filter instances confirmed.
2. Restore the calls, but build a private align and a private colorizer inside the snapshot and use only those. The stacking stays gone even though the align work still runs: ownership of the block is the variable, not the align math.
3. Strip every filter from the app, including the loop's colorizer, and upload raw frames only. If images still accumulate, the view itself is at fault — inspect surface recreation and the clear-between-draws behaviour. In the reported case the stacking vanished at step 2 and never reached this branch.

The fix: one consumer per processing block

The rule: one processing block instance per consumer, and the display path uploads only frames that path itself produced.

1. Strip applyFilter from the streaming runnable. Upload the raw frameset inside its FrameReleaser scope.
2. Create the capture path's own blocks once, when the pipeline starts: one align-to-color and one colorizer. The capture object owns them; the loop never references them.
3. In the snapshot, pull a fresh frameset in its own FrameReleaser, apply the capture align, then the capture colorizer, take the color frame from the aligned frameset, and write the files.
4. If you want the color map on screen, give the loop its own second colorizer and set the palette on that instance. Uploading the raw frameset also renders on its own — that is the exact configuration that ended the stacking in the reported case.
5. Rebuild and take five snapshots, then twenty. The layer count stays at one image per stream.

// display path: raw frames only, no shared blocks
try (FrameReleaser fr = new FrameReleaser()) {
    FrameSet frames = pipeline.waitForFrames().releaseWith(fr);
    glView.upload(frames);
}

// capture path: blocks created once at pipeline start, owned here
try (FrameReleaser fr = new FrameReleaser()) {
    FrameSet aligned = pipeline.waitForFrames().releaseWith(fr)
            .applyFilter(saveAlign).releaseWith(fr);
    try (FrameSet colored = aligned.applyFilter(saveColorizer)
            .releaseWith(fr)) {
        // read the depth and color frames, then write the files
    }
}

Worked example: 20 snapshots, 720p depth, 1080p color

Inputs, with the basis for each: a stereo depth camera of this class streams 1280 × 720 depth at 30 fps, 16-bit at 2 bytes per pixel, alongside 1920 × 1080 RGB color — the common configuration for this hardware on Android. The save count of 20 comes from the reported case: 20 button presses, 20 stacked images.

The arithmetic:

• Raw depth frame: 1280 × 720 = 921,600 pixels × 2 B = 1,843,200 B, about 1.8 MB.
• Preview throughput before the fix: 30 fps × 1.8 MB = 55 MB/s of depth flowing through the shared colorizer every second the loop runs.
• Aligned, colorized depth: resampled to 1920 × 1080 = 2,073,600 pixels × 3 B per RGB pixel = 6,220,800 B, about 6.2 MB per save. The color frame in the same frameset is the same size, another 6.2 MB.
• Layers per snapshot: up to two, aligned depth and color, roughly 12.4 MB. Twenty saves: 20 × 12.4 MB ≈ 248 MB of pixel data the surface keeps painting and never frees.
• Frame period: 1 / 30 fps ≈ 33 ms. The snapshot's own waitForFrames blocks for up to one frame period; with separate blocks the loop keeps its cadence.

The decision: any filter shared between display and capture puts you on the wrong side of zero. Split the paths before you ship — the leak grows linearly with the save count, and a long capture run takes the app down with it.

Checks after the fix

• Take five snapshots: the preview shows one image per stream after each save, never layers.
• Watch graphics memory across twenty saves: the curve stays flat within a few MB. Before the fix, each save added about 6 MB at 1080p.
• Open a saved pair: the colorized depth image frames the same field of view as the color image.
• Check the raw array's shape: alignment belongs to the saved-image path, so the array keeps its native 1280 × 720 unless you deliberately save the aligned grid.
• Rotate the device and background the app: no layers return after the surface is recreated.

When the stacking is something else

• One duplicated pair, fixed in size and not growing per save: a resolution or layout mismatch between the streams, not a leak. The depth to RGB registration offset in pixels guide walks the rectified-stream settings that cause it.
• Faint lines tracking object edges inside the saved aligned image: a resampling artifact of the align step, not a display fault — see lines in aligned depth images.
• The raw array looks misaligned next to the saved images: you may be comparing raw against rectified frames. Depth to color registration: raw versus rectified and checking whether the depth image is aligned to RGB settle which frame you are holding.
• The preview turns gray or garbled instead of stacking: a display format mismatch — fixing the depth image 16UC1 display covers the conversion path.

Hardware this code runs against

Any depth camera with separate depth and color streams behind stateful processing blocks shows this pattern, including the projected structured-light and laser galvanometer stereo models among our 3D sensors. Two of that family:

Both of these project active illumination: follow the laser class printed on the product label and never defeat an interlock. To filter the whole range by sensor type and interface, use the camera finder.

Prevention checklist

• One processing block per consumer, and its owner is exactly one path.
• Raw frames to the display; processing happens on the path that saves.
• Every frameset and frame lives inside a FrameReleaser scope.
• The capture task references nothing from the streaming runnable.
• Storage permission is granted at run time before the first save.
• After any pipeline change, five snapshots and a memory trace confirm layer count and memory stay flat.

If the stacking survives a clean two-path split, contact the applications team with the loop code, the snapshot code and the SDK version — this one usually closes in the first reply. For choosing the hardware side, start from the category card below.