Torn frames in a camera callback: copy the buffer before you queue it

Frames arrive marked Complete, yet parts of the image belong to the previous or next exposure. Your callback checks the status, queues the image and hands the buffer back — and the artefact still appears, worst exactly where your product goes bright-to-dark between frames. This walkthrough gives you the failure sequence, a five-minute test that settles the cause, the corrected handler and the numbers that show what the fix costs. It applies to any machine-vision SDK whose streaming callback hands you a reusable frame buffer, over USB3, GigE or CoaXPress, in Python, C++ or C#.

The symptom: complete frames, torn pixels

The artefact is a rectangular patch or a horizontal boundary inside an otherwise correct image: rows from one exposure butted against rows from another, both sharp, both correctly exposed. The delivery status reads Complete. The frequency tracks the scene — bright and dark transitions between frames make it obvious, a static scene hides it.

That combination is a signature, not a coincidence. The camera delivered intact pixels and something overwrote part of them between delivery and use. The scene dependence is the tell: the overwrite happens at the same rate whatever the scene; you only see it when adjacent frames differ.

Why the status check cannot catch it

The status describes the buffer at the instant the callback fired. Nothing re-checks it while your consumer thread reads that buffer later. The sequence:

1. The exposure ends and DMA fills buffer B; the callback fires with status Complete.
2. Your handler converts the frame to an ndarray and puts that object on a queue.
3. The handler hands buffer B back to the driver for reuse.
4. The camera starts filling buffer B with the next exposure, because your queue holds a pointer to B, not the pixels.
5. The consumer reads B while the camera writes it: rows above the race point are old, rows below are new.

The status was true when your code tested it. The race starts one frame later.

Cause 1: the queue holds a view, not a copy

In this API family, frame.as_numpy_ndarray() wraps the frame buffer: it returns an ndarray pointing into the driver's memory, with no pixel copy. Queueing that object exports a borrowed pointer across a thread boundary. Re-queueing the frame with cam.queue_frame(frame) — or letting the driver recycle it when the callback returns — starts the next exposure into that same memory.

The SDK's shipped example, multithreading_opencv.py, states the rule outright: queue a copy of the frame, never the frame itself, because the driver overwrites the buffer as soon as it is reused.

A second bug hides in the same handler. The re-queue call sat inside the Complete branch, so any frame delivered with another status was never returned to the pool. After enough of those, the announced buffers run out and streaming stalls for no visible reason. Hand every frame back, whatever its status.

Queue design is a separate problem: bounding the frame queue and reusing buffers fixes memory growth, but only the copy fixes tearing. The same aliasing bites at save time — stereo frames saved wrong is this failure with a file at the end of it.

The fix: copy in the callback, then hand the original back

import copy
import queue

class CopyingHandler:
    # cam, frame, frame.get_status(), FrameStatus.Complete and
    # cam.queue_frame() are the SDK's API names; CopyingHandler is
    # an illustrative wrapper declaring its own queue.
    def __init__(self) -> None:
        self.frame_queue: queue.Queue = queue.Queue(maxsize=2)

    def __call__(self, cam, frame) -> None:
        if frame.get_status() == FrameStatus.Complete:
            frame_cpy = copy.deepcopy(frame)   # copy before queueing
            if self.frame_queue.full():
                self.frame_queue.get_nowait()  # drop the oldest copy
            self.frame_queue.put(frame_cpy)
        cam.queue_frame(frame)                 # every frame goes back

The consumer converts each queued copy with as_numpy_ndarray() when it needs pixels. If you only need pixels, copying the array is cheaper than copying the frame object with all its metadata: pixels = numpy.array(frame.as_numpy_ndarray(), copy=True). Faster again, keep a small ring of pre-allocated arrays and copy into it — reusing image buffers instead of per-frame allocation covers that pattern.

Run a depth of 1 to 3 with drop-oldest: a slow consumer then loses whole frames instead of reading half-written ones. Depth 1 always delivers the newest frame; 2 or 3 smooths a consumer that stalls in bursts. To overwrite stale frames instead of blocking on a full queue, stale-frame queue overwrite mode explains that policy on another camera pipeline.

Proof the fix holds, using your own worst case — a bright/dark transition on every frame:

# Test: toggle the light each frame for 200 frames. A torn frame's
# mean lands between the two clean levels instead of on one of them.
means = [float(handler.frame_queue.get().as_numpy_ndarray().mean())
         for _ in range(200)]
bright, dark = max(means[:20]), min(means[:20])
mid = (bright + dark) / 2.0
assert all(abs(m - mid) > (bright - dark) / 4.0 for m in means)

Five-minute tests that separate the two causes

1. Patch in the copy first. One line; if the stripes vanish you are done. This fixes most Complete-but-torn reports.
2. Run the vendor's viewer at the same exposure, pixel format and frame rate. Clean viewer images with torn application images convict your handler; torn viewer images send you to the link path below.
3. Freeze the scene. Cap the lens. If the artefacts disappear, nothing was fixed — the overwrite continues, but the pixels written in are identical. It confirms aliasing; it cures nothing.
4. Halve the load. Drop the frame rate, or lower DeviceLinkThroughputLimit — the GenICam feature that caps the camera's share of the link. Tearing that survives at a crawl, copy in place, is not a software race.

Cause 2: a marginal USB3 link

A sick link announces itself differently: frames marked incomplete, dropped frames, or the host renegotiating down to the 480 Mbit/s USB2 rate, after which the frame rate collapses. Work it in this order:

• Shortest cable that reaches. A 3 m passive USB3 cable keeps signal margin; long passive runs are where stripes and drops begin. The LS-CBL-0219 is a 3 m USB3.0 camera cable rated for drag-chain use. Ground yourself before reseating the connector — an ESD hit can leave a USB3 port negotiating at the slower rate for good.
• One camera per host controller, no hubs. Controllers are scarce on small embedded boards, and choosing the host for board-level cameras is worth reading before you commit to one. Compact models such as the LS-BCM-0003 — a 3.1 MP USB3 board camera with a 1/1.8" global shutter and 120 fps — will saturate a shared controller at full speed.
• On Linux hosts, raise the USB transfer buffer limit before blaming hardware: USB3 frame rate and latency fixes through usbfs memory and raising the host buffer for two full-resolution USB3 cameras give the exact settings.
• Update the camera firmware, from the manufacturer's support pages only.

Worked example: what the copy costs on a 12 MP USB3 stream

Inputs: the LS-CMR-0166 delivers 4000 × 3000 pixels at 30 fps over USB3. Assume one byte per pixel (a Mono8-class format), a fixed 500 µs exposure typical for this kind of task, and 5 GB/s of usable host memory bandwidth — a fair estimate for a DDR4-class board doing a memcpy.

• frame_bytes = 4000 × 3000 × 1 B = 12,000,000 B ≈ 11.4 MiB
• data_rate = 12,000,000 B × 30 fps = 360,000,000 B/s ≈ 360 MB/s ≈ 2.9 Gbit/s
• frame_period = 1000 / 30 = 33.3 ms
• copy_time = 12,000,000 B ÷ 5,000,000,000 B/s = 2.4 ms

USB 3.0 carries 5 Gbit/s on the wire; after 8b/10b encoding and protocol overhead, figure 350 to 450 MB/s of sustained image data. At 360 MB/s the stream fits with little headroom — add a second camera on the same controller and it stops fitting, and the link then reports incomplete frames, not torn ones.

Decision: copy, always. The copy costs 2.4 ms against a 33.3 ms frame period — 7%. Even two copies per frame (frame object plus array, or copy plus encode) cost 4.8 ms, 15% of the period. Watch the 360 MB/s instead: below roughly 300 MB/s of image data, stop calculating and copy; above it, count bytes at every stage and keep the total inside the link's practical ceiling.

When the usual answer is wrong

• Torn in the vendor viewer too, copy or no copy. Your code is innocent. Cable, controller, firmware — in that order.
• Repeating bands across the whole frame. A strobe shorter than a rolling-shutter readout paints fixed bands aligned with the readout; a reuse tear moves with the race and shows one hard boundary.
• Appears only when a second camera starts. The controller was marginal already; the new stream pushed it over. Separate controllers, or cap each camera with DeviceLinkThroughputLimit.
• You never queue at all. Processing entirely inside the callback needs no copy, because the buffer is valid until you return. Wrapping an SDK frame buffer in a cv::Mat without copying is legitimate exactly then — the Mat must be finished before the callback returns.

Checklist before you escalate

• Copy inside the callback, before any put(); queue owned memory only.
• Hand every frame back to the driver, on every status, exactly once.
• Never block in the callback; use a bounded queue with drop-oldest.
• Re-run the bright/dark toggle test after any handler change — it is the only test that reproduces the worst case reliably.
• Confirm in the vendor viewer at identical settings before opening a fault.
• On Linux, raise the usbfs memory limit before judging frame drops.
• Update firmware only from the manufacturer's support pages.

If you are rebuilding the station around this, the camera finder and the accessories finder cover USB3 cameras and drag-chain-rated cables, and the system configurator walks camera, cable and host in order. A link that stays stubborn after this list belongs with our support team — free expert support comes with every order.