Return the camera buffer after GStreamer is done, not after push

Your camera SDK hands you frames; a GStreamer pipeline turns them into a window on the operator's screen. Somewhere in that glue code an assumption about who owns the pixels is wrong, and the display starts blending one frame's rows into another. This is the early-return bug: the code returns the acquisition buffer the moment it pushes it, while the pipeline still holds that memory in several places. Below, you will count the buffers a display pipeline really holds, see why push-and-return works only while the host keeps up, place the buffer-release notification correctly, and verify the fix with counters that survive an hour of loaded running.

Why does the displayed image corrupt while the camera stream runs?

The corruption means the pipeline is still reading frame memory that your code has already returned to the acquisition library, so the next frame's pixels overwrite an image that GStreamer has not finished drawing yet.

On screen it looks like a lower third that belongs to the previous exposure, or content that jumps half a frame while nothing moved in front of the camera, or horizontal bands mixing two exposures. It worsens when the host is busy — a logger writing, a browser open, a second camera streaming — and it vanishes the instant you pause acquisition. That signature, damage that follows frame content, separates it from electrical faults, which corrupt regardless of what the camera saw. If the tear already exists when your callback receives the frame, before any display code runs, you have the acquisition-side variant instead; Torn frames in a camera callback: copy the buffer before you queue it covers that path.

How many buffers is a GStreamer pipeline holding at once?

A display pipeline holds one frame on screen plus every frame waiting in its queue elements — two is the floor, and a loaded host routinely holds four to six — so never design for exactly one.

Count the waiting places between your push call and the screen: the buffer the sink is presenting now, the sink's slot for the next one, and every queue element in the chain, each holding as many frames as its configured depth allows. A minimal display path with one queue therefore carries two frames in flight; a chain with two queues of depth two reaches six. A host that keeps up runs the queues near-empty, and a host that falls behind fills them to the ceiling — which is why the same code corrupts on one machine and not another.

GStreamer tracks each GstBuffer by reference count: every element still working on a frame holds a reference, and the memory is released only when the last one drops. The buffer's release notification — the callback that runs at that final drop — is the only trustworthy consumer-done signal in the system. A timer or a frame counter on your side is a guess about pipeline internals. If you instead keep newest-only buffers at the transport layer, the same counting logic decides your latency; see Stream buffer handling mode: set newest only on the transport layer.

Why does returning the camera buffer right after push corrupt the image?

Returning the buffer at push time is a bet that the pipeline finishes each frame before the next arrives, and on a loaded host the camera wins that race, so the library reuses memory that queued frames still reference.

Zero-copy wrapping is the right idea — a GstBuffer pointing at the SDK's frame memory avoids a 12 MB memcpy per frame at 12 MP rates, and the same no-copy discipline for analysis code is in Wrap an SDK frame buffer in an OpenCV cv::Mat without copying. The failure is purely the return timing, and it runs step by step: you push frame N, whose buffer wraps the SDK memory; a queue parks it behind frames still in flight; the camera delivers N+1; your early return marks the memory free, and the library fills it with N+1 while the pipeline still intends to draw N. When the sink finally presents N, part of its rows are N+1.

How do I know whether my host can keep up with the frame rate?

Compare one number with another: the frame period, 1000 divided by the frame rate in hertz, against the per-frame host cost you measure; if the host cost is larger, buffers pile up in flight and an early return starts corrupting.

The two streams in the arithmetic below use the same 12 MP IMX226 sensor class, on two different links:

Give the host a per-frame cost of 45 ms for unpack, scaling and draw — an example input; measure yours with a stopwatch around the render call, on the machine you will actually deploy, because an underpowered industrial mini PC changes the verdict more than any library setting.

Against the USB3 stream the budget fails: 45 ms of cost against a 33.3 ms period leaves an 11.7 ms deficit per frame, the queues back up within a second, and the pipeline holds three to five frames in flight. Against the GigE stream the same host passes: 45 ms against a 111 ms period leaves 66 ms of slack, each frame drains before the next lands, and even an early return happens to be safe. Same code, opposite verdicts, because the race is between two numbers you can compute before writing any glue.

Two consequences follow. Latency: every frame held in flight adds its period to what the operator waits — three frames at 30 fps is about 100 ms of queue alone — and measuring that honestly is its own exercise in Measuring camera latency glass to glass the right way. Bandwidth: 360 MB/s fits USB3 and is six times USB 2.0's 60 MB/s line rate, so a wrong port or hub shows up as dropped frames and a low fps counter, never as blended rows.

When should my code return the camera buffer to the acquisition library?

Return each acquisition buffer from the buffer-release notification — the destroy callback GStreamer runs when the pipeline drops its last reference to the wrapped GstBuffer — because that is the first moment no element can still read the pixels.

That notification is not a refinement; it is the only instant at which a return cannot corrupt anything, since no reference remains to read through. In code:

/* Illustrative C. GstBuffer, gst_buffer_new_wrapped_full,
   GST_MEMORY_FLAG_READONLY and GDestroyNotify are documented GStreamer API.
   Frame, Stream, stream_return_buffer and g_weak_stream stand in for your
   acquisition library's own names. */
static void frame_released(gpointer user_data) {          /* GDestroyNotify */
    Frame *frame = user_data;              /* the buffer the library lent you */
    if (stream_is_live(g_weak_stream))     /* releases still arrive in shutdown */
        stream_return_buffer(g_weak_stream, frame);
}

GstBuffer *wrap_frame(Frame *frame) {
    return gst_buffer_new_wrapped_full(GST_MEMORY_FLAG_READONLY,
                                       frame->data, frame->size, /* data, maxsize */
                                       0, frame->size,           /* offset, size */
                                       frame, frame_released);
}

Three details matter more than the call itself. The read-only flag makes any element that would edit the buffer in place copy first, instead of writing pixels the acquisition library still owns. The notification can fire on a GStreamer streaming thread, so keep it to a single return call: no locks the pipeline holds, no allocation, no interface work. And the liveness guard exists because releases keep arriving during teardown, which is the next section.

Prove the fix with counters rather than by watching the display. Stamp a sequence number on each frame at push and log it again at release, then check three properties: every push gets exactly one release, so nothing leaks or returns twice; the release of frame N trails its push by roughly the in-flight depth times the frame period — about 100 ms at 30 fps with three frames in flight — and that gap stays stable under load; and a ten-minute run with the CPU saturated shows zero blended frames while the counter sits at three to five.

What happens to the release callback when I shut the pipeline down?

Release callbacks can still fire after you stop the acquisition stream, because the pipeline drains its queues when it shuts down, so the callback must hold a weak reference to the stream and return silently if that stream is already gone.

Shutdown order does most of the work: stop feeding new frames, set the pipeline fully down and let it drain, and only then stop or close the acquisition stream. Draining delivers the queued frames, fires their release notifications, and empties the pipeline while the stream object still exists to receive the returns. Reverse the order and releases arrive addressed to a stream your code has already freed — a use-after-free one layer below this bug, covered in Heap corruption in vision software: fix image buffer lifetime.

Never stop the pipeline from inside the callback: it runs on a pipeline thread, so a synchronous stop waits for the thread that called you. If a conversion or copy step stalls during the drain instead, Camera frame conversion hangs: check buffer ownership walks that cousin failure.

Multi-camera hosts add nothing new: each stream returns its own buffers through its own notifications, whichever transport carries the frames. Hosts that aggregate several GigE cameras on one box often hit a link budget before they hit a code bug, and a NIC that adds a second powered port is one way to buy headroom:

The ownership rule itself stays per stream, so the fix in your glue code is the same one callback per camera.

Is the corruption the software's fault or the cable's?

Blended or jumping content that follows the frame data is software; damage that ignores content — a black line, a split or shifted image, static-like noise — points at the transport, and a ten-minute swap test settles it.

Run the acquisition library's own viewer on the same camera and cable, with no GStreamer in the path. Clean frames there clear the transport and camera and condemn your glue code; corruption there sends you to the link instead. Content-independent damage maps to physical causes: a steady black line says cable or connector, and damage that moves when you wiggle the jacket says conductor or strain relief — which is why a camera on a moving drag-chain run gets a flex-rated cable from the camera cables category rather than a patch lead.

Bandwidth starvation imitates neither pattern. A link that cannot carry the rate drops or repeats whole frames, and you see stutter or a low fps counter, not blended rows. When the render-budget arithmetic says your link is marginal, size the next stream from the camera finder — the ownership rule holds across area scan cameras on every interface, so the fix in the glue code does not change with the transport.

How do I keep the bug from coming back on the next camera?

Make one ownership rule non-negotiable: an acquisition buffer is refilled only after its release notification, and any consumer that outlives the call — display, snapshot, second library — takes a reference or a copy at capture time.

• Wrap read-only. Set the read-only flag when you wrap SDK memory, so in-place edits force a copy instead of writing pixels the library still owns.
• One return path. Every return happens in the release notification, including error and teardown paths; search the code for the push call and audit each one.
• Guard the callback. Weak reference to the stream, no locks the pipeline holds, no pipeline calls from inside it.
• Count in flight. Push and release counters stay visible in production; a gap pinned at the queue limit is the early warning that a host is falling behind.
• Test loaded. Reproduce at the highest resolution you ship with the CPU saturated — the idle lab machine keeps up and hides the bug.
• Copy on capture. Snapshots and save-on-demand take a reference or a copy at trigger time, never a pointer into wrapped memory.

When you are sizing the next stream rather than fixing this one, put the frame period against the measured host cost before ordering: the camera configurator pairs the sensor with an interface the render budget can survive, the systems topic page collects the acquisition pipeline's other failure modes, and the contact page reaches the applications team if you want a second pair of eyes on the pipeline.