Out of memory in a video loop: fix the per-frame image leak

A loop that binarizes video and finds contours runs clean for minutes, then dies partway through a longer file: the allocator refuses a new image of a few hundred kilobytes, and the display layer returns a null frame. Nothing about that signature points at the file. It points at one lost allocation per frame — the classic C-style image leak. This page is for anyone maintaining capture-and-process loops written against the C image interface, where every cvCreateImage and cvCloneImage needs a matching cvReleaseImage. You will compute what one lost frame per iteration costs per second, rank the likely causes, prove the leak with a slope measurement, fix it with paired releases or reused buffers, and verify the result with a soak test.

The signature: minutes of clean frames, then a tiny allocation fails

The failure arrives from two layers at once. The GUI image class logs 'QImage: out of memory' and returns null; OpenCV's allocator raises its 'Insufficient memory' error naming the request it could not satisfy — 691,200 bytes in a representative case, which is exactly one 960 × 720 single-channel frame. Both messages land on a machine with several gigabytes free. A few hundred kilobytes refused on a machine with RAM to spare means the heap is exhausted or fragmented; it does not mean the file is too large.

The pattern of which files fail confirms it: short clips process start to finish, longer or heavier files kill the run partway. The variable is the number of frames processed, because the loss repeats on every frame. The same growth pattern underlies inspection loops that leak, and the Vision software memory leaks: find and fix inspection loop growth walkthrough covers the measurement side in depth.

Whether frames arrive from a file, a frame grabber or an area scan camera, the loop owns every buffer it creates. The causes below are ranked by how often each turns out to be the one.

Likely causes, in order of probability

1. Clone then overwrite. An image is created and the same pointer is reassigned before any release, so the first buffer becomes unreachable. Test: a create-versus-release counter pair that grows by a fixed amount per frame.
2. Releases skipped on exit paths. An early return, a break, or a thrown conversion error jumps over the release lines. Test: the counter grows only on iterations where a step fails or exits early.
3. Per-frame retention. Results, thumbnails or debug images are appended to a container each frame and never cleared. Test: the container grows by one element per frame, and the slope equals the stored payload size.
4. A leak below the loop. The capture backend or the display layer loses memory on its own. Test: your counters balance, yet the measured slope stays positive.

Why file size is a red herring

Decoding is streaming by design: the reader holds one compressed frame at a time, decodes it into one image, and hands it over. Written correctly, an hour of 640 × 480 video at 30 fps — 108,000 frames — costs a constant few megabytes while it runs. The size of the file on disk never enters your process.

The only quantity that scales with frames processed is memory you fail to hand back. 108,000 lost frames × 921,600 bytes = 99,532,800,000 bytes ≈ 99.5 GB of cumulative allocation requests — more than any machine survives, even though no single frame is large. That is why test clips pass every time and production files fail mid-run: the pass/fail line is frame count, not megabytes on disk.

The pattern that leaks: allocate, then overwrite the pointer

Every cvCreateImage and cvCloneImage returns the only address of a new buffer. Assigning over that pointer destroys the address; the buffer stays allocated until the process exits. Two lines in a typical binarization branch do exactly that:

IplImage* SecondImage = cvCloneImage(frame);  // clone A: a full color copy of the frame
SecondImage = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1); // pointer overwritten, clone A lost

Clone A was never needed: the very next line replaces it with a single-channel image. The same branch reassigns the working image the same way, so a second copy is lost on iterations that take that path. One lost image per frame is enough to sink a run; this branch loses one on every binarized frame.

The balance counter pinpoints it. Keep two integers, creates and releases, both initialized to 0; add 1 wherever an image is created or cloned, subtract 1 wherever one is released. After N frames the difference must equal the number of images intentionally still alive — 0 or 1 in a loop like this. On the broken branch the difference grows by exactly 1 per binarized frame; after the fix it holds at 0 across a 10,000-frame run. That is the test that proves the result.

What one lost frame per iteration costs

The loss per frame is just the pixel payload:

Worked example, 640 × 480 color: bytes_per_frame = 640 × 480 × 3 × 1 = 921,600 bytes. At 30 fps the loop loses 921,600 × 30 = 27,648,000 bytes/s ≈ 27.6 MB/s (decimal megabytes), or 27.6 × 60 ≈ 1.66 GB per minute. A free pool of 8 GB empties in 8,000,000,000 / 27,648,000 ≈ 289 s, which is 4 min 49 s. Treat that as an upper bound: the OS, the SDK and the GUI share the pool with your loop.

A vision controller such as the MV-VC3301H-128G66 ships with 8 GB of memory and is built for exactly these multi-camera loops.

On a 32-bit build the same math is harsher: the usable address space is about 2 GB, so 2,000,000,000 / 27,648,000 ≈ 72 s — the run dies in roughly a minute. Frame size and frame rate scale the loss linearly, which is why heavier files crash sooner:

One lost frame per iteration, illustrative budgets; double the frame rate and every loss doubles with it.

Measure the slope before you change code

Confirm the diagnosis first. Watch the process working set — Task Manager on Windows, top or ps on Linux — and record it every 100 frames.

1. Compute the slope: memory growth divided by frames processed. The broken 640 × 480 loop reads about 921,600 bytes per binarized frame. A slope of one frame payload means one lost image per frame; two or three payloads mean two or three images.
2. Bisect by disabling. Run once with display calls removed, once with the binarization branch stubbed out. Whichever change flattens the slope locates the layer that loses the memory.
3. Cross-check with the counter pair: a difference growing by 1 per frame while the slope equals one payload is a closed case.
4. Re-check after the fix: the slope must stay flat — within a few hundred kilobytes — across a 10,000-frame soak.

Fix: one release per allocation, on every exit path

Three rules fix the leak and keep it fixed:

• One allocation, one release, same iteration: every cvCreateImage or cvCloneImage pairs with exactly one cvReleaseImage before the next frame.
• Never overwrite a live pointer: release what a pointer holds before assigning it again.
• Initialize every image pointer to NULL at declaration, so releasing a never-created image frees nothing instead of freeing garbage. cvReleaseImage accepts a NULL pointer safely.

Applied to the binarization branch, the fix also removes the pointless clone — one working image instead of three:

IplImage* gray = cvCreateImage(cvGetSize(frame), IPL_DEPTH_8U, 1); // one working image per frame
cvCvtColor(frame, gray, CV_RGB2GRAY);
cvThreshold(gray, gray, bin1, 255, threshold_type); // in place: source and destination are the same image
ShowResultImage(gray);
cvReleaseImage(&gray); // freed before the next frame

Nothing in the branch needs a copy of the full frame, so the clone disappears entirely. Early exits and thrown errors still need care: every return between create and release must release first, or the exception path strands the buffer. C-style code makes that easy to get wrong, which is the strongest argument for the section after this one.

Better: allocate once and reuse, or let cv::Mat own the memory

The durable fix removes per-frame allocation altogether.

• Pre-allocate before the loop: create the working image once, before frame 0, and let cvtColor and cvThreshold write into it every frame. While size and type stay unchanged, the same buffer is reused, so steady-state allocation is zero.
• Prefer cv::Mat: it counts references and frees its buffer when the last header leaves scope, so temporaries clean up on every exit path, exceptions included. A default-constructed Mat starts empty and allocates on first use.
• Wrap, do not copy, camera buffers: when frames come from an SDK rather than a file, wrap the delivered buffer in a Mat header. Wrap an SDK frame buffer in an OpenCV cv::Mat without copying shows how.
• Design reuse in: a fixed ring of buffers that the loop rotates beats a fresh allocation per frame. The sizing reasoning is in Reuse image buffers instead of per-frame allocation.

When the slope survives a balanced loop

If your counters balance and the slope still climbs, the loss is below the loop:

• Capture backend: public bug reports against certain backend builds describe growth from a few kilobytes per thousand frames up to roughly 1 MB per minute. Swap the backend or move to a maintained build, then re-measure the slope the same way.
• Conversion failures: a cvtColor call that throws on an unexpected pixel type can strand its buffers. Guard the conversion so a bad frame skips the frame, not the cleanup.
• Display layer: per-frame pixmaps and scaled GUI images accumulate on the GUI side. Disable display entirely and compare the two slopes.

The same ownership discipline applies when an SDK hands you buffers from another language runtime. C# camera memory leak: manage image buffer ownership walks that boundary.

Prevention: budget the memory and soak-test every build

Budget at design time: simultaneous buffers × bytes per frame should land in the megabytes. A four-camera cell at 640 × 480 with three live buffers per camera holds 4 × 3 × 921,600 = 11,059,200 bytes ≈ 11 MB steady-state. If that steady-state number sits in gigabytes, something is being retained.

Soak-test every build: a 10-minute run at 30 fps is 18,000 frames. Assert that the working set at frame 18,000 sits within a few megabytes of frame 0, and fail the build when it does not. In production, log the working set per minute; a linear climb is an alarm, a flat line is a pass.

For the neighboring failure modes — frames arriving faster than they can be processed, timeouts, pacing — see Acquisition faster than processing: fix the frame rate gap and the systems topic page. If this loop runs on a shared desktop, moving it to dedicated vision controllers with camera ports and built-in light channels gives the run a predictable pool. Send your frame size and frame rate through the contact page and we will help you size it.