Wrap an SDK frame buffer in an OpenCV cv::Mat without copying

cv::VideoCapture will not open a GigE Vision or USB3 Vision camera, and no OpenCV setting changes that: stock builds ship backends for UVC, V4L2 and network streams, but none for the industrial camera standards. The working pattern is to grab frames with the camera's own SDK and lay a cv::Mat header directly over the SDK's pixel buffer, which is one constructor call and zero copies. This guide gives you the constructor, the pixel-format-to-type table, the stride rules that decide whether the image renders straight or sheared, a five-step wrap with a verification test, and the bandwidth arithmetic for a real 12 MP USB3 camera.

Why can't cv::VideoCapture open my industrial camera?

Stock OpenCV builds have no backend for GigE Vision or USB3 Vision, so the camera never appears to cv::VideoCapture; a generic industry-standard backend has been an open feature request on the OpenCV tracker since 2024, filed as issue 26398 in November of that year. Enumeration for these cameras lives in the vendor SDK, because both standards define their own discovery and streaming protocols on top of ordinary Ethernet and USB hardware. OpenCV's video I/O layer has no module that speaks them.

For a deeper look at that first failure, read Why OpenCV VideoCapture cannot open a GigE Vision camera. If your camera is a plain UVC device, skip all of this: no config file needed, get a UVC camera streaming over USB shows the direct path.

The pipeline you build instead has three parts. The SDK enumerates and configures the camera and hands your code a frame structure holding a pointer, width, height, stride and pixel format. You wrap that memory in a cv::Mat. OpenCV then processes pixels it never had to allocate. Compiling and linking the SDK into your project is its own step, and Camera SDK undefined reference: diagnose C++ linker errors covers the undefined-reference errors that appear the moment the library is left off the linker line.

Which cv::Mat constructor wraps a buffer without copying?

Use the external-data constructor cv::Mat(int rows, int cols, int type, void* data, size_t step = AUTO_STEP): it builds a header over the SDK's memory, copies no pixels, and never deallocates memory it did not allocate. The cv::Mat class reference documents this as the user-allocated-data case. Three rules follow from it:

• step is in bytes, not pixels and not elements. A 4000-pixel Mono8 row is 4000 bytes; a 4000-pixel RGB8 row is 12,000 bytes.
• AUTO_STEP tells OpenCV to assume packed rows and compute the step itself. Pass an explicit stride whenever the SDK pads rows.
• The reference counter that normally frees Mat memory is not attached to external data, so header copies share the same SDK pointer until you clone.

How do I map my camera's pixel format to an OpenCV type?

Match bytes per pixel first: Mono8 and every 8-bit Bayer format wrap as CV_8UC1, RGB8 wraps as CV_8UC3, Mono16 wraps as CV_16UC1, and packed YCbCr or packed 10/12-bit formats wrap as nothing. The last group stores several pixels per byte group, so no rows-times-cols array of one fixed type describes them.

Two traps sit in this table. First, camera RGB8 wrapped as CV_8UC3 keeps RGB order while most OpenCV colour code assumes BGR, so swap with cv::cvtColor and COLOR_RGB2BGR or your reds and blues trade places. Second, demosaicing allocates a new CV_8UC3 image at three times the bytes; that is a copy you choose to make, not one the wrap saves you from. Lock the pixel format in the SDK before the run starts so the type you wrap never changes mid-stream.

Where does the row stride come from?

The stride is the row pitch in bytes that the SDK reports with every frame; it equals width times bytes-per-pixel only when rows are packed, and row padding breaks that equality. SDKs align the start of each row to a boundary, commonly 16, 32 or 64 bytes, because aligned reads and DMA transfers are faster. A 4000-pixel Mono8 row is exactly 4000 bytes, which is not a multiple of 64, so a 64-byte-aligned driver pads every row to 4032 bytes. The frame grows from 4000 × 3000 = 12,000,000 bytes to 4032 × 3000 = 12,096,000 bytes, and the last 32 bytes of every row are filler your Mat must skip.

Pass AUTO_STEP on a padded frame and OpenCV assumes 4000-byte rows; every row then starts 32 bytes later than the header expects, and the image shears. The tell is a clean top band and a skew that grows linearly downward, one pad-width per row.

How do I wrap and verify a frame in five steps?

Set the camera to Mono8 or BayerRG8, grab one frame, build the Mat from the SDK's pointer, width, height and reported stride, then assert the header fields before you process anything. The whole wrap is five steps and contains no copy.

1. Lock the format. In the SDK set the pixel format to Mono8 or an 8-bit Bayer for greyscale work, RGB8 for ready-made colour.
2. Grab one frame. Call the SDK's retrieve function; it fills a frame structure. Read four values from it: base pointer, width, height, stride in bytes.
3. Sanity-check the stride. Compute width × bytes-per-pixel. A larger SDK stride means padded rows, which is normal; a smaller one means you are reading the wrong field.
4. Construct the Mat.

#include <opencv2/core.hpp>

// Names below are ours; your SDK reports the same values under its own field names.
unsigned char* pixelData = nullptr;  // grab call fills this: base address of the frame
int widthPx     = 4000;              // frame width in pixels
int heightPx    = 3000;              // frame height in pixels
int strideBytes = 4032;              // SDK-reported row pitch in bytes (padded)

cv::Mat wrapped(heightPx, widthPx, CV_8UC1, pixelData,
                static_cast<size_t>(strideBytes));

5. Assert the header before processing.

CV_Assert(wrapped.rows == heightPx);
CV_Assert(wrapped.cols == widthPx);
CV_Assert(wrapped.elemSize() == 1);  // Mono8: one byte per pixel
CV_Assert(wrapped.step == static_cast<size_t>(strideBytes));
CV_Assert(wrapped.isContinuous() ==
          (static_cast<size_t>(strideBytes) ==
           static_cast<size_t>(wrapped.cols) * wrapped.elemSize()));

Then prove the Mat reads live sensor data rather than stale memory: cap the lens and confirm cv::mean(wrapped)[0] sits near zero, uncap it and watch the mean jump. A mean that never moves between capped and uncapped frames means the pointer is not the frame you think it is.

When must I copy the frame instead of wrapping it?

Copy the moment the image must outlive the buffer: once your SDK requeues or reuses its frame memory, a wrapped Mat reads stale pixels, so call wrapped.clone() before the buffer goes back to the queue. Most SDKs run a ring of fixed buffers, and the frame you hold is recycled on the next grab. Wrap, process and finish inside the grab call, and the zero-copy path is safe. Any of these pushes the image past that moment:

• You return the Mat from the grab callback or hand it to another thread.
• You write it with cv::imwrite or a VideoWriter and the encoder runs later.
• You keep a history of frames for background subtraction or stereo matching.
• You cut an ROI and downstream code needs contiguous memory.

In all four, clone: it allocates OpenCV-owned memory and copies the pixels once, 12,000,000 bytes for a 12 MP Mono8 frame. Budget that copy where the lifetime demands it and nowhere else. Stereo frames saved wrong: copy Mat data from the SDK buffer shows what saved files look like when this copy is skipped.

Will a 12 MP stream fit my USB3 or GigE link?

One 12 MP Mono8 frame is 12,000,000 bytes, so 30 fps moves 360 MB/s: that fits a USB 3.0 link with margin, while a single 1 GbE link carries only about 100 to 110 MB/s of image payload. Here is the arithmetic on a concrete camera. This example uses the LS-CMR-0166, a 12 MP USB3 camera with a 4000 × 3000 sensor at 30 fps.

• One Mono8 frame: 4000 × 3000 × 1 byte = 12,000,000 bytes, about 11.4 MiB.
• Stream rate: 12,000,000 × 30 = 360,000,000 bytes/s, or 360 MB/s.
• USB 3.0 line rate is 5 Gbps = 625 MB/s, and cameras typically deliver 350 to 450 MB/s of it, so 360 MB/s fits with headroom.
• Forced to RGB8 in the camera, the same frames need 36,000,000 × 30 = 1,080,000,000 bytes/s, or 1080 MB/s, which is more than the link carries.

The decision follows: leave the camera in BayerRG8, wrap as CV_8UC1, and demosaic on the host only when the algorithm needs colour. The demosaiced BGR frame is 36,000,000 bytes and you create it when you choose to, not automatically 30 times a second.

On GigE the ceiling is hard. At 108 MB/s a 12 MB frame tops out near 9 fps, which is exactly where a 12 MP GigE camera sits. The same IMX226 sensor ships on both interfaces: a 12 MP GigE camera at 9 fps and a 12 MP USB3 camera at 30 fps:

When two USB3 cameras share a host, the loads add: two 360 MB/s streams will not fit one 5 Gbps controller, so give each camera its own controller or cut the frame rate. Two USB3 cameras at full resolution stream: raise the host USB buffer covers the host-side tuning that follows.

Which cameras does this wrap pattern apply to?

Every camera you reach through an SDK, whether GigE Vision, USB3 Vision, CoaXPress or Camera Link, hands your code a pointer, a size and a stride, and this same constructor wraps them all. Anything in area scan cameras works with exactly this pattern, because the SDK normalizes discovery for you. Browse the camera finder by interface, or start the camera configurator to build the whole chain. On tightly integrated machines, board-level cameras drop into the same pipeline with the same wrap; the LS-BCM-0003 is a 3.1 MP USB3 board camera with a 1/1.8" global shutter at 120 fps.

Three checks before you order:

• Format control. The SDK must let you lock the pixel format to one wrappable type; confirm Mono8, an 8-bit Bayer or RGB8 is offered.
• Stride reporting. The frame structure must expose the row pitch in bytes, per frame, so padded rows are handled correctly.
• Link headroom. Keep the stream under about 80% of the link's real payload capacity; the 360 MB/s example uses 58% of 625 MB/s, which is why it runs without drops.

More integration patterns like this one sit under the systems topic. If you want a second pair of eyes on a specific camera-and-SDK combination, use the contact page; free expert support comes with every order.