Convert a raw camera buffer to a processing-ready image

A camera SDK hands you a pointer and a byte count. Your vision library wants width, height and a type. Every failed conversion lives in the gap between them: the colour scene that shows as a grey mosaic, the image sheared into diagonals, the 12-bit frame that displays almost black. This guide turns a raw acquisition buffer into a display-ready image in five steps: read the pixel format, compute bytes per line and the stride, wrap the pointer without a copy, keep Bayer raw, and unpack packed formats. A worked example with the arithmetic shown gives you numbers to check your own camera against.

What you need

Collect five facts before you write conversion code. Four come from the SDK, one from your library.

1. The pixel format the camera is actually delivering while streaming, such as Mono8, BayerRG8 or Mono12p. Read it from the running acquisition, not from your configuration file.
2. Width in pixels and height in lines, as the camera reports them.
3. The payload size of one buffer in bytes, plus the stride or line pitch if the SDK exposes it.
4. The frame rate you need, in fps, because it decides whether a per-frame copy is affordable.
5. Your library's image constructor signature: which arguments take width, height, element type and step.

Put a uniform diffuse target in front of the lens for the verification checks later; any even grey surface works.

Step 1 - read the pixel format the camera is delivering

Pixel format names follow the GenICam pixel format naming convention: the letters give the layout and the number gives bits per pixel. Mono8 is 8-bit mono at 1 byte per pixel. BayerRG8 is raw colour in the same 1 byte per pixel footprint, with red, green and blue samples interleaved in a 2 × 2 pattern; the two letters give the order of the first row. Mono12p is 12-bit packed at 2 pixels in 3 bytes. Mono12 without the p is unpacked, one 12-bit value stored in a 16-bit container at 2 bytes per pixel. YUV422 and RGB8 are colour-processed formats at 2 and 3 bytes per pixel.

Set the format deliberately: unpacked formats cost link bandwidth, packed formats cost unpacking CPU time on the host. If you are unsure how much bit depth the task needs, the choice between 8, 10 and 12 bits is laid out in machine vision camera bit depth: choosing 8, 10 or 12 bits.

A mono GigE camera in the class below streams these standard formats straight into your receive buffers.

Step 2 - compute bytes per line and find the real stride

Three numbers decide the whole conversion: bytes per pixel from the format, width in pixels, and the stride in bytes per row.

Compute bytes per line from the format first. Then get the stride. If the SDK reports it directly, use that value. If it reports only the payload size, divide by the height; the result is authoritative because the payload is what the driver actually allocated. When the stride equals width × bytes per pixel there is no padding. When it is larger, the difference is per-row padding, and any constructor call that omits the stride shears the image.

Camera drivers commonly pad rows to 4 bytes; some host SDKs pad to 8, 16 or 32 for aligned memory access. The payload divide settles it in one step. The buffer-size arithmetic behind multi-channel formats is worked through in check multi-channel image buffer size before processing.

Step 3 - wrap the pointer with no copy when the stride allows

The fast path is a wrap, not a copy. Most vision libraries can build an image around foreign memory: you pass the pointer, width, height, an element type and a stride, and the library treats the camera buffer as its own image. No byte moves.

Zero-copy works when four conditions hold:

1. The constructor accepts an external pointer plus a stride parameter.
2. The element type matches the buffer layout: 8-bit mono or raw Bayer maps to an 8-bit single-channel type, a 16-bit container to a 16-bit single-channel type.
3. The stride either equals width × bytes per pixel or is passed explicitly.
4. The buffer stays valid for the life of the image. Wrap inside the callback that owns the buffer and finish processing before releasing it back to the SDK's queue.

If any condition fails, do one explicit copy per frame into the library's own container. The worked example below puts a number on when that copy stops being affordable. For what the bytes in the buffer actually represent, see image bytes vs pixel data: decode before processing, and for the cleanup discipline around borrowed buffers, camera handle cleanup after Python script errors.

Multi-camera hosts should budget the receive path too: every camera port needs bandwidth headroom, which is where the network cards below come in.

Step 4 - keep Bayer raw and let the library decode colour

A raw colour buffer is single-channel data, 1 byte per pixel for BayerRG8, exactly the same footprint as Mono8. Pass it to the library as a raw single-channel image tagged with its Bayer order, then run the library's debayer or raw-to-RGB conversion on display or on demand. The library's debayering is optimised and usually tuned better than a quick in-house interpolation.

Two failures dominate this step. Displaying the Bayer buffer as mono shows a grey 2 × 2 mosaic across the whole scene. Treating it as 3-byte interleaved RGB produces a frame twice the expected size with garbage channels, the classic symptom behind colour frame comes out doubled: fix Mat type pixel format mismatch. And if the scene renders grey despite a colour camera, the pixel format itself fell back to a mono default; that repair is walked through in GigE camera shows no color image: fix the pixel format default.

A colour sensor that streams raw Bayer in the same footprint as mono:

Step 5 - unpack 10- and 12-bit packed formats once

Packed formats save link bandwidth and cost host CPU. Neither Mono10p (4 pixels in 5 bytes) nor Mono12p (2 pixels in 3 bytes) matches a library image type directly, so the plan is one unpack per frame into 16-bit buffers, after which everything behaves like any other 16-bit image.

Mono12p stores two pixels in three consecutive bytes b0, b1 and b2, little-endian, each byte holding 0 to 255:

Unpacked 12-bit formats need a byte-order check instead. One camera stores the 12-bit value in the low byte of each 16-bit word, another in the high byte; the wrong assumption shifts every value by a factor of 16 and shows up as noise in bright areas. A full worked Bayer packed conversion, with a test image to compare against, is in packed Bayer 12-bit conversion to a 16-bit image.

Worked example - converting a 2048 × 1080 Mono8 frame at 60 fps

Take a typical mid-size area-scan frame. The sizes are illustrative; your camera's reported width, height, format and payload replace them one for one.

Inputs:

• Width 2048 px, height 1080 lines, from the camera's streaming report.
• Pixel format Mono8, so 8 bit per pixel, 1 B per pixel.
• Payload size reported by the SDK: 2,211,840 B for one full buffer.
• Target 60 fps over one GigE port, which carries about 115 MB/s of usable payload once protocol overhead is paid.

Rule in words: bytes per line equals width × bytes per pixel; the stride equals payload ÷ height; a no-copy wrap is possible exactly when the stride equals the computed bytes per line.

Arithmetic:

• Bytes per line: 2048 px × 1 B/px = 2048 B.
• Stride: 2,211,840 B ÷ 1080 lines = 2048 B per line.
• 2048 B = 2048 B, so there is no row padding.
• Frame size: 2048 B × 1080 lines = 2,211,840 B ≈ 2.21 MB.
• Bandwidth at 60 fps: 2,211,840 B × 60 = 132,710,400 B/s ≈ 133 MB/s, which is 1.06 Gbit/s.

Result: wrap the pointer with height 1080, width 2048, an 8-bit single-channel type and stride 2048 B. Nothing copies, and the library displays the frame directly.

Decision: 133 MB/s exceeds the roughly 115 MB/s one GigE port carries, so 60 fps of this frame does not fit a single GigE link. Either drop the rate to 115 ÷ 2.21 ≈ 52 fps, shrink the ROI, or move the camera to a 2.5G, 10GigE or CoaXPress link; the interface trade-offs are compared in 5GBASE-T camera interface: bandwidth, cables and distance, and camera burst capture faster than link bandwidth: use the buffer covers riding out short overruns with the SDK buffer. Keep the wrap either way: a per-frame copy would add another 133 MB/s of host memory traffic.

The padded variant closes the loop. At 2047 px width the same Mono8 format needs 2047 B per line, and a 4-byte-aligned allocator rounds the stride up to 2048 B. The wrap then must carry the stride explicitly; a constructor without a stride parameter forces one copy of 2.21 MB per frame.

Verify the conversion before you trust it

Four checks catch every wrong value above, and they take under a minute:

1. Print the wrapped image's width, height and element type and compare them against the camera's report; a mismatch here is the top cause of downstream crashes, and the habit is described in converted image object pixel format: verify before processing.
2. Point at the uniform target. The histogram should form one smooth hump with the mean where your exposure puts it, and no periodic structure.
3. Check the brightness ceiling. On a bright target a 12-bit image should reach values near 4095; a ceiling near 255 means the unpack collapsed to 8-bit somewhere along the way.
4. Measure the achieved fps against the camera's rated rate. A large shortfall usually means a hidden per-frame copy or a buffer queue too shallow for the burst pattern.

Common mistakes and the value that causes them

Each symptom in the table maps to exactly one wrong input, which makes diagnosis fast:

Hardware that reports clean buffer facts

The conversion only needs an SDK that states the pixel format, the size and the payload plainly. GigE and USB3 area scan cameras expose all three as standard GenICam features, and CoaXPress cameras deliver the same facts through a frame grabber. The card below is an octo CXP frame grabber that hands camera streams to host memory as buffers, subject to exactly the same stride rule.

To pick hardware by these numbers, filter the area scan cameras by interface and resolution in the camera finder, sort the host cards in the accessories finder, or start at the camera step of the system configurator. If your numbers do not close, send the camera model, pixel format and frame rate through the contact page and we will work the arithmetic with you. For the display side after conversion, keep the on-screen region tight using the approach in machine vision image display ROI design for responsive tools, and browse the rest of the systems guides for the surrounding pipeline.