You capture frames through a CUDA-accelerated camera pipeline, and OpenCV wants NumPy arrays. The conversions people reach for first — per-pixel loops, encoded snapshots, np.hstack over two host copies — burn hundreds of MB/s and blow the frame budget on an embedded module. This guide shows the path that moves each frame once: map the CUDA allocation into host memory, wrap it with np.frombuffer, reshape to (height, width, 4), and hand the view to cv2 as BGR. You get the byte arithmetic for 1080p and 12 MP frames, a single-copy way to join two camera views on the GPU, and the point at which the frame should never reach the host at all. Everything runs on ordinary NumPy and OpenCV calls; the only library-specific step is the mapping call your capture library provides.
How to convert a CUDA image to a NumPy array for OpenCV
Map the CUDA allocation into host-visible memory once, wrap it with np.frombuffer, and reshape to (height, width, 4): a 1920 × 1080 RGBA frame becomes a NumPy array of shape (1080, 1920, 4) holding 8,294,400 bytes, and cv2 consumes it after a single cvtColor call.
Two documented routes cover every case, and neither copies pixels on the host. If your capture library ships a mapping helper — the name in common CUDA capture libraries is cudaToNumpy() — call it once per image object and keep the returned array: the mapping is persistent, so the array and the CUDA allocation share one block of memory for the buffer's life. If you hold a mapped host pointer instead, the documented NumPy pair np.frombuffer() plus reshape() wraps it in one line. What you must not do is loop over pixels in Python: a 1080p frame holds 2,073,600 pixels, and at one microsecond per pixel that is 62 seconds of work per second of 30 fps video.
- Capture a frame into the library's CUDA image object.
- Map it once:
rgb = cudaToNumpy(img), or wrap a mapped host buffer withnp.frombuffer(buf, dtype=np.uint8, count=H*W*4).reshape(H, W, 4). - Convert to the channel order OpenCV expects:
bgr = cv2.cvtColor(rgb, cv2.COLOR_RGBA2BGR). - If a later call writes in place, make your one copy here:
work = bgr.copy(). - Run the test in the block below before you trust the pipeline.
import numpy as np
import cv2
W, H = 1920, 1080 # int: frame geometry (illustrative values)
img = source.capture() # illustrative: the library's CUDA image object
# Mapping helper documented in common CUDA capture libraries.
# Call once per image object; the returned array shares the allocation.
rgb = cudaToNumpy(img) # NumPy view, shape (1080, 1920, 4)
# Alternative when you hold a mapped host buffer of exactly
# H*W*4 = 8,294,400 bytes (documented NumPy APIs, no copy):
# rgb = np.frombuffer(buf, dtype=np.uint8, count=H*W*4).reshape(H, W, 4)
# The documented OpenCV conversion:
bgr = cv2.cvtColor(rgb, cv2.COLOR_RGBA2BGR) # (1080, 1920, 3), uint8 BGR
# Test that proves the result:
assert rgb.shape == (1080, 1920, 4) and rgb.dtype == np.uint8
assert bgr.shape == (1080, 1920, 3) and bgr.dtype == np.uint8
print(bgr.shape, bgr.dtype, float(bgr.mean()))
The two asserts are the whole verification: shape proves the geometry, dtype proves the byte width, and a mean that is neither 0 nor 255 proves the mapping sees live pixels rather than an empty allocation. The same reshape-then-verify pattern applies to plain host buffers from any camera API — converting a raw camera buffer to a processing-ready image walks that side step by step.
Does np.frombuffer copy the frame?
No: np.frombuffer wraps memory that already exists and moves zero bytes; the only unavoidable traffic is the device-to-host transfer itself, 8,294,400 bytes per 1920 × 1080 RGBA frame — about 2.8 ms at an effective 3 GB/s, 6.9 ms at 1.2 GB/s.
np.frombuffer records a pointer, a dtype and a length, and costs microseconds. A persistent mapping goes further and shares memory with the CUDA allocation, so call the helper once per buffer at start-up rather than once per frame; from then on the capture writes pixels and your array simply sees them. Treat the wrapped array as read-only: every OpenCV function that only reads its input — cvtColor, matchTemplate, threshold — accepts it, and the first in-place writer needs the single copy from step 4.
Do not let the driver compress on the way through either: an encoded frame costs a decode before cv2 can touch it and throws away detail you paid for. Unencoded capture to a NumPy array puts numbers on that. And if a mapped image comes out garbled rather than merely wrong-coloured, suspect row pitch or plane layout before the copy — checking pitch and planes is the first test.
What shape and pixel format does OpenCV need?
cv2 wants a C-contiguous uint8 array — (height, width, 3) in BGR order for colour, (height, width) for mono — so an RGBA frame needs exactly one call, cv2.cvtColor(frame, cv2.COLOR_RGBA2BGR), and mono8 needs only the reshape.
Channel order and channel count are the only layout facts cv2 reads. An RGBA8 frame arrives as (1080, 1920, 4) and comes out of cvtColor as (1080, 1920, 3). An 8-bit mono frame needs no channel axis at all: reshape to (1080, 1920) and cv2 reads it as grayscale, converting to three channels with cv2.COLOR_GRAY2BGR only when a drawing function demands it.
Depth and disparity arrive as float32 (height, width), which maps to CV_32F and works directly with filters such as cv2.GaussianBlur. Keep 16-bit frames at 16 bits for measurement and narrow to 8 bits only for display, or the dynamic range is gone for good. If the driver's output surprises you — Bayer, planar, packed 10-bit — settle it before picking the cvtColor code: verify the pixel format you received first, and remember image bytes are not pixel data whenever a buffer is larger than width × height × channels.
How to concatenate CUDA images horizontally with a single copy
Compose the join on the device and copy it once: a preallocated 3840 × 1080 RGBA buffer holds 16,588,800 bytes, while the copy-then-hstack route pushes those bytes across, reads and writes them again in host RAM, and allocates 16,588,800 bytes afresh at 30 fps — about 498 MB/s of allocation churn.
The instinct — copy each frame to the host and call np.hstack — works and wastes: np.hstack always builds a fresh 1,080 × 3,840 × 4 array per frame, so you pay a second full-frame pass in host RAM on top of the copies. Preallocating the join on the device removes both.
- Allocate the combined CUDA buffer once, outside the loop: 3,840 × 1,080 × 4 B = 16,588,800 B. Both inputs must share height and channel count.
- Fill it on the device each frame: left frame into columns 0 to 1919, right frame into columns 1920 to 3839 — a device-side 2D copy or your library's composite function. Nothing crosses to the host yet.
- Map the combined buffer once at start-up and read the (1080, 3840, 4) view each frame.
- Convert and process as one image: cvtColor, then your OpenCV steps.
- If an acquisition callback can overwrite the buffer mid-read, take the one copy and process the snapshot — the guard described in torn frames in a camera callback.
What does the conversion cost at 30 fps?
At 30 fps the entire budget is 33.3 ms per frame, and the mapped conversion spends almost none of it: the 8,294,400-byte transfer takes about 2.8 ms at an effective 3 GB/s and 6.9 ms at 1.2 GB/s, leaving 26 ms or more for your processing.
Resolution scales the stream faster than frame count does. A 4000 × 3000 RGBA frame is 4000 × 3000 × 4 B = 48,000,000 B = 48.0 MB, so one 12 MP sensor at 30 fps is a 1,440,000,000 B/s = 1.44 GB/s stream, enough to saturate a shared host link on its own. Mono8 cuts that to 12,000,000 B = 12.0 MB per frame and 360 MB/s at 30 fps — usually the difference between a workable pipeline and a stalled one. Two cameras double both figures, which is why the device-side join above keeps a two-camera 1080p rig at one 16.6 MB crossing per frame instead of a crossing plus a 33.2 MB host pass.
Which cameras and connections keep this loop fed?
Size the link from the stream rate, not the sensor: one GigE port carries about 118 MB/s practical against a 125 MB/s line rate, so a 1080p RGBA stream at 249 MB/s already needs USB3 or 10GigE, while mono8 at 62 MB/s fits GigE with room to spare.
The area scan cameras catalog spans GigE, USB3, 10GigE and CoaXPress, and the camera finder filters by interface, resolution and shutter. The LS-CMR-0166 delivers 4000 × 3000 at 30 fps over USB3; the LS-CMR-0053 reads 12 MP at 9 fps over GigE, trading frame rate for a single-cable, single-port link.
- Sensor format
- 1/1.7"
- Interface
- USB3
- Resolution
- 12 MP
- Shutter
- Rolling
Two cameras on one host pair naturally with a dual-port PoE NIC such as the MV-GE2002P, one camera per port. The two GigE cameras on one host article covers port assignment and the bandwidth budget; multiple USB cameras on one host does the same for USB3, where per-port power shares a ceiling with bandwidth. If acquisition timing ever outgrows the NIC, frame grabbers add hardware-timed acquisition for CoaXPress and Camera Link.
- Interface
- GigE
- Ports
- 2 ports
Cabling is the quiet failure point. On a moving gantry you want a high-flex GigE cable built for drag-chain use with screw locks — the LS-CBL-0199 is exactly that. Static benches can pick from the camera data cables range, which runs from about 1 m to over 20 m. A cable that survives flexing but is pinched at a guide pulley fails weeks later with dropouts that look exactly like software bugs.
When should the frame stay on the GPU and skip NumPy?
When every step after capture is itself a GPU operation — a detection network, CUDA filters, a GPU display — leave the frame on the device: NumPy and cv2 live in host memory, so every conversion is 8.3 MB of travel per 1080p frame that buys nothing.
The hybrid that works in production keeps full-resolution frames on the device and gives the host a reduced copy. A 480 × 270 mono preview is 480 × 270 × 1 B = 129,600 B, 64× less than the 8,294,400 B RGBA frame, so thresholds, contours and barcode pre-processing on the host cost almost nothing while a network still sees every pixel.
Feed networks through the device pointer rather than a host array — feeding GPU tensors to a vision model shows that wiring — and drive live displays through CUDA and OpenGL interop instead of a host preview window.
The decision in one line: map once, reshape once, one cvtColor, join on the GPU before any copy, and keep NumPy out of the hot path when every stage is already a kernel. The systems topic page collects the surrounding buffer-and-bandwidth pieces, and the contact page reaches the applications team when you want a frame budget checked before ordering hardware.
The configurator matches a camera, lens, light, light controller and cables to each other, step by step, and lists only parts that fit what you already picked.
Questions
Why does the image come out sheared or striped after the reshape?
Each row in the buffer is padded to an alignment boundary, often 4, 16 or 64 bytes, so the block holds more bytes per row than width × channels and a naive reshape smears the image into diagonal stripes. View the buffer as (height, pitch_in_bytes), slice the first width × channels bytes of every row, then reshape to (height, width, channels). If the pitch already equals width × channels, look at torn frames or a wrong plane count instead.
Why does my colour frame come out covered in a green-magenta grid?
That grid is raw Bayer data: the sensor's colour filter pattern arrived as a single-channel (height, width) array that was displayed as mono. Drivers usually debayer for you; if yours does not, one cvtColor call with the code matching the sensor's pattern order restores colour, and that order is on the sensor's data sheet. Guessing the pattern swaps red and blue, so read it rather than trying all four codes.
How much host memory do capture buffers and mapped views need?
Multiply frame bytes by queue depth. A 4000 × 3000 RGBA8 frame is 48.0 MB, so a ring of ten buffers is about 480 MB, and a second processing copy doubles that. On a module with 4 to 8 GB of memory shared between CPU and GPU, five to eight buffers is a realistic ceiling — count what your acquisition library allocates by default before you raise the queue depth.
How do I measure my real device-to-host bandwidth?
Wrap 100 conversions at your real frame size in a monotonic timer, divide total bytes by elapsed seconds, and ignore the first few iterations as warm-up. Desktop-class PCIe hosts commonly land between 3 and 6 GB/s effective; small embedded modules often sit between 1 and 2 GB/s. Budget with the measured number, not the data-sheet peak; the peak is a ceiling under ideal conditions.