Concatenate CUDA images in Python without host round trips

Two cameras feed one inspection and you want one image, so the reflex is np.hstack or cv2.hconcat. Both run on the host: every pixel rides PCIe down, gets stitched by the CPU, and rides back up. At 1080p RGB8 that detour moves about 24.9 MB per frame pair and costs roughly 6.3 ms of raw transfer before the stitching itself. The device-side path is shorter: one preallocated output buffer on the GPU, one pitched device-to-device copy per source, and no host wake-up at all. This guide gives engineers driving a CUDA camera SDK from Python the pitch arithmetic, the code shape, the byte counts to expect, and the controller and camera side that carries the streams.

Can you concatenate CUDA images without leaving the GPU?

Yes: allocate one output buffer on the device whose row pitch equals the sum of the source widths in bytes, then issue one pitched device-to-device copy per image, and no pixel ever crosses the PCIe bus to the host.

Horizontal concatenation is not a special operation on device memory. It is n rectangular writes into one wide rectangle at fixed byte offsets, which is exactly what a pitched 2D copy performs. The CPU path hides that shape behind np.hstack and charges you the bus crossing plus a synchronous stall while the host code runs. The composition logic is identical either way; only the address math moves.

Converting a single frame into a NumPy array for OpenCV is its own problem, covered in the guide on CUDA image to NumPy conversion without round trips. This guide composes several frames and leaves the stitched result resident on the device for whatever runs next.

What does the round trip actually cost at 1080p RGB?

At two 1920 × 1080 RGB8 frames the CPU route moves about 24.9 MB across PCIe — 12.4 MB down plus 12.4 MB back — roughly 6.3 ms on a PCIe 3.0 x4 link, while two device-to-device copies move the same 12.4 MB on the card in about 0.25 ms.

The worked example, one frame pair:

• One source frame: 1920 × 1080 × 3 = 6,220,800 bytes (6.22 MB).
• Stitched output: 3840 × 1080 × 3 = 12,441,600 bytes (12.44 MB).
• CPU route: two downloads plus one upload = 2 × 12,441,600 B = 24,883,200 B ≈ 24.9 MB across PCIe.
• PCIe 3.0 x4 raw rate: 4 lanes × 984.6 MB/s = 3,938 MB/s, so 24.88 MB ÷ 3.94 GB/s ≈ 6.3 ms — raw; protocol overhead and the sync stall push the real figure higher.
• Device route: 12.44 MB of device-to-device copies at an illustrative 50 GB/s of GPU memory bandwidth ≈ 0.25 ms, and the CPU never wakes.

The penalty scales with area and frame rate. A 4000 × 3000 mono8 frame is 12,000,000 bytes, so a stitched pair is 24,000,000 bytes; round-tripping at 30 fps would push 48 MB × 30 = 1.44 GB/s through PCIe, more than a whole USB3 link provides. The device-side copies cost nothing on the wire at any frame rate.

What do you need before you write the code?

You need four numbers per image — width and height in pixels, the SDK's real row pitch in bytes, and bytes per pixel — because the copy geometry is specified in bytes, not pixels, and a padded pitch declared as width × bpp shears every row after the first.

Collect these once at start-up:

• Width and height in pixels for each frame.
• The row pitch each SDK frame reports, in bytes. SDKs commonly pad rows to 256- or 512-byte boundaries, so width × bpp is frequently not the real stride. The illegal-access guide to pitch and planes shows what a misread pitch does to a mapped frame.
• Bytes per pixel from the pixel format: mono8 is 1, RGB8 is 3, and packed 10- or 12-bit formats are their own arithmetic — why 10-bit sensors ship 8-bit images walks through that.
• One common height, or a padding plan if the cameras differ.
• An output buffer allocated once, outside the frame loop, per reuse image buffers instead of per-frame allocation.

How do you lay out the output buffer?

Allocate the output once, outside the frame loop: its row pitch is (w1 + w2) × bpp bytes — 11,520 bytes for two 1,920-pixel RGB8 rows — and its total size is 12,441,600 bytes for two 1920 × 1080 sources.

The output pitch is yours to define, and 11,520 bytes per row is already a multiple of 256, so no extra padding is needed for most consumers. Each source keeps its own pitch, padded or not. The offsets are fixed for a given camera geometry, so compute them once and reuse them every frame.

How do you copy each image into its slot?

Run one 2D strided copy per source — the runtime's cudaMemcpy2D semantics — writing image i at byte offset x_offset_i inside every output row, with the output pitch as the destination pitch and each source's reported pitch as the source pitch.

1. Read each source's device pointer and its reported row pitch from the SDK frame object.
2. Compute the byte offsets once at start-up: x_offset_1 = 0, x_offset_2 = w1 × bpp = 5,760 bytes in the worked example.
3. Issue one pitched copy per source with kind set to device-to-device: destination pointer plus the offset, destination pitch = out_pitch, source pointer and reported source pitch, width w_i × bpp bytes, height h rows.
4. Synchronize once after the last copy, or let the consumer's stream ordering handle it when everything shares one stream.
5. Hand the output to the next stage as device memory — an inference input as in feeding GPU tensors to a vision model without a CPU round trip, or a display path that wraps the buffer without copying, as in wrapping an SDK frame buffer in an OpenCV Mat.

# Illustrative layout: two RGB8 frames, equal height.
w1, w2 = 1920, 1920          # source widths, px (int)
h      = 1080                # common height, px (int)
bpp    = 3                   # RGB8, bytes per pixel (int)

out_pitch = (w1 + w2) * bpp  # 11520 bytes per output row (int)
x1        = 0                # image 1 starts at byte 0 of each row
x2        = w1 * bpp         # image 2 starts at byte 5760 of each row

# out_buf: device buffer of h * out_pitch bytes, allocated once and reused.
# One pitched copy per source, cudaMemcpy2D semantics, kind = device-to-device:
#   pitched_copy_2d(dst_ptr + x_offset, dst_pitch, src_ptr, src_pitch,
#                   width_bytes, height)
pitched_copy_2d(out_buf + x1, out_pitch, img1.ptr, img1.pitch, w1 * bpp, h)
pitched_copy_2d(out_buf + x2, out_pitch, img2.ptr, img2.pitch, w2 * bpp, h)

How do you prove the result is right?

Fill each source with a different constant — 100 and 200 — run one concatenation, synchronize, and read two bytes through a zero-copy mapped view: the byte at column 0 must read 100 and the byte at column w1 × bpp must read 200, which proves both offsets without trusting any timing.

# Proof test: constants in, byte reads out.
img1 = constant_image(w1, h, bpp, 100)  # illustrative helper: every byte = 100
img2 = constant_image(w2, h, bpp, 200)  # every byte = 200
concat_horiz(img1, img2, out_buf, out_pitch)
device_synchronize()

view = mapped_view(out_buf, h, out_pitch)   # shares device memory, no copy
assert view[0, 0] == 100              # first byte of image 1, row 0
assert view[0, w1 * bpp] == 200       # first byte of image 2, row 0
assert view[h - 1, w1 * bpp - 1] == 100  # last byte before the seam
assert view[h - 1, w1 * bpp] == 200     # first byte after the seam

The last-row reads catch an off-by-one in the offset or the pitch, and they cost four bytes of traffic because the mapped view shares the device memory with the host rather than copying it. For the before/after figure, wrap each route in device events and average over 100 frame pairs; a one-shot number is dominated by first-call overhead and tells you nothing.

When should you still concatenate on the CPU?

Concatenate on the host when the consumer runs there anyway, when the sources have different heights you cannot pad, or when frames are small enough that launch overhead dominates — a 640 × 480 mono8 frame is 307,200 bytes, and at that size two kernel launches rival the copies themselves.

A saved image, a serial reply or a CPU-only classifier ends up on the host regardless, so the round trip is not waste but the shortest path. Different heights that you decline to pad are the same story: crop both sources to the shorter height first, which changes the content, not the route. And for a quick debug look, np.hstack over the mapped views is fine — it is slow precisely because it reads every byte across PCIe, which is the cost you measured above.

Which hardware keeps two camera streams on one GPU?

You need one controller that hosts the SDK and its GPU with a camera port per stream — two USB 3 or GigE ports minimum for two cameras — plus PCIe expansion if your software runs its CUDA stages on an added accelerator card.

Everything above runs on whatever device your SDK already uses, so the buying decision is port count, memory and expansion, in that order.

The MV-VC3301H-128G66 carries 5+ LAN ports, 3–4 USB 3 ports, four built-in light channels, 8 GB of memory and an Intel Core i3, with PCIe expansion supported for the accelerator card your software needs.

Larger pipelines share the box with inference: the MV-VC5525-256G70-NN pairs PoE camera ports and 5+ LAN ports with eight built-in light channels, 32 GB of memory and an Intel Core i5.

Where PoE and PCIe expansion matter on a compact board, the MV-VC4712-256G10-2TH offers PoE camera ports, 3–4 LAN ports, 2 USB 3 ports and six built-in light channels.

On the camera side, resolution sets the stream budget before any of this runs. The LS-CMR-0166 delivers 4000 × 3000 at 30 fps over USB3, and two such streams are exactly the case where host buffering and port budget decide whether frames reach the GPU cleanly — see two USB3 cameras at full resolution on one host.

Shortlist against the area scan camera finder and the vision controllers category, and start the camera side at the camera configurator. If you want a second pair of eyes on the byte budget, send the frame sizes and rates through the contact page. The rest of the pipeline guides live on the systems topic page.