RDMA over RoCE v2 for GigE Vision cameras: when it pays off

Add a fourth 10GigE camera to a host and the next bottleneck is usually not the network — it is the CPU. Every UDP packet arrives at software: a filter driver reads headers, validates sequence numbers, requests resends, and copies payload bytes into application buffers. When four streams do that at once, the resend machinery competes with your inspection code for cores. RDMA over Converged Ethernet (RoCE v2), now an optional transport in GigE Vision 3.0, moves the whole job into the NIC: pixels land directly in pinned host memory and the CPU stays out of the data path. This guide explains what changes in the pipe, which hardware the RDMA path requires, and — with one worked example — how to decide whether a multi-camera build gains enough to justify it. If you run a single 1GigE camera, the short answer is no; the numbers below show where the line sits.

What RDMA changes in the data path

On the classic UDP transport, the camera sends image packets and the host does everything else. A filter driver reads each GVSP header, tracks sequence numbers, asks the camera to resend missing packets, and copies payload bytes into your application's buffers. Every step runs on a CPU core, so transport load grows with bandwidth and packet count — not with how demanding your inspection is.

RDMA (remote direct memory access) over RoCE v2 hands that work to silicon. Image packets travel from camera memory straight into a block of host memory that the application registered in advance; registration pins the pages so the OS cannot move or page them out. Each connection is a queue pair — a send queue and a receive queue — managed by the NIC, and a completion queue tells your code when a buffer holds a finished frame. Reassembly, ordering, flow control and retransmission all run in the NIC hardware. Nothing is copied between kernel and user space on the way, which is what zero-copy means in practice.

Three consequences matter for vision: frame arrival stops depending on how busy the cores are, the CPU budget shifts to processing, and per-host camera counts can grow without the transport becoming the ceiling.

Why the UDP path struggles on multi-10GigE hosts

At 1GigE the software resend machinery is a solved problem, and it holds at 2.5GigE and 5GigE too: the rates are low enough for the driver to manage retransmissions without strain. The picture changes when several 10GigE cameras share a host. Per-camera packet volume rises roughly tenfold over GigE, header processing rises with it, and one lost packet now triggers a resend while three other cameras keep streaming at full rate. CPU load climbs, the driver falls behind, more packets drop, and images arrive with missing lines. That cascade is the multi-10GigE failure mode, and its packet-level symptoms are covered in GigE Vision packet loss: horizontal lines in live images, blank frames from stream packet loss and a multi-camera frame-drop troubleshooting guide.

Tuning buys time before any transport change: interrupt moderation, large packet sizes and the high-performance NIC driver each return CPU cycles — see interrupt moderation: balancing latency and CPU load and what the high-performance NIC driver gains. When a tuned UDP stack still saturates cores, the transport itself is the reason to move.

What the RDMA path requires

Four things have to line up, and all four are ordinary catalog items today.

• A camera with the RDMA transport. GigE Vision 3.0 defines how RDMA streams are set up, run and torn down, down to the device registers and packet formats; cameras implementing it expose the GVRSP streaming protocol. UDP remains part of the standard, so an RDMA-capable camera still serves an older host.
• A RoCE v2-capable NIC, called a host channel adapter (HCA) in RDMA terms. RoCE v2 dates to 2014 and has carried data-center traffic since, so the silicon is mature and multi-port 10GigE cards with RDMA support are common.
• A managed switch with flow control. RoCE v2 rides ordinary routable UDP/IP, so no special fabric is needed; the network's job is lossless behavior, configured through the switch's flow-control (DCB) settings. A camera can also connect straight to a NIC port with no switch at all.
• Host software that exposes the transport. Linux provides the RDMA stack as libibverbs; Windows exposes it through the Network Direct interface. Your SDK build decides whether the RDMA transport is selectable for a given camera.

One more resource is easy to underestimate: registered buffers stay pinned in physical RAM for the life of the stream. The worked example below sizes that number.

The frame-grabber range lists 13 cards with a 10GigE interface. Two of the multi-port options:

Worked example: four 5 MP 10GigE cameras on one host

Inputs, with their source:

• Four cameras at 5 MP mono, 8 bits per pixel, 75 fps — a typical mid-range 10GigE inspection cell (illustrative values in the range of current 5 MP 10GigE bodies).
• 10 Gbit/s per camera link — the 10GigE line rate.
• A 2-second image ring per camera — a common burst-tolerance depth; shorten or lengthen it to match your pipeline.

The rule in words: a camera's data rate is pixels per frame × bytes per pixel × frames per second; the host aggregate is the sum over cameras; pinned RAM is aggregate rate × ring time.

Arithmetic:

• Per frame: 5,000,000 pixels × 1 byte = 5 MB.
• Per camera: 5 MB × 75 fps = 375 MB/s = 375 × 8 = 3,000 Mbit/s = 3 Gbit/s. That is 30% of a 10GigE link, leaving burst headroom.
• Host aggregate: 4 × 375 MB/s = 1,500 MB/s = 12 Gbit/s arriving at the NICs. One camera per port keeps every port at 30%; two cameras per port puts 6 Gbit/s on each port, 60% of its line.
• Pinned memory: 1,500 MB/s × 2 s = 3,000 MB = 3 GB of RAM locked for camera buffers, on top of processing memory.

The same math bounds the largest bodies: a 25 MP sensor at 75 fps would need 25,000,000 × 75 = 1,875 MB/s ≈ 15 Gbit/s — more than one 10GigE link — so the frame rate settles near 40 fps, where 25,000,000 × 40 = 1,000 MB/s = 8 Gbit/s is 80% of the line. A current 25 MP 10GigE body of this class:

Decision: once the aggregate into one host passes roughly 10 Gbit/s — this cell sits at 12 — plan the RDMA path. At that rate, per-packet software work and resend handling are what saturate cores first, and a NIC doing reassembly in hardware is what buys the cores back. Below the threshold, stay on UDP: a single 1GigE camera moves at most 125 MB/s of line rate, leaves the host plenty of slack, and the software transport is proven at 1, 2.5 and 5 Gigabit speeds.

When zero-copy pays off, and when UDP stays

If frames arrive faster than your code consumes them, changing transport will not fix it — that gap is pipeline design, covered in acquisition faster than processing: fixing the frame-rate gap. The link-level version of the same budgeting is two GigE cameras on one host: ports and bandwidth, and the host-wide planning case is dividing a multi-camera GigE bandwidth budget across the link first.

Per case, then: keep UDP on the bench and in single-camera cells; tune it before replacing it; move to RDMA when the camera count or frame rates push a host past the threshold and the camera class offers the transport.

Checks before you order

• The camera page lists RDMA or GigE Vision 3.0 support; that is a property of the camera's transport stack, settled at selection, not an accessory added later.
• The NIC page lists RoCE v2 / RDMA, and its port count matches the per-port rates from the example.
• The switch is managed and its flow-control (DCB) settings are reachable; cabling and connectors carry over unchanged.
• The RAM plan covers pinned buffers: aggregate MB/s × ring seconds — 3 GB in the example above.
• Your SDK build and OS expose the RDMA stack (libibverbs on Linux, Network Direct on Windows).
• The UDP path stays configured as a fallback during bring-up, on the same cables.

Where that leaves the build

One line per case: single 1GigE camera — UDP, nothing to change; two or three 10GigE cameras — tune UDP, watch filter-driver CPU, keep RDMA as the planned upgrade; four or more 10GigE cameras or 25GigE-class bodies — build on RDMA from day one, with a RoCE v2 NIC and RAM sized from the ring time.

Shortlist both ends of the link in the area scan cameras and frame grabbers categories, or go straight to the camera configurator and the host-side extras step. The camera finder filters by interface and resolution, the accessories finder does the same for cards, and the systems topic page collects the transport-level reading. If a build sits between the thresholds, send the camera list and frame rates and we will size the link with you.