GigE stream packet resend ratio: fix buffer timeouts

You rebuilt your capture program against a newer Aravis release, and a GigE Vision stream that ran for months now hands back late or incomplete buffers. The camera answers pings, AcquisitionStart succeeds, a few frames appear, and then arv_stream_timeout_pop_buffer returns buffers whose status is not ARV_BUFFER_STATUS_SUCCESS until your reconnect logic gives up. Nothing about the optics, the exposure or the wiring changed, and that is the clue. From Aravis 0.5.10 onward the GVSP stream carries a packet resend policy whose default is conservative, and code that never set it inherits the small window. This guide covers how the resend mechanism works, the one-line fix, a bandwidth check worked through for 1600 × 1200 at 50 fps, and what to test when the ratio alone is not enough.

The symptom points at the receiver, not the camera

Timeouts that begin with a library version change, on the same direct PC-to-camera link, are a receiver-side regression until the opposite is shown. The 0.5.10 release added a packet-request-ratio parameter to the GVSP stream object. Its default is compiled into the library as ARV_GV_STREAM_PACKET_REQUEST_RATIO_DEFAULT in arvgvstream.c and sits near 0.1, meaning only about a tenth of a frame's packets can ever be asked for again. Programs written before the parameter existed never set it, so they inherit the small window on the first run.

Separate two failure pictures early. Frames that arrive complete but with torn or horizontally displaced rows mean packets were lost and recovery could not keep up; that path is covered in GigE Vision packet loss: horizontal lines in live images. Buffers that never complete at all, so your pop times out, mean the receiver stopped asking for the missing packets. That is the resend window, and it is the case here.

How GVSP resend recovers lost packets

GVSP runs over UDP, which never retransmits, so GigE Vision adds its own recovery. Every packet carries a block ID, saying which frame it belongs to, and a sequential packet ID giving its position in that frame. The receiver tracks the IDs as packets land; when a missing packet has not arrived within its wait time, the receiver sends a resend request naming that packet, and the camera transmits it again. The standard defines the mechanism but leaves the implementation and the maximum response time open, so the receiving library sets the actual limits.

Retransmission is not free. Every request travels camera-ward and every repeated packet adds load to a link that is already busy. Libraries therefore cap recovery with a ratio: the fraction of a frame's packets that may be requested again. Aravis exposes that cap as the packet-request-ratio property on the stream object. With the default near 0.1, one burst that eats more than a tenth of a frame's packets leaves the remainder unrecoverable, the frame never completes, and the buffer comes back late or failed.

The 36 bytes are the IP, UDP and GVSP headers carried inside every packet, so a 1500-byte packet moves 1464 bytes of image data.

The fix: one property before acquisition starts

Create the stream, set the property, then start acquisition. The order is arv_device_create_stream, the property set, your arv_stream_push_buffer calls, then AcquisitionStart through arv_device_execute_command.

ArvStream *stream = NULL;           /* returned by arv_device_create_stream() */
double packet_request_ratio = 0.5;  /* half a frame's packets may be re-requested */

g_object_set (stream, "packet-request-ratio", packet_request_ratio, NULL);

The longer GLib route works too: declare a GValue, g_value_init it to G_TYPE_DOUBLE, g_value_set_double it to 0.5, then g_object_set_property under the name "packet-request-ratio". g_object_set does the same job in one call and is the form to keep.

Prove the fix by counting, not by watching the preview window:

guint payload = 0;
ArvBuffer *buffer = NULL;
size_t buffer_size = 0;
int delivered = 0;
int failed = 0;

payload = arv_device_get_integer_feature_value (camera, "PayloadSize");

for (int i = 0; i < 1000; i++) {
        buffer = arv_stream_timeout_pop_buffer (stream, 100000);
        if (buffer != NULL) {
                arv_buffer_get_data (buffer, &buffer_size);
                if (arv_buffer_get_status (buffer) == ARV_BUFFER_STATUS_SUCCESS &&
                    buffer_size == payload)
                        delivered++;
                else
                        failed++;
                arv_stream_push_buffer (stream, buffer);
        } else {
                failed++;
        }
}

Before the change this loop collects failures within seconds. After it, at 1600 × 1200 and 50 fps, the count is 1000 delivered and 0 failed. Anything short of that means loss is still outpacing the window, so work the causes table below.

Worked example: 1600 × 1200 at 50 fps on a 1 Gb link

Inputs, each with its source:

• 1600 × 1200 pixels: the camera's full-resolution mode, mode 0 of its ResolutionSelReg list.
• 50 frames per second: the camera's rated stream in that mode.
• PixelFormat set to ARV_PIXEL_FORMAT_MONO_8, so one byte per pixel.
• 1500-byte packets on a direct 1 Gb link: the default setup.

The arithmetic:

• Frame size: 1600 × 1200 = 1,920,000 px × 1 B = 1,920,000 B, about 1.92 MB.
• Payload rate: 1,920,000 B × 50 fps = 96,000,000 B/s, × 8 = 768 Mbit/s.
• A 1 Gb Ethernet link carries roughly 950 Mbit/s of UDP payload at 1500-byte packets, so the load is 768 / 950 ≈ 81%.
• Packets per frame: 1500 − 36 = 1464 B of image data each, so 1,920,000 / 1464 ≈ 1312 packets, plus a leader and a trailer, about 1314 per frame and ≈ 65,700 packets per second at 50 fps.
• Resend budget: 0.1 × 1312 ≈ 131 packets at the default; 0.5 × 1312 = 656 packets after the fix.

At 81% link load, a few milliseconds of host stall bursts enough loss to blow a 131-packet budget, and every affected frame is abandoned. A 656-packet budget absorbs those bursts and the stream runs clean, which is exactly what raising the value to 0.5 did on this setup. The rule to carry forward: any stream above roughly 70% of the usable link rate should set packet-request-ratio to 0.5 at startup. Below that the default survives, and the larger window costs nothing when no packets are missing.

Causes in order of probability

Work down this table when the ratio is already 0.5 and failures remain, or when the symptom predates any library change.

Checks before you change hardware

Count first. The 1000-frame loop above plus the stream's completed, failed and underrun counters tell you whether frames are being lost, arriving damaged, or never finishing at all. Each pattern points at a different row of the table.

One capture settles the argument. Wireshark's GVSP dissector exposes the resend flag as gvsp.flag.packetresend, so you can count requests against gaps. Requests appearing right after gaps, with frames completing, means recovery is working. Frames timing out with almost no requests confirms the window is the problem; a flood of requests with frames still late means the link or the host is. The capture-first method is laid out in diagnosing blank frames from GigE Vision packet loss.

Redo the payload arithmetic for your real pixel format before blaming hardware. Bayer8 keeps this stream at 768 Mbit/s. A packed YUV 4:2:2 format doubles the payload to 1536 Mbit/s and RGB8 triples it to 2304 Mbit/s, and neither fits a 1 Gb link; no resend setting changes that arithmetic.

On the physical side, keep the run simple: a screw-locked, shielded GigE camera cable, drag-chain rated when the head moves:

Confirm 1 Gb negotiation at both ends, and treat any run through an unmanaged switch as a suspect until the capture says otherwise. If the decision is to change the camera rather than the settings, start from a color area scan camera with a global shutter on GigE:

Run whichever part you choose through the bandwidth arithmetic above before committing to it.

When the ratio alone is not enough

If 0.5 still leaves failures, shrink the loss itself. Larger packets are the first lever: at a 9000-byte MTU each packet carries 8964 B of image data, so the frame needs 1,920,000 / 8964 ≈ 215 packets instead of 1312, an 84% cut in packet count, header overhead and interrupts. The MTU must match on the camera, every switch and the host NIC; the failure pattern when it does not is covered in GigE Vision stream timeout: packets, buffers and MTU.

Second lever: the host. If the kernel is dropping UDP packets for lack of receive buffer space, resends only double the traffic; raise the buffer as row 2 of the table describes. Third lever: the camera side of the handshake. The standard sets no deadline for answering a resend request, so a camera may answer late. If your capture shows requests but retransmissions trail behind, drop the frame rate and watch whether failures scale with it; if they do, the link budget, not the library, is the limit.

Pre-flight checklist

• Set packet-request-ratio to 0.5 right after arv_device_create_stream, before the first buffer push and before AcquisitionStart.
• Keep the pop timeout at 100000 µs while proving the fix, so a genuine failure surfaces quickly.
• Run the 1000-frame count: delivered must reach 1000 with 0 failed before you call it fixed.
• Take one capture and confirm gvsp.flag.packetresend appears when loss appears.
• Keep payload at or below about 80% of usable link rate, and count RGB8 at three bytes per pixel.
• Check packet size against the smallest MTU on the whole path, camera to host.
• Use a screw-locked cable and confirm 1 Gb negotiation at both ends.

The fix itself is one line; the discipline is the counting. When the stream has outgrown its link and you are re-speccing hardware, GigE area scan cameras and the camera finder list current GigE, 10GigE and CoaXPress models, and camera cables covers locked and drag-chain variants. Start the configurator at the camera step to pair a camera with a lens and light, or reach us through the contact page with your frame size and rate and we will run the arithmetic with you.