GigE camera lost after a resource error: rediscovery and recovery

A camera that streams all morning and then dies mid-run with error -14 is not failing at random, and it is not broken. The code means resources not available: the host ran out of frame buffers or memory. The camera then looks gone from rediscovery because your process still holds stale state, or because discovery broadcasts never reach the container, not because the camera stopped answering. Here you get the recovery sequence that brings the camera back without a power cycle, the container setting that fixes discovery, and the queue arithmetic that stops the error from returning. It is written for engineers running their own grabber loop under Linux on a GigE area scan camera.

What error -14 actually tells you

In the SDK behind this failure, code -14 maps to resources not available, with memory as the stated example. Two mechanisms produce it in a grabber loop.

The first is leaked buffers. Acquisition works as a queue: you announce a fixed set of frame buffers, the camera fills them, and every frame you handle must go back into the queue. A path that skips the hand-back, an early return, an exception between grab and re-queue, a branch that keeps the frame for later, leaks one buffer per frame. At 30 frames/s, a ten-frame pool is gone in 0.33 s and the next allocation fails. The per-call allocation mode in the failing loop (AllocAndAnnounceFrame) allocates and announces a fresh buffer for every request, which makes the leak faster still when frames are never returned.

The second is receive buffers that are too small. When packets arrive faster than the kernel socket or the NIC drains them, the driver drops frames and the SDK burns its working set on retries and re-announcements. The symptom is the same -14, minutes into a run, usually on a host that shares its interface with other traffic.

The camera keeps streaming through both failures. Treat -14 as a host-side diagnosis, not a camera fault.

Why rediscovery fails afterward

Calling the SDK's get_camera_by_id on the instance that threw the error returns nothing, because that instance's handle table and discovery list are stale. Rank the causes this way:

1. Stale instance state in the running process. The acquisition thread died holding the camera handle; the context manager never exited; discovery inside that process sees an empty list.
2. Container network isolation. Discovery is a UDP broadcast on the local subnet, and a container on a bridge network sits on a different subnet, so the broadcast never reaches the camera and the reply never comes back. A laptop on the same switch still lists the camera: the good-network, blind-container pattern.
3. Firewall on the host interface. Discovery rides UDP; a rule that filters the discovery reply after a crash leaves the camera invisible even though streaming once worked.
4. A zombie process holding the control lease. The camera belongs to a session whose owner never exited cleanly. Discovery may still list it, but opening fails or the camera ignores the new session until the lease times out.

A laptop that sees the camera while the workstation cannot proves the switch, cable and camera are good. The fault sits inside the host or the container, so fix that layer first.

The recovery sequence

Work top to bottom and stop as soon as discovery returns the camera:

1. End the grabber completely; do not restart it. Stop the service or container, then list running processes on the host and kill anything that still references the camera. A restart that leaves the old process alive just moves the stale lease around.
2. Let the control lease expire. The camera keeps its session until the heartbeat times out, usually a few seconds after its owner exits. Wait about ten seconds before you try again.
3. Rediscover from a fresh instance in a fresh process. Create a new SDK instance, turn broadcast discovery on where your SDK requires it (set_network_discovery(True) in the Python API this error comes from), and run discovery alone. Never run a get_camera_by_id lookup against the instance that failed.
4. If the list is still empty, move the container onto the host's network stack. Re-run discovery. If the camera appears, bridge isolation was the cause; keep that setting for production, because streaming depends on the same path as discovery.
5. Open and check. Open the camera in the new instance, stream for a minute at production settings, and read the dropped-frame or resend statistics before releasing the line.

Power-cycle only when all five steps fail. At that point the suspect list moves to hardware: cable, NIC, switch port, covered further below. If your failure was a plain shutdown with no resource error, the broader drill lives in Release a camera after a crash on Linux: GigE restart fix; the -14 path here is narrower because buffers fail first and discovery second. Python exception paths that leave handles dangling get their own treatment in Camera handle cleanup after Python script errors.

Running the grabber in a container without losing discovery

Three settings decide whether a containerized grabber survives a crash:

• Network stack. Put the container on the host's network stack for camera traffic. A bridge network hides discovery broadcasts and puts the container on a subnet the camera cannot answer.
• Restart policy. Let the container exit when the grabber crashes instead of restarting in place. A full restart drops every stale buffer and handle at once, which is why a camera reconnect looks like the only cure when the real cure is a clean process restart.
• Firewall rules. Discovery and the stream both ride UDP. Confirm the rules pass both on the camera's interface, not just one.

Test the way you test the host: run the SDK's list tool inside the container, then from a clean shell on the host. If the host sees the camera and the container does not, the container's network is the problem, and the general drill in Linux GigE Vision camera discovery troubleshooting picks up from there.

Worked example: a 2.5 MP mono camera at 30 fps on one gigabit link

Take the MV-CU020-90GC as the camera: a 2.5 MP, 1/1.7-inch, global-shutter, C-mount GigE model. Assume 8-bit mono output, 30 frames/s, a 1,000 Mbit/s link carrying nothing else, and ten frames queued.

Frame size: 2,500,000 pixels x 1 byte = 2,500,000 bytes, about 2.5 MB per frame.

Data rate: 2.5 MB x 30 frames/s = 75 MB/s, and 75 MB/s x 8 = 600 Mbit/s. That is 60% of the link, so a dedicated gigabit link fits with 400 Mbit/s of headroom for resends. The same stream cannot pass a 100 Mbit/s link: 600 / 100 = 6, six times the capacity, so a switch port that renegotiates down guarantees loss.

Link ceiling: 1,000 Mbit/s / 20 Mbit per frame = 50 frames/s theoretical. Running 30 frames/s leaves margin for the resends that packet loss forces.

Packet count: a standard 1,500-byte packet carries roughly 1,470 bytes of image data, so 2,500,000 / 1,470 gives about 1,701 packets per frame, roughly 51,000 packets/s. With 9,000-byte jumbo packets the payload is roughly 8,970 bytes: 2,500,000 / 8,970 gives about 279 packets per frame, roughly 8,400 packets/s, about six times fewer interrupts, retries and drop opportunities.

Queue depth: the frame period is 1 / 30 s = 33.3 ms. Ten queued frames absorb 10 x 33.3 ms = 333 ms of processing stall. If one branch skips the re-queue, the pool leaks one buffer per frame and runs dry in 0.33 s, which is the moment -14 appears.

Decisions: keep this camera on its own gigabit port; enable 9,000-byte jumbo packets only when camera, NIC and switch all support them; size the queue from the worst measured stall, rounded up, plus two frames of margin. A 200 ms stall needs ceil(200 / 33.3) = 7 frames plus 2, so queue nine.

Prevention: keep the queue fed and the link clean

Re-queue inside the same try block. Put the hand-back where an exception cannot skip it, so the queue never drains whatever the processing does.

Fix the buffer count at startup. Announce the number of buffers you plan to keep in flight and allocate nothing per frame afterwards. The buffering mechanics behind bursts are in Camera burst capture faster than link bandwidth: use the buffer.

Raise the host's receive buffers. Bigger kernel socket buffers and deeper NIC receive descriptors absorb burst jitter before it becomes loss; the sizing walk-through is in GigE camera buffer timeouts on embedded Linux: socket buffer tuning.

Give the stream its own interface. If the camera shares an onboard port with other traffic, a dedicated dual-port PoE NIC such as the MV-GE2002P gives the stream ports of its own and its own interrupt budget.

Use cables built for the run. A marginal cable shows up as resends long before the link drops. Fixed runs want screw-locked industrial cable; where the harness flexes, a drag-chain-rated GigE cable such as the LS-CBL-0199 survives the cycles that crack ordinary patch leads.

Watch the loss counters. Read the SDK's dropped-frame or resend statistics every shift; a rising trend precedes the resource error by hours. For the same failure seen from the driver side, the troubleshooting GigE camera buffer overflow errors knowledgebase page goes deeper.

Queue handling in a callback has one more trap: copy the buffer before handing it back and you avoid torn frames. Torn frames in a camera callback: copy the buffer before you queue it shows why.

When the fixes point at hardware instead

Recovery that ends with the camera invisible everywhere, laptop included, moves the fault down the chain:

• Negotiated speed. A port that fell back to 100 Mbit/s cannot carry a 2.5 MP stream; that failure and its fix are in GigE camera packet size error when your link drops below gigabit.
• Power. A PoE port whose budget is shared can reboot the camera mid-stream; confirm the budget or feed the camera separately.
• A second owner. Discovery lists the camera but opening fails: another program holds the control connection, and Frame grabber in use: find the owner and release access shows how to find it.
• Transport layer. Discovery empty on a host where ping reaches the camera points at the SDK's transport layer rather than the network; Ping works, zero cameras: GigE transport layer not loaded covers that signature.
• Swap test. Different switch port, different cable, different NIC. If the fault follows the port, replace the port; if it follows the cable, replace the cable.

Sustained -14 errors after a clean restart, a fresh instance and zero packet loss are rare; when they happen, the cause is nearly always physical.

Checks before you redeploy

Run this list after any -14 event and before the line restarts:

• Discovery from a clean process returns the camera.
• The grabber container runs on the host's network stack, or camera and container share one subnet.
• The re-queue sits in the same try block as the grab, reachable from every branch.
• Queue depth covers the worst measured stall plus two frames, nine frames for a 200 ms stall at 30 fps.
• The dropped-frame counter reads zero after a ten-minute soak at production rate.
• Jumbo packets, if enabled, match across camera, NIC and switch; otherwise every device stays at 1,500 bytes.
• The camera has its own NIC or at least its own switch port.

Each line exists because skipping it produced a field failure: the blind container, the drained queue, the renegotiated link.

Where to go next

If you are building the acquisition side rather than repairing it, the area scan cameras category and its camera finder filter by interface, resolution and shutter, and the camera data cables and frame grabbers pages cover the chain around them. The camera configurator steps through sensor to cable in one pass. More recovery and integration drills sit on the systems topic page. In-stock items ship within one business day, and every order includes free expert support; if the loss counters still climb after these checks, send the SDK error, the queue size and the statistics through the contact page and we will work the sequence with you.