GigE camera stuck in read-only mode: recover without replugging

You stop a grab loop that has wedged, and the next open of the camera fails: your toolkit reports the device is reachable but offers read-only access only, and the request for full access is refused. The reflex — pull the cable or power cycle the camera — costs a walk to the cell and a minute of stopped production, and it is almost never needed. The camera is not damaged; it is still honoring the control lease of the process that died, because a hard kill skipped your teardown code. This guide is for integrators writing their own acquisition code. It gives the mechanism in one paragraph, a five-step software recovery that runs in seconds, one worked example that sets your automatic re-open delay, and the teardown habits that keep the lock from coming back.

What the read-only message actually tells you

The error describes access modes, not hardware damage. A GigE Vision device grants two access levels: exclusive control, where one client holds full read-write access, and read-only, where a client may look but not change. Your message says discovery answered, so the camera is powered and reachable, but a control session is already registered on it. Right after a hard kill that session is a ghost: the owning process is gone, yet the camera keeps the lease until the heartbeat expires or the lease is released. Read-only opens still succeed inside that window, which is exactly the door the recovery uses.

Diagnose in this order, because the fixes differ:

One shortcut: if a read-only open fails too, the lease is not a ghost. A live client holding exclusive control blocks every other open, read-only included — that is the two-clients case covered in GigE camera open fails in a second tool: exclusive control.

Why a hard kill leaves the camera locked

The GigE Vision control protocol works on a heartbeat. The client that holds exclusive control must send keep-alive messages at a steady rate; the camera keeps the lease as long as they arrive and frees it once they stop for the timeout period. The documented default timeout is 3 s, and most cameras ship with it. A clean close releases the lease inside your teardown. A hard kill — the task manager's end process, a signal the runtime cannot catch, a power blink on the PC — runs no teardown, so the camera waits out the full timeout before it accepts a new controller.

The same heartbeat explains the cousin symptom: pause a debugger in a live session and the expiry frees the lease mid-run, stopping your stream. That case is covered in why the GigE Vision heartbeat timeout stops your stream after a pause; there the expiry is the accident, here it is the cure.

One value to watch while you diagnose. Debugging guides tell you to raise the heartbeat timeout so breakpoints do not kill the session; raise it to 300 s on the bench and forget to reset it, and every production crash becomes a 300 s lockout — five minutes of refused opens instead of three seconds.

While you wait, the read-only session keeps register read access, so a second tool can confirm the camera's state; the technique is the one in access GigE camera registers at runtime from your control code.

The recovery: five steps from your desk

1. Rule out a live holder. Close every application that might hold the camera, and on the production machine check for a second copy of the acquisition process or an orphaned child that survived the crash. If a live client owns the lease, read-only fails too; return to the table above.
2. Open the camera read-only. Pass the access mode explicitly in your toolkit's open call — the error message itself told you read-only is the only grant on offer. Discovery listing the camera is not proof the lock has cleared; only a successful exclusive open is.
3. Send AcquisitionStop in that read-only session. AcquisitionStop is the standard GenICam command that ends image acquisition. It stops the stream the dead client left running and parks the camera in a defined idle state. Some cameras refuse command writes from a read-only session; skip the step then, because the wait in the next step clears the lease on its own.
4. Wait one heartbeat timeout, then close. At the documented default, the wait is one timeout measured from the crash, not from your open. Close the read-only session afterwards so your own session is not the next thing left to clean up.
5. Open exclusive and prove it. Request full access, start acquisition, and run a short grab — 100 frames is a sensible test size — confirming the count completes without errors. If the crash left acquisition parameters changed, reapply them in the same session; apply new camera settings without replug: set, stop or reset walks that sequence.

Worked example: re-open delay after a hard kill on the default heartbeat

Inputs, with their sources:

• Crash at t = 0 s — the client was terminated while inside the camera context, the failure behind the read-only message.
• Heartbeat timeout T = 3 s — the documented default of the GigE Vision control protocol.
• Watchman poll interval p = 1 s — your configuration choice; 1 s is a typical process-watchdog setting.
• Open and stop overhead o = 0.5 s — one read-only open plus the AcquisitionStop round trip, budgeted high for a switched LAN (an estimate).

Rule in words: the camera frees control access one timeout after the last keep-alive it received. A crashed client sends no keep-alives, so the lease frees at most T after the crash; the recovery is a read-only session at any time, the expiry, then an exclusive open on the first watchman tick after expiry.

Arithmetic:

• Lease frees: t_free = 0 s + T = 3 s.
• First watchman tick at or after t_free: ticks land at 1, 2, 3, 4 s; the 3 s tick races the expiry, so plan on 4 s.
• Exclusive open plus the 100-frame verification grab: 4 s + 0.5 s = 4.5 s to producing again.

The camera is producing again within about 4.5 s of the kill, and the automatic re-open belongs at the 5 s mark: one timeout of expiry, 1 s of poll, 1 s of margin. Set the watchman's restart threshold to 5 s. If the exclusive open is still refused at 10 s — more than three times the default expiry — something alive is renewing the lease: stop polling, find the holder or power cycle.

Prove the recovery on the bench

The recovery is a handful of lines, and the call names differ by toolkit: the mode argument, the feature write and the close are your SDK's API. Two names here are not toolkit-specific — AcquisitionStop is the standard GenICam command, and GigE device IDs take the form DEV_ followed by 12 hex digits. The value below is an example; substitute your own.

# Illustrative pseudo-code: open_with_mode, write_feature, close and sleep
# stand in for your toolkit's API. AcquisitionStop is the standard GenICam
# command name. DEV_001122334455 is an example device ID.
def recover(device_id: str = "DEV_001122334455",
            heartbeat_s: float = 3.0) -> bool:
    cam = open_with_mode(device_id, mode="read-only")
    if cam is None:
        return False                # live holder, not a ghost lease
    write_feature(cam, "AcquisitionStop")
    close(cam)                      # end the read-only session
    sleep(heartbeat_s)              # lease expiry: one timeout
    return open_with_mode(device_id, mode="exclusive") is not None

Test that proves the result: start the grab loop, kill the owning process hard at t = 0 — a signal the runtime cannot catch, or the task manager's end process — then call recover() twice. The first call, made immediately, must return False because the heartbeat has not expired. The second, made at t = 5 s, must return True, and the 100-frame grab afterwards must complete without drops. False at 1 s and True at 5 s is the heartbeat diagnosis demonstrated in one bench run; if even the first call returns True, the lease was never your problem.

Prevention: teardown that survives a kill

• Wrap every open. Put AcquisitionStop and the close in a finally block or your language's context manager, so an exception inside the grab loop releases the lease instead of leaving it behind.
• Handle the polite signals. Catch the interrupt and terminate signals so Ctrl-C and a service stop run the same finally path. A hard kill cannot be caught in code — no handler runs — which is why the next item matters.
• Give the watchman a two-stage stop. Ask the acquisition process to stop gracefully, allow a grace period (10 s is a workable budget), hard-kill only past it, then wait the 5 s re-open threshold from the worked example before restarting.
• Keep one owner per camera. Exactly one process takes exclusive control; every other reader opens read-only. Two exclusive clients fighting over one camera is a design fault, not a camera fault.
• Reset the debug heartbeat. The raised timeout that carried you through breakpoints goes back to the documented default before deployment, or every crash inherits the debug lockout.

When the standard answer is not the answer

• The message says in use rather than read-only, or the lock appears without any crash of yours. Same lease mechanism, different entry point; Camera stuck in use after a crash: recover without a reboot covers that variant.
• Discovery lists nothing at all. No lock is involved. A camera that answers no discovery has a network or power problem; work the addressing fix in connect a GigE camera on a different subnet: set remote and local IPs.
• The refusal outlasts 10 s after every crash. Something renews the lease: a runtime service watching the camera, a second integration on the same host, or a debug timeout that never got reset. Find the renewer before you touch hardware.
• The camera refuses even after a power cycle. Stop treating it as protocol state; that is a service case for the camera itself.

Where the recovery runs

Nothing above needs new hardware; it needs a machine that stays up, owns the camera's GigE port and, in a multi-camera cell, also runs the watchman. That is the job of the vision controllers class rather than an office desktop: several Ethernet ports for the camera subnet, industrial I/O, and built-in light channels so the illumination wires to the same box. The MV-VC2140-128G40-1T lists 3-4 LAN ports, 2 USB 3 ports and 4 built-in light channels; the MV-VC3301H-128G66 lists 5+ LAN ports, 3-4 USB 3 ports and 4 built-in light channels. Both ship with 8 GB of memory:

The 60-second checklist

• Watchman re-open threshold set to 5 s (one heartbeat expiry + 1 s poll + 1 s margin)
• Debug heartbeat timeout reset to the documented default before deployment
• AcquisitionStop and close in a finally path for every camera open
• One exclusive owner per camera; everyone else opens read-only
• Recovery proven with a hard kill: False at t = 1 s, True at t = 5 s, 100-frame grab complete
• Power cycle reserved for refusals past 10 s

Run the bench test this week — one hard kill, two calls to recover(), one 100-frame grab — and a crashed grab loop becomes a 5 s delay instead of an outage. If the cell needs hardware as well, the GigE camera finder and the area scan cameras behind it are the starting chain; for a configured proposal, start at the quote page or write to us through the contact page.