ROS camera driver shutdown: release cameras for other apps

Three cameras stream happily under ROS, your visualization windows show every frame, and then the stack shuts down — and the cameras vanish from every other tool. Your usual viewer cannot open them, or opens them and shows nothing, and an unplugging session brings them back one by one. The cameras are fine. One of three shutdown faults is at work: a process still holds the device handle, the teardown never ran, or the driver left the camera latched in software trigger mode. This guide is for anyone wrapping GigE or USB3 Vision cameras in a ROS driver node. In ten minutes you can tell the three causes apart, recover each one without touching a cable, and write a shutdown path that hands each camera to the next tool ready for work.

What the lock looks like

Run one split test with your usual GenICam viewer before touching anything else; it tells you which of the three causes you own.

• The viewer refuses to open the camera. A device-busy or access-denied message means the device handle is still claimed. Something is alive, and no register edit fixes this.
• The viewer opens the camera but no frames arrive. The handle is yours and the failure sits in the camera's registers. Open the feature tree and read the trigger settings: this is where you find TriggerMode still reading On.

Both signatures share one root cause: the driver exited without completing its cleanup, and an area scan camera keeps the last state it was left in. That state lives in the camera's registers and in the host's handle table — not in ROS — which is why nothing inside ROS can repair it afterwards.

Cause 1: a live process still holds the device

GenICam access control is exclusive: one application holds control of a device at a time, and every other open attempt fails until that holder lets go. Stopping the stack often leaves survivors — a node that took a hard kill, a helper the driver spawned, a launcher that outlived its children. Three camera nodes mean three handles to release, not one.

The interface decides how long a dead holder still matters:

• GigE Vision: the controlling application must keep a heartbeat alive, and the camera releases control a few seconds after the heartbeat stops — at typical settings, count on 5 s. If the camera is still busy 30 s after every process has died, a live holder remains.
• USB3 Vision: the host driver claims the device for the owning process, and the claim ends only when that handle closes or the process dies. There is no heartbeat and no self-release.

Test: list the processes left from the last session and stop anything driver-shaped. If a second tool still cannot open the camera, the exact failure it reports is the one behind GigE camera open fails in a second tool.

Cause 2: the shutdown hook never ran, or ran out of order

Ctrl+C tells ROS to deliver a clean exit to each node, and a correct driver runs its full teardown then. A crash, a debugger detach or a kill -9 skips every hook, and the cleanup lines that would have printed never do — the cheapest diagnostic is a log line at the top of the shutdown path.

Order matters as much as execution. Closing the device while acquisition is still active, or destroying buffers a callback still references, leaves a half-open state on both ends. The GenICam GenTL standard has required the same discipline since its first release: stop acquisition on the device before you close the stream and the device. The same ordering removes the close-time crashes in camera SDK access violation on close, and the USB-side version of the rule lives in stop acquisition before you close.

Cause 3: software trigger left latched on the camera

Many ROS camera drivers run their default acquisition in software-trigger mode: the node commands each frame itself at a fixed rate, which keeps the pipeline deterministic. That design holds TriggerMode at On for the whole session, and if the node exits without cleanup, the register stays On for as long as the camera stays powered.

The next tool then opens the camera, starts acquisition and receives nothing — the camera is waiting for trigger commands that no one sends. The arithmetic is absolute: 0 trigger commands per second produce 0 frames per second at any exposure setting. If the driver saved its settings into the camera's default user set, even a power cycle will not clear them.

Test and fix: read TriggerMode in your GenICam tool and write it to Off; frames return within one frame period. If the node reads as not writable, another application holds control — the diagnosis in GenICam node not writable applies. The wider family of opened-but-silent cameras is covered in software trigger image timeout.

Recover the camera without unplugging it

Work the list in order; each step rules out one cause.

1. Stop every surviving process. List what is still running from the last session — nodes, driver helpers, launchers — and stop them. One live holder beats every other fix.
2. Give GigE one heartbeat cycle. Wait 5 s and retry the open. Still busy at 30 s means a live process — return to step 1. USB3 does not self-release: the holder must die or close.
3. Open the camera in your GenICam tool. If the open fails, return to step 1. If it opens, read TriggerMode before anything else.
4. Write TriggerMode to Off, set the acquisition mode the viewer expects, and start acquisition.
5. Leave the camera clean. Stop acquisition in the tool and close the device, so the next program inherits an idle, unclaimed camera.
6. Power-cycle only if steps 1–5 fail. Unplugging reloads the default user set and drops any lock — and destroys the evidence the code fix needs.

The teardown order that releases the camera

Build the shutdown path once and every consumer after ROS inherits a working camera. The order is the reverse of set-up:

camera.AcquisitionStop()            # GenICam standard command: end streaming
revoke buffers, unregister events   # your SDK's cancel and unregister calls
camera.TriggerMode = "Off"          # GenICam standard feature: idle state for the next tool
camera.closeDevice()                # illustrative name for your SDK's device close
destroy driver and capture objects  # your SDK's teardown calls

Two details decide whether this works. First, the hook must run on every exit path: register it so Ctrl+C and normal shutdown both reach it, and guard the set-up so an exception mid-launch still closes the device. Second, closing alone frees the handle but leaves the trigger latched — restoring TriggerMode is what makes the camera usable, and closing is what makes it openable. The proof is one command away: after the exit, your viewer opens each camera and streams with TriggerMode reading Off.

If the node must stay resident between external triggers instead of exiting, the resident pattern in keep a camera open between grabs is the alternative shape; if you only need to pause a stream while the process lives, stopping a stream without closing the app covers that case.

Worked example: three cameras on one NIC

Once teardown is fixed, the stack restarts with all three nodes plus a viewer streaming at once — and the next fault appears: torn, missing or discarded frames. The usual place is one shared 1 Gb/s port carrying everything.

The example camera is a 12 MP GigE model running 9 fps; the LS-CMR-0053 is exactly that class of camera.

Frame size: 4000 px × 3000 px = 12,000,000 px, and mono 8-bit data costs 1 B per pixel, so one frame is 12 MB. At 9 fps the camera pushes 12 MB × 9 = 108 MB/s, which is 864 Mb/s of payload.

A 1 Gb/s port moves 125 MB/s of raw line rate, and Ethernet framing, IP/UDP headers and inter-packet gaps take roughly 8%, so about 115 MB/s ≈ 920 Mb/s survives as payload. One camera alone takes 864 ÷ 920 ≈ 94% of a port. Three cameras demand 3 × 108 = 324 MB/s = 2,592 Mb/s, roughly 2.8 times the port, and the driver starts discarding frames — the bandwidth family of errors.

What moves the result: color data doubles the frame to 24 MB and each camera to 1,728 Mb/s; frame rate scales it linearly — at 30 fps one camera needs 2,880 Mb/s, past the whole 920 Mb/s of usable payload on a 1 Gb/s port, which makes it a 10GigE or USB3 decision rather than a tuning problem. The 12 MP USB3 version uses the same sensor and runs at 30 fps. Jumbo frames recover a few percent of headroom. The decision: one camera per port, and pick the interface from the frame rate, not the other way round. Filter the camera finder by interface when you re-spec.

Give every camera its own port and the contention disappears. When you spec the host machine, count LAN ports first: the MV-VC3301H-128G66 exposes 5+ LAN ports and four built-in light channels.

Checks before the next run

A restart is only clean when all of these hold:

• Every process from the last session is gone — check the process list twice.
• Your viewer opens each camera and streams with TriggerMode reading Off.
• The NIC counters show zero discarded frames across 60 s of continuous streaming.
• Each port sits under about 90% of the 920 Mb/s planning load at your frame rate.
• The shutdown hook logged its full sequence on the last Ctrl+C.

If you are speccing the station rather than repairing it, the camera configurator walks interface, resolution and lens choices in order.

When a power cycle is the right call

Three cases justify the cable: a hung host driver after a crash on USB, a camera in an unknown state after power loss mid-write, and hardware swaps. Everything else is deferral. Track the exit path that preceded each lock — one replug after a crash is recovery, a third replug in a week is a defect in the teardown path.

The decision, per case: the open fails after ROS exits, hunt the live holder (Cause 1); the open succeeds without frames, reset the trigger state (Cause 3); the open fails after a crash with no survivors, wait out the GigE heartbeat, or follow the USB path in camera disappears after capture crash. When you add camera nodes rather than debugging them, the vision controllers range groups hosts by LAN-port count, the systems topic page collects the hand-off guides, and setup questions go through the contact page.