Stereo camera low fps, no detection: measure the delivered frame rate

A stereo stack that runs clean on a recorded bag but goes silent on the live cameras is rarely a detector problem. The pair synchronizer matches left and right images by timestamp, and it holds only the newest handful of messages per side. When either camera's delivered rate is low or ragged, frames lose their partners and get pushed out of the queue, so nothing downstream publishes. Every node starts, subscribes and logs no error, which is what makes this fault so quiet. The fastest path back is to measure the delivered rate on each camera topic, then trace the shortfall to exposure time, trigger mode or the network path, in that order.

Why replay works but live detection stays silent

The bring-up log looks healthy: nodes constructed, subscriptions made, nothing on screen but info lines. Playing a bag finds faces; pointing at the cameras finds none. That combination clears the detector logic, the calibration handling and the message plumbing, because the replay exercises exactly the same downstream nodes. What differs is where the frames and their timestamps come from.

rosbag play paces publication against wall time, but the synchronizer pairs on the header stamps recorded inside the file, and those are complete and evenly spaced, because the recorder kept every frame. Any replay rate pairs cleanly. Your live cameras stamp each image with its real capture time, and delivery is where frames go missing: any image that arrives late or never arrives leaves its partner unpaired. With approximate sync enabled and a queue of 15 messages per topic, as this pipeline launches them, an unpaired frame survives only until 15 newer ones replace it.

Matched, steady rates keep pairs alive even when both cameras run slow together. What empties the window is a rate that is low and ragged at once: bursts, dips, unequal sides. Set the floor at 15 fps delivered per camera and hold both sides well clear of it. Below that floor, ordinary jitter consumes the queue and the matched-pair output collapses toward zero while both nodes keep running.

Step 1: measure delivered fps on both camera topics

Bring the stack up as usual and leave the detector off for the first measurement. Run the topic rate tool on each camera topic in turn — rostopic hz /stereo/left/image_raw, then the right-hand topic — and let it print for a full 30 seconds. Read two numbers: the average rate, and the spread between the slowest and fastest intervals. A healthy pair sits within a frame or two of its configured rate, 29 to 30 fps on a 30 fps setting, with a tight spread. A failing pair shows single-digit averages, or a wide spread that means frames arrive in bursts.

Run the same command on the disparity output topic. If both camera topics hold 30 fps but disparity sits near zero, the sync window is the fault and the camera settings are innocent — jump to the plumbing checks near the end. If the camera topics themselves fall short, keep going in the order below.

When one number looks impossible, isolate it. Disconnect the right camera and confirm the left still holds its rate, then swap the two cables and ports to see whether the fault follows the camera or stays with the port. Testing one stream at a time untangles two suspect numbers far faster than watching them together, and the same discipline scales to three or more cameras on one host.

Cause 1: exposure time over the frame period

A frame cannot finish before its exposure does, so exposure sets a hard ceiling on rate:

An indoor scene on auto exposure easily lands at 60 to 70 ms, and 1 s / 0.066 s = 15.1 fps, which is the floor before the network enters the arithmetic at all. Auto exposure adds a second failure: the exposure it chooses moves frame to frame as the scene shifts, so the delivered rate is ragged exactly when it is low, and ragged is what drops pairs.

For a 30 fps target the period is 33.3 ms, so cap exposure at 16 ms and let readout keep the rest. If the image turns too dark at 16 ms, add light or open the iris; a camera sitting at 250 ms and 3 fps is reporting a lighting problem, not a network one. Lock exposure and gain with the auto modes off — the same habit that keeps brightness from drifting during a run.

The rate falling as exposure climbs is expected physics, and the full mechanism is in why your GigE camera's frame rate drops at longer exposure times. This cause is the most common and the cheapest to test: set the exposure, remeasure, and watch the average jump.

Cause 2: trigger and acquisition settings that cap the rate

Next, check how frames are being requested. A camera node launched with a trigger setting and no trigger source publishes nothing at all — rate zero, not rate low. The camera nodes in this bring-up run in streaming trigger mode, which is correct, but a settings file loaded after them can override it. Set streaming explicitly and confirm frames reappear; a node that stays silent is the acquisition-mode problem, and why a GigE camera ignores its external trigger sorts the usual suspects in order.

Two quieter caps hide in the same settings. The acquisition frame rate may be pinned below what the sensor can deliver: request 30 fps and read back what the camera actually accepted, because the advertised mode list decides what is available — the frame-rate mode check shows how to verify it first. Pixel format multiplies everything downstream as well: 12-bit or packed formats double the bytes per pixel on the wire, which turns a comfortable bandwidth budget into an exceeded one.

Cause 3: two cameras sharing one network path

The last cause is arithmetic, and it catches pairs the hardest because each camera alone looks healthy.

On the wire, an 8-bit mono or raw Bayer frame costs one byte per pixel; the colour image is assembled later, on the host. A gigabit path carries about 115 to 120 MB/s of image payload once protocol overhead is paid; plan on 118 MB/s.

Worked example: a pair of 6 MP cameras, 3,072 × 2,048 pixels, streaming 15 fps each into one shared gigabit path.

• Per frame: 3,072 × 2,048 = 6.29 MB at 8 bits.
• Per camera: 6.29 MB × 15 fps = 94.4 MB/s.
• Pair demand: 94.4 × 2 = 188.7 MB/s against a 118 MB/s path.
• Delivery scales by 118 / 188.7 = 0.62, so each camera lands near 9.4 fps — under the 15 fps floor, and the detection window goes dark.

Each camera on its own link would be comfortable at 94.4 MB/s, 80% of one ceiling; only the sharing breaks it. Fixes from cheapest upward: 2 × 2 binning divides pixels by four, taking the pair to 47.2 MB/s; a smaller ROI trims to the region the detector reads; a second network interface gives each camera its own path; USB3 or 10GigE hardware raises the ceiling itself.

Budget the USB3 host the same way. A hub shares one controller, and the USB hub bandwidth budget does that arithmetic per port. On some systems the driver's own buffers need raising before a USB3 camera holds rate, as in the USB3 low-frame-rate fixes, and missed frames can originate on the host side rather than the wire, a pattern why a USB camera misses frames on a weak host link demonstrates. A rate that falls off a cliff mid-run points at the link rather than the settings: see the ranked causes in camera FPS drops suddenly.

Check the negotiation too: both link partners should report 1000 Mb/s, full duplex. A marginal or over-long run renegotiates lower or retransmits, and a camera that raises a packet-size error at runtime has dropped below gigabit — the packet-size error walkthrough traces that case end to end. Keep every run within its cable's rating and terminate in locking connectors; the camera data cables page filters by length and type.

Fix it in this order

1. Measure first. Thirty seconds of rate output on each camera topic is the baseline; change nothing before it exists.
2. Cap the exposure. Lock exposure at 16 ms for a 30 fps target and turn the auto modes off. Remeasure — the average should rise to the readout limit immediately.
3. Clear the acquisition path. Set streaming trigger mode, request the target rate, and read back what the camera accepted.
4. Check the budget. Multiply the pair payload and compare it with 118 MB/s per shared path; hold every shared path under about 88 MB/s so retransmissions keep headroom.
5. Trust the link, then the cable. Confirm 1000 Mb/s full duplex at both ends, reseat and replace suspect cable, and raise the packet size only afterwards, on a dedicated network.
6. Remeasure, then retest the detector. Raising queue_size before the input is fixed only buffers the shortage; revisit it last, and only if bursty arrival still drops valid pairs.

When both cameras hold rate and detection still stays silent

Once both camera topics sustain target rate and the disparity topic is still quiet, the fps theory is dead and the rest is plumbing. Run the rate tool on the rectified topics first: the processing node only produces them when it holds valid calibration for both sides, so a missing or wrong camera_info file appears as rectified topics that never publish. Then check namespaces and remaps, so the detector subscribes to the same topic names the processing node emits — a silent mismatch looks identical to starvation from the outside. Confirm the display flag is set so the result window actually opens.

If a grab that used to run now stalls mid-stream, recover the stream and check the link as in recovering an out-of-sync image stream. When the rates are healthy but the host CPU is saturated, the shrink-the-search tactics in pattern matching at low fps buy the update rate back. More rate diagnostics collect on the systems topic page.

Prevention checklist

• Measure at every bring-up. Thirty seconds of topic-rate output on both camera topics catches driver, link and exposure regressions in one number each.
• Run locked, not automatic. Exposure and gain pinned, auto modes off; automatic exposure moves frame rate and brightness together, invisibly.
• Do the budget at design time. Pixels × bytes × fps × cameras, against 118 MB/s per shared path, keeping 25% headroom.
• Keep the camera network dedicated. No office traffic on the camera switch, and no unilateral packet-size changes on shared plant infrastructure.
• Terminate for vibration. Screened camera data cable, locking connectors, run lengths within rating.
• Record a 60-second bag at full rate. If the replay finds faces and the live pair does not, the fault sits between sensor and sync queue, not in the detector.

Parts that hold the rate

A pair that must hold 30 fps over GigE wants modest pixels per frame, a global shutter, and one gigabit path per camera. At 2.5 MB per frame, 30 fps costs 75 MB/s per camera — 64% of one path, leaving room for retransmissions.

If the host sits close enough for USB3, moving a camera to USB3 takes its traffic off the plant network entirely. A USB3 port sustains roughly 350 to 400 MB/s of real payload after encoding overhead, about three times a gigabit link, so 6 MP at 30 fps fits with room to spare: 6.29 MB × 30 fps = 188.7 MB/s.

Terminate every run for the vibration it will see. A short run that flexes with a stage wants a 3 m high-flex cable with screw locks; a wall or gantry run wants a ready-made 15 m length with locking RJ45 studs rather than a field splice. The configurator's extras step pairs cabling with the camera choice.

Filter the area scan camera category by interface, resolution and shutter to shortlist the pair, or walk cameras, mount and cabling end to end in the camera configurator; the camera finder does the same filtering without a build. If the pair must live inside an existing cell and the numbers will not close, ask the applications team with your two rate measurements attached — that one number halves the diagnosis.