Two camera closed loop latency: grab and process in parallel

A two-camera closed loop that takes 720 ms per iteration when one camera took roughly half that does not need a faster PC; it needs its stages to stop queueing. The doubling almost always comes from a non-reentrant processing subVI that admits one call at a time, and from two frame grabs wired one after another. This page is for anyone steering mirrors, gimbals or stages from the centroids of two GigE camera images. You will measure each stage, compute the parallel floor of your own loop from three numbers, and apply three fixes in the order that pays. In the worked case below the loop drops from 720 ms to about 350 ms, doubling the correction rate from 1.4 to 2.8 iterations per second.

What the doubling tells you

Add up what the loop does and the ratio tells the story. One camera's grab plus centroid takes a stage time t. Two cameras running in parallel should take close to the slower of the two stages. Two cameras in series take the sum. A measured 720 ms for two cameras, where one camera costs roughly half that, is the signature of series execution: the second stage starts only after the first finishes.

Compute growth looks different. It creeps with resolution, template size and every addition to the code, and it does not double cleanly when you add a camera. A clean 2× that appeared with the second camera points at scheduling, not at math.

Three measurements are enough to rebuild the whole budget: the total per iteration with both cameras running, the extra time when the processing runs through a subVI instead of inline, and the time of one call into the centroid DLL. For the view from camera pixel to actuator output, the image-based control latency budget walks the full feedback chain.

The likely causes, in order of probability

Work down this list; each cause produces the same symptom and each has a quick test.

1. A non-reentrant processing subVI. With reentrancy off, only one call can enter and execute the subVI at a time, so two parallel branches still run through it one after another. This is the most common cause of a clean 2×.
2. Two grabs on the same wire. A frame wait returns one image from one session; wired in sequence, the second camera's wait cannot start until the first image lands, so acquisition itself serializes.
3. A blocking DLL call inside the loop. A measured call costs 123 ms; configured on the UI thread it also funnels every parallel branch through one thread and freezes the front panel while it runs.
4. One shared host link. Both sessions through a single NIC port with aggressive interrupt moderation can queue frames behind each other; the signs are arrival jitter and one pinned core. See GigE camera interrupt moderation: balance latency and CPU load and Two GigE cameras on one host: NIC ports and bandwidth budget.

Time each stage before you change anything

Half an hour of measurement tells you which of the four you have.

1. Record the driver's buffer timestamps for every frame, not just wall-clock time around the loop; timestamps separate capture time from hand-off time (Camera buffer timestamp latency: interpret frame timing).
2. Run one camera alone and write down grab wait, processing time and total over, say, 100 iterations.
3. Start the second camera exactly as the loop runs it. If the total lands near the sum of the two single-camera runs, execution is serialized, and that number is your diagnosis.
4. Set the processing subVI reentrant and re-measure. No change means the queue is not in that subVI; look at the grabs and the DLL call.
5. Call the DLL with dummy parameters of the same sizes. If the call collapses from 123 ms to microseconds, the time lives inside the DLL's own code; put a timer inside the function, or call it from a minimal test program, to split call overhead from compute.
6. Watch per-core CPU while the loop runs. One saturated core beside idle ones marks a funnel: a UI-thread setting, a non-reentrant node, or a single NIC queue.

Worked example: the budget for a 720 ms two-camera loop

The inputs come straight from such a timing test: 720 ms total per iteration with both cameras in series; 140 ms of extra time when the processing runs through the non-reentrant subVI rather than inline; 123 ms for one call into the centroid DLL. The budget assumes both camera stages are equal and each overhead occurs once per iteration.

The rule in words: a serialized loop costs the sum of its stages plus overheads; a parallel loop costs the longest branch plus only the overheads that genuinely cannot overlap.

The arithmetic, step by step:

• Per-camera stage: (720 - 140 - 123) / 2 = 457 / 2 ≈ 228 ms, call it 230 ms of grab and image handling per camera.
• Closure check: 2 × 230 + 140 + 123 = 723 ms, which matches the measured 720 ms within rounding, so the budget is consistent.
• Parallel floor with the DLL called once on both images: max(230, 230) + 123 = 353 ms, about 350 ms.
• Parallel floor with the DLL thread-safe and called per camera: each branch costs 230 + 123 = 353 ms and the branches run together, so the floor is the same. Both call strategies converge, which is the useful result.

Rates: 1000 / 720 ≈ 1.4 iterations per second before the fix; 1000 / 353 ≈ 2.8 after.

Decision: treat 350 ms as the floor this hardware holds and 2.8 Hz as the loop rate to expect. If the loop sits above about 400 ms after the fixes, one branch still queues behind another, so go back to the cause list. Below 350 ms, stop tuning: the remaining time is the DLL's own compute and the cameras' frame periods.

Fix one: an acquisition branch per camera

Give each camera its own branch: its own session, its own grab, its own wait, joined before the shared processing step. Two queues or a rendezvous feed the centroid stage, which starts once both images have arrived.

Never place two frame waits on the same wire. A wait returns one frame from one session, so the second cannot begin until the first returns; that one-wait-per-frame rule is its own failure mode (Frame didn't arrive within timeout: one wait call per frame).

For a correction loop, run both sessions continuously and read the latest completed frame each iteration instead of issuing a fresh grab per cycle; the continuous acquisition mode article shows that loop shape. If the two cameras must expose at the same instant, trigger them from one source as described in hardware trigger configuration per camera.

Fix two: reentrant processing subVIs

Set every processing subVI that both branches call to run reentrant. The semantics are the point: with reentrancy off, when several callers hit the same subVI at once, only one call enters and executes it at a time while the rest queue. That queue is your measured 140 ms: the call mechanism itself costs microseconds, so nearly all of it is the second branch waiting for the first to leave.

LabVIEW offers two reentrancy modes, shared clones or one clone per instance. For two camera branches, give each caller its own clone so each branch's image buffers sit in their own data space. Reentrancy is necessary but not sufficient: it removes queueing inside the subVI, while the order of the grabs and the DLL call on the diagram still decides whether the branches overlap at all.

Fix three: the DLL call that costs 123 ms

The node that calls the DLL offers two thread settings, Run in UI Thread and Run in any thread; choose Run in any thread. A UI-thread call drags every branch through one thread and blocks the front panel for the duration, which alone can double a loop that was written to run in parallel.

The call mechanism's own overhead is microseconds, so a 123 ms call means the function's internal code runs that long. Three ways to place it in the budget:

• Call it once on the pair of images and accept 123 ms as shared cost. The floor stays near 350 ms and the loop runs 2.8 Hz, which meets the correction rate in the worked example.
• Make the function thread-safe and call it once per branch. The floor is the same, so this only pays if you also shorten the code itself.
• Profile inside the DLL. Two centroid passes over images of about 1.7 MB each should not need 123 ms; an order of magnitude is usually recoverable there.

For the buffer and callback side of calling camera code from LabVIEW, see Call a camera SDK DLL from LabVIEW: callbacks, buffers and snap.

Two traps after the fix

Timing still doubles with reentrancy on. Reentrancy removes queueing inside the subVI and nothing else. If the two grabs sit in sequence, the images still arrive one after another and the processing branches start staggered however the subVI is configured. Check the buffer timestamps: the second frame's capture time should fall within one frame period of the first, not one stage time later. A non-reentrant DLL call node funnels branches the same way.

Memory full after switching configurations. Each clone of a reentrant subVI allocates its own data space and buffers, so a two-camera configuration holds roughly double the image memory of a one-camera one. If the previous configuration's sessions are never stopped and released, allocations stack up; that is why dropping to a one-camera run after a two-camera run can report memory full while repeating the two-camera run does not. Stop acquisition, close every camera session, let the buffers deallocate, then reconfigure.

When the usual answer is wrong

Parallelization has three limits worth knowing before you start.

The camera frame period is the floor. If one exposure plus readout already costs 200 ms, no branch structure helps; read the frame period your session reports and put it under the branch time. A spot centroid needs very little light, so cutting exposure from 100 ms to 2 ms moves that floor by 98 ms and the centroid of a bright spot barely moves. Where capture timing sits — when the shutter actually opened relative to your grab call — is covered in Trigger call versus exposure time: understand capture timing.

The link is almost never the cause. Moving two such frames at GigE's 125 MB/s line rate costs about 27 ms, well under 4% of a 720 ms iteration, so bandwidth explains only tens of milliseconds here. Suspect the network only when timestamps show jitter or late arrivals, as worked through in GigE Vision frame arrival jitter: isolate camera and host timing.

Some stages are physically serial. If both cameras must image through optics that move alternately, the sum is real and the fix is a shorter frame, not parallel branches.

Parts that hold a 350 ms loop

Nothing exotic is required: a camera whose frame period sits far below the branch time, and a host link that gives each session its own port. The MV-CU020-90GC is a 2.5 MP GigE camera with a global shutter and a C-mount, the class of camera a two-camera correction loop uses at this rate.

On the host side, one port per camera keeps the two sessions out of each other's queues. The MV-GE2002P is a dual-port PoE GigE NIC in full height, and the MV-GE1104 is a 4-port GigE frame grabber; either leaves ports free for the next station.

The frame grabbers and area scan cameras categories hold the wider range, and the camera step of the configurator matches a camera to your frame-period budget.

Checks before you run the loop

Work down the list after the changes, then again a week later in production.

• Ratio: one camera at t, two cameras near the slower stage plus shared cost — about 350 ms in the worked case, not the 720 ms you started from. Above about 400 ms, a queue remains.
• Timestamps: the second frame captured within one frame period of the first.
• Reentrancy: every shared processing subVI reentrant, each caller with its own clone.
• DLL node: Run in any thread, called once on the pair or per branch with a thread-safe function.
• Memory: sessions stopped and released before any configuration change.
• CPU: no single core pinned while others sit idle.

Mirror-steering loops usually exist to aim a laser, and that laser's safety integration around industrial processing beams is designed outside the vision code.

Re-run the three-measurement harness, apply the fixes in this order, and read the result against the parallel floor you computed. If the camera frame period or the host link turns out to be the limit, pick a faster part from the camera finder or reach us through the contact page with your timing table; more timing and synchronization guides sit under the systems topic.