When stereo pipeline output queues stall: diagnosis

A stereo application can show one image and then stop even though both cameras worked before stereo processing was added. The cause may be flow control rather than failed acquisition: an unread output fills, a blocking read holds the only consumer thread, or processing cannot keep pace. This troubleshooting procedure helps you distinguish those cases using stream counters, bounded reads and controlled graph changes. Start with the existing hardware, and change one queue or consumer behavior at a time.

Recognize the two different ways queues stop progress

A full queue and an empty queue can both cause waiting, but they require different fixes.

A producer may wait when it tries to write into a full blocking queue. If several outputs share processing resources, that pressure can propagate upstream and interrupt apparently unrelated streams. An unused depth or disparity output is therefore worth investigating even when you display only rectified images.

A consumer may wait when it requests a frame from an empty queue. If the same thread services every stream, waiting for one missing output prevents it from draining the others. This can turn a temporary stream imbalance into a wider stall.

Neither mechanism is universal. Queue ownership, transport buffering and overflow behavior depend on the implementation. Treat these as hypotheses to test, not conclusions drawn from the frozen preview alone.

For acquisition callbacks and buffer ownership more generally, use the companion troubleshooting procedure for multi-camera callback and buffer-queue freezes.

Map every enabled output to a consumer

Draw the complete path from camera inputs through stereo processing to application consumers. Include synchronized images, rectified images, disparity, depth, previews and recording branches. An output remains part of the graph even when its display window is closed.

For each enabled output, record:

• Whether the processing node actually produces that output in the selected mode.
• Whether it is connected to a transport or host endpoint.
• Which thread, callback or worker drains it.
• The queue capacity and full-queue policy at each exposed stage.
• Whether the consumer can wait indefinitely for an image.
• Whether downstream publication, storage or display can block that consumer.

Distinguish device-side input queues, transport queues, host receive queues and application work queues. A setting on one layer does not necessarily control the others.

Pay particular attention to outputs that were added during development but never given a consumer. Depending on the implementation, leaving an endpoint unopened may discard data, retain buffers or block progress. Check the documented behavior instead of assuming that an unused stream is harmless.

When multiple algorithms share one source, the discussion of camera-stream fan-out and independent consumers helps identify where a slow branch can hold shared resources.

Calculate how quickly a queue can fill

Queue capacity buys time; it does not increase sustained processing throughput. Use measured production and consumption rates to estimate the interval before a blocking queue becomes full.

Suppose an output produces 30 frames per second, but its consumer handles only 20 frames per second. An initially empty queue with 8 available slots fills in approximately:

If nobody consumes that output, the estimate becomes:

Actual timing depends on other buffers and scheduling. Nevertheless, a repeatable freeze shortly after startup is consistent with finite buffers filling.

A periodic consumer also needs headroom. A 33 ms timer offers about 30.3 callback opportunities per second before accounting for execution time or missed scheduling opportunities. At an input rate of 30 frames per second, draining only one frame per invocation leaves little margin for disturbances.

For timing beyond the queue itself, follow the method for budgeting the full image-based feedback loop.

Run isolation tests in a controlled order

Use the same resolution, frame rate and stereo settings across tests unless the test explicitly changes them. Save a compact configuration record with each run.

1. Establish a raw acquisition baseline. Capture both inputs without stereo outputs. Count received frames without display, conversion, publication or recording. If this already fails, investigate acquisition before queue coupling.
2. Enable the smallest useful stereo graph. Connect only the outputs required for the first test. Where supported, remove unused output links rather than leaving connected endpoints unattended.
3. Drain every connected endpoint. Consume and immediately release each frame. Record only counters and timestamps initially; detailed per-frame logging can itself slow the consumer.
4. Replace indefinite reads with nonblocking reads or finite waits. An empty stream must not prevent the test from checking all other streams and reporting its status.
5. Restore outputs individually. Add disparity, depth and additional image branches one at a time. Observe which addition changes queue growth or frame arrival.
6. Restore downstream work individually. Reintroduce image conversion, publication, display and storage separately. Measure the added service time at each stage.

Run each condition long enough to exceed its estimated buffer-fill interval several times. For the 0.8-second example, a 10-second run can reveal immediate accumulation, but it is not a production endurance test.

Keep the acquisition path nonblocking and bounded

Changing to a nonblocking read prevents waiting on an empty queue. It does not necessarily change what happens when a queue fills. Configure and verify those behaviors separately.

Avoid sequential indefinite reads in the only servicing thread. The first read may succeed while the second waits forever, leaving every later queue untouched. Finite waits can also accumulate: four consecutive waits of 100 ms can occupy approximately 400 ms when all reach their timeout.

A generic servicing loop can use bounded nonblocking reads:

while running:
    made_progress = false

    for each enabled output:
        repeat up to per_stream_budget times:
            frame = try_read(output)
            if frame is absent:
                break

            update counters and metadata
            hand off using an explicit bounded policy
            made_progress = true

    if not made_progress:
        wait briefly for an event or shutdown signal

This is pseudocode, not a library-specific API. Limit the work per stream so a continuously busy output cannot starve the others. Event notifications or dedicated workers may be better where the acquisition library supports them.

Keep expensive processing outside the acquisition callback. An asynchronous handoff must either retain valid ownership of the image buffer or copy the required data. A borrowed frame view may become invalid after its acquisition buffer is returned.

The handoff queue also needs a capacity and overflow policy. Otherwise, moving work to another thread merely moves the backlog.

Choose an overflow policy that matches the task

Balanced consumption means servicing every required output sufficiently often. It does not mean assuming all outputs have identical production rates.

Do not choose drop-oldest behavior merely because it stops the freeze. A preview can usually tolerate missing intermediate images; an inspection requiring evidence for every part may not.

For lossless operation, the long-term consumer rate must match or exceed the producer rate. Buffering can absorb temporary bursts only when the system later has enough spare capacity to catch up.

Monitor progress, image age and stereo correspondence

Add lightweight telemetry before restoring the full application. Useful per-stream measurements include received count, last arrival time, sequence identifier, queue occupancy where exposed, processing duration and application discard count.

Sample the measurements periodically rather than printing every frame. A one-second reporting interval is a useful starting point for observing sustained rates without excessive console traffic.

Interpret the measurements together:

• Rising occupancy with continued arrivals suggests a consumer throughput deficit.
• One stream stopping before the others identifies a branch to investigate, not necessarily the root cause.
• Continued arrivals with a frozen display points toward presentation or application processing.
• Stable throughput with increasing image age indicates accumulating latency somewhere in the path.

Use device and host timestamps only after checking their clock domains. Subtracting timestamps from unrelated clocks does not produce a meaningful transport latency.

Frames read next to each other are not automatically a stereo pair. Match sequence identifiers or documented timestamp relationships, and expire unmatched entries within a bounded interval. The guide to matching asynchronous images and results with timestamps explains the association problem separately from queue servicing.

Know when the queue explanation is wrong

A pipeline that never produces its first stereo output may have a graph, configuration or startup problem rather than a queue that gradually fills. Investigate supported input formats, output modes, calibration availability and successful device startup.

Use stereo-depth startup and stream-disconnect diagnosis when the failure occurs during initialization rather than after sustained capture.

If every stream continues at a reduced rate, check exposure time, stereo workload, transport payload and processing limits. A documented maximum rate is not necessarily available under every configuration. The procedure for finding camera frame-rate bottlenecks helps separate those limits.

If the process remains responsive but cannot stop, inspect cancellation paths and outstanding waits. Every blocking operation should have a documented route to completion or interruption. Follow the separate checks for slow multi-camera pipeline shutdown rather than treating termination delays as evidence of the original capture fault.

Check hardware only after isolating software flow control

Replacing cameras or interface cards is not the first response to an unread output. If discard-only consumers still reveal transport errors or missing inputs, then review the acquisition hardware and its documented compatibility.

For a host-based design, the area scan camera category and camera finder help organize interface and shutter requirements. The LS-CMR-0053 is a GigE area scan camera. Its listing identifies rolling shutter; suitability for a particular stereo task needs separate evaluation.

For the host interface, compare frame grabbers and interface cards through the accessory finder. The LS-FGB-0004 provides a GigE interface. Confirm host compatibility and acquisition support before selecting it.

These are hardware reference points, not a validated stereo combination or a remedy for queue backpressure. Product listings alone do not establish synchronized exposure, stereo calibration support or compatibility with a particular processing graph.

Verify recovery and prevent recurrence

Before declaring the fault resolved, check the complete operating cycle:

• Every connected output has a documented consumer or deliberate discard policy.
• Acquisition service remains responsive when any individual output is absent.
• Queue occupancy and image age remain bounded during representative operation.
• Any discarded, expired or unmatched frames are counted explicitly.
• Slow display, publication or recording cannot silently block unrelated acquisition work.
• Startup and shutdown complete without abandoned waits or retained frame buffers.
• The application meets its required pairing accuracy, throughput and image-age limits.

Repeat the test with realistic downstream workloads, not just a preview window. Include temporary consumer delays and verify the chosen recovery behavior.

Keep the successful graph configuration, queue settings and counter logs together. If you need help checking acquisition hardware, start with the camera configuration step or send the stream map, measured rates and queue policies through applications support.