Multicamera alignment crash: isolate concurrent processing faults

An alignment call can be where corrupted memory becomes visible, rather than where the damage began. This makes intermittent crashes difficult to diagnose when several cameras, acquisition threads, and processing workers run together. The practical approach is to separate frame ownership, processing-state ownership, and camera lifecycle changes. You can then change one concurrency boundary at a time, identify the smallest failing configuration, and collect evidence that distinguishes an application race from a runtime defect.

Start with the failure boundary, not a replacement camera

A double-free report means the memory allocator detected an invalid repeated deallocation. It does not, by itself, identify which component first broke the ownership contract. The visible stack might show alignment, allocation, or profile creation because that operation encountered previously damaged state.

Work through these hypotheses in order:

1. A frame view outlives its owning object, or a producer overwrites pixels while a consumer reads them.
2. Multiple workers access mutable processing state without the synchronization its API requires.
3. Startup, profile changes, reconnects, or shutdown overlap active processing.
4. Application binaries and runtime libraries are incompatible, or a runtime defect remains after application ownership is verified.

This is a practical investigation order, not a statistical ranking. Move a hypothesis earlier when the first sanitizer report points directly to it.

If the crash specifically occurs during registration or undistortion, the undistortion pipeline crash investigation provides a related transformation-focused path. Here, the distinguishing question is whether concurrent execution changes ownership or shared state.

Establish an explicit ownership contract

Draw the path from acquisition to final consumer. For every arrow, record whether it transfers an owning frame handle, a deep copy, or a borrowed view.

A matrix or image wrapper constructed around an external pointer may own only its metadata. Copying that wrapper does not necessarily copy the pixels. Likewise, a reference-counted frame can preserve allocation lifetime without preventing another component from modifying its contents.

Use one of these contracts at each asynchronous boundary:

• Retain the documented owning frame object until every reader finishes, and prohibit writes while readers hold it.
• Copy pixels into application-owned storage before releasing the source frame.
• Transfer exclusive ownership of a buffer, returning it to the producer only after processing completes.

Document the processing object's ownership separately. A worker-local wrapper may still share internal state after a shallow copy. Prefer an independently constructed processing instance for each camera's stable stream configuration, subject to the runtime's documented threading rules.

Keep a complete frame bundle together: camera identity, stream profiles, frame handles, capture metadata, and configuration generation. Do not build a bundle by taking the latest color image and latest depth image from unrelated shared variables.

The frame-memory lifetime guide explains the borrowed-memory distinction across language boundaries. If camera identities become mixed, follow the separate cross-camera frame-buffer investigation.

Run a controlled concurrency matrix

Begin with a fixed scene, fixed stream profiles, and no optional recording, inference, display, or reconnect logic. Keep camera identities explicit and change only one test condition at a time.

For the serialization experiment, use one diagnostic lock shared by all relevant workers. Locking each worker's private mutex does not prevent their calls from overlapping. Include construction or profile initialization in a separate controlled experiment if those operations appear in the failure trace.

Do not present serialization as a universal repair. It changes scheduling and can hide unrelated races. If it prevents failure, repeat with ownership checks and preserved metadata before deciding that concurrent runtime access is responsible.

Use a test budget rather than a vague statement that a run was stable. At an assumed 30 frames per second, 100,000 processed frames represent:

Record both processed-frame count and elapsed time. Passing that budget is evidence, not proof that a rare race is absent.

Make buffer reuse safe before tuning throughput

A double buffer is not automatically a synchronized queue. Two slots still fail when a producer wraps around before the consumer releases its previous slot.

Use a clear ownership transition:

• Free: the producer may acquire and write the slot.
• Filling: no consumer may read the slot.
• Ready: a complete bundle is published through a synchronized handoff.
• Reading: the producer cannot reuse the slot.
• Released: the final consumer returns ownership.

Protect these transitions with a correctly implemented queue, mutex, or documented atomic protocol. A flag without the necessary memory ordering may publish metadata before pixel writes are visible. Multiple consumers also need an explicit rule for deciding when the last reader has finished.

Choose a deliberate full-queue policy: block acquisition where supported, discard a new bundle, or discard an unclaimed old bundle. Never overwrite a slot currently held by a consumer. If every frame matters, measure whether sustained processing capacity exceeds the incoming rate.

A small sizing example makes the pressure visible. Assume tightly packed color images with 3 bytes per pixel and depth images with 2 bytes per pixel:

That is approximately 36.9 MB of payload storage, excluding processing outputs, retained runtime frames, and metadata. For actual allocations, calculate each plane from its row stride and height; padding and planar layouts can change the total.

Use the multi-channel buffer-size checks to validate layout assumptions. For stalled handoffs rather than crashes, investigate callbacks and buffer queues.

Keep alignment state separate from timing and lifecycle state

Alignment maps image data between coordinate systems. It does not make independently captured exposures simultaneous. A geometrically valid transformation can still produce poor correspondence when capture times differ.

For each submitted bundle, log:

• Stable camera identifier and configuration generation.
• Source and target stream identifiers, dimensions, and pixel formats.
• Frame counters, timestamps, and timestamp domains.
• Producer and processing-worker identifiers.
• Processing-instance identifier and queue-slot generation.
• Acquisition, processing-start, and processing-finish times from a monotonic host clock.

Compare device timestamps only when their domains and synchronization behavior permit it. Preserve the runtime's documented paired-frame relationship rather than assuming equal counters or identical timestamp values across streams. The asynchronous result-matching guide covers associating later processing output with its original frame.

For a fixed stream configuration, keep processing state persistent where the API supports that pattern. Repeatedly creating and destroying alignment objects adds lifecycle activity and makes initialization effects harder to separate. Persistence does not establish thread safety; ownership and permitted concurrent calls still need verification.

Treat a profile change or reconnect as a controlled generation change. Stop accepting new work, drain or cancel outstanding tasks according to the API contract, release old resources, and then initialize the new configuration. Never destroy a pipeline or processing instance while another thread can still enter it.

If shutdown is the trigger, use the dedicated acquisition-stop crash investigation. For geometric correctness after stability is restored, check raw versus rectified depth-to-color registration.

Capture the first actionable memory error

Build a reduced reproducer with debug symbols and AddressSanitizer where your compiler and target support it. Preserve the complete first report, including allocation, earlier deallocation, and failing-access stacks when available.

The final alignment stack alone is insufficient. Look for the earliest invalid access or ownership violation, then identify the camera, frame generation, and worker associated with that event.

Run ThreadSanitizer separately where supported. It detects many data races, while AddressSanitizer targets memory-access and lifetime errors. Their instrumentation requirements differ, so do not assume they can be enabled together. Sanitizer overhead also changes scheduling; a failure disappearing under instrumentation is not a clean bill of health.

Application instrumentation may not expose accesses inside an uninstrumented runtime library. State that limitation when interpreting a clean run. If a failure remains inside runtime internals, preserve the exact library build identifiers and obtain supported diagnostic builds or guidance through the manufacturer's support channel.

The native crash tracing guide explains how to separate the visible crash site from an earlier invalid memory operation.

Avoid changes that merely disguise the problem

Changing alignment direction is a useful experiment only if you also account for the changed output geometry and processing path. A successful reverse transformation does not establish that the original direction was invalid.

Similarly, switching from polling to a blocking acquisition call changes scheduling and CPU usage. Follow the documented acquisition model. If polling is required, avoid an uncontrolled busy loop and measure its scheduling effect separately from alignment correctness.

Adding sleeps, increasing queue capacity, or catching exceptions around processing can conceal symptoms. None repairs a use-after-free or unsynchronized write. Native memory corruption also cannot generally be recovered safely by catching a language-level exception and continuing.

Record every modification in the test matrix. If you change stream size, frame rate, queue policy, and runtime version together, you lose the ability to identify which change mattered.

Check hardware requirements without assuming a purchase fixes it

Investigate hardware changes only when measurements identify a resource or compatibility limit. Check the required interfaces, sustained transfer rate, supported host architecture, runtime version, and processing headroom.

For a separate conventional imaging task, the area scan camera category and camera finder help narrow acquisition requirements. The LS-CMR-0166 is a USB3 area scan camera, not evidence of a depth-alignment solution.

For station integration, review vision controllers. The MV-VC2140-128G40-1T is a vision controller, but the supplied catalog information does not establish compatibility with your depth-camera runtime or processing workload.

Neither listing should be treated as a repair for an unresolved software race. Confirm the complete device, host, and runtime combination before selecting replacement hardware.

Verify the fix and prepare an escalation package

Before accepting a repair, repeat the originally failing concurrent configuration, not just the reduced single-camera test. Include startup, steady operation, queue saturation, and orderly shutdown. Test reconnect behavior separately if your application supports it.

Use this final checklist:

• Every asynchronous image reference has a documented owner and release point.
• Consumers cannot read buffers that producers may overwrite.
• Processing instances follow documented concurrency rules.
• Frame bundles retain camera identity and configuration generation.
• Old-generation work cannot access replacement resources.
• The failing test now passes a recorded frame-count and duration budget.
• Runtime builds, executable symbols, logs, and sanitizer reports are preserved.

If escalation is still necessary, provide the smallest reproducer, the test-matrix outcomes, and the first complete diagnostic report. Include exact software and firmware versions without assuming that the visible stack identifies the responsible component.

For help translating the evidence into acquisition requirements, use technical support. If testing establishes a genuine need for a different conventional camera, start with the camera configurator and carry the measured requirements forward.