Asynchronous vision result frame matching with timestamps

An inspection result can arrive after several newer images have entered the acquisition pipeline. Displaying the newest image beside that result creates a convincing but incorrect overlay. Clearing a queue may change the symptom without repairing the association.

For asynchronous inspection, treat each captured image and its result as a matched pair. The procedure below shows how to preserve that pair using identifiers, validate timestamp-based alternatives, and bound memory without releasing image data that processing still needs. The same design applies to classification, measurement, code reading, and defect inspection.

Choose exact matching before nearest timestamps

Start by checking what the acquisition interface and processing API actually return. A timestamp attached to a result might describe capture, processing start, or callback delivery. Those meanings are not interchangeable.

Prefer these association methods in descending order:

A completion callback should retrieve the image belonging to its request, not ask acquisition for whichever image happens to be newest.

This differs from a scene that has physically moved before a result is used. For that separate problem, use the checks in conveyor scene timing and stale pick data. Correct image association does not make an old result suitable for an actuator.

Record identity and image ownership at acquisition

1. Create a key before submitting work. Use the camera or stream identity, an acquisition-session identifier, and a frame identifier. Change the session identifier whenever acquisition restarts or the device reconnects.
2. Preserve pixels with their layout. Record width, height, pixel format, row stride, and the metadata needed to interpret the image. Either copy the pixels into application-owned storage or retain the SDK buffer through its documented ownership mechanism.
3. Register the image before processing can complete. Publish the pending record before submitting the job. Otherwise, a fast callback can arrive before its lookup entry exists. If submission fails, remove the entry and release its ownership safely.

A useful conceptual key is:

Do not use the memory address of a frame buffer as its identity. Acquisition systems commonly reuse buffer storage after an application returns it.

Store the capture timestamp and host arrival time as separate fields. The first supports acquisition timing; the second helps measure transport and scheduling delays. Neither should silently replace the other.

For reusable acquisition buffers, the critical question is when their contents stop belonging to your application. The guide to camera frame memory lifetime in managed code explains why retaining a wrapper is not necessarily equivalent to retaining its pixels.

Propagate the key through asynchronous processing

4. Attach the key to the request. Use documented user context, a job handle, or a closure that belongs to that specific submission. Do not attach results to a mutable variable called “current frame.”
5. Resolve the result by that same key. Retrieve the pending record, validate its session and processing status, and build the displayed output from that record's pixels.

The association flow should look like this:

capture -> create key -> retain image -> register pending record
        -> submit inspection with key

completion -> recover key -> find pending record
           -> combine retained image with result
           -> release ownership when all readers finish

The pending-record store must support concurrent acquisition and completion safely. Protect lookup, state transitions, and ownership changes with the synchronization appropriate to your runtime. Avoid holding its lock while performing image processing or waiting for a display update.

Handle duplicate callbacks explicitly. A record can move from pending to completed once; another completion for the same job should be logged or rejected according to the API contract.

For several inspections on one image, keep separate job identities beneath the shared frame key. Release the shared image only after all dependent jobs and display consumers finish.

If a callback contains no request identity, check whether the API documents completion order. Submission order alone is not evidence that callbacks remain ordered. Without preserved identity or a documented ordering guarantee, exact association may require changing the integration interface.

Use timestamps only with explicit clock domains

6. Confirm timestamp meaning before comparing values. Establish the event being timestamped, its units, its clock source, and its behavior during restart. Compare timestamps directly only when they share a clock domain or have a validated conversion between domains.

A camera timestamp and a host monotonic timestamp can both increase steadily while having unrelated origins. Their rates may also differ slightly. Do not subtract them and interpret the difference as inspection latency without establishing that relationship.

The camera and IMU timestamp clock-domain guide covers the same distinction between timestamp values and the clocks that generate them.

When a result returns the original capture timestamp unchanged, first attempt an exact lookup within the correct stream and session. Use nearest-time matching only if the interface genuinely requires it:

For example, a stream running at 20 frames/s has a nominal frame interval of:

Suppose validation establishes that the timestamp disagreement remains within 3 ms for the intended operating conditions. An illustrative acceptance tolerance of 8 ms is larger than that disagreement and smaller than half the nominal frame interval, which is 25 ms.

Those numbers are not a universal setting. Verify the minimum actual spacing between candidate images, including burst acquisition and timestamp rounding. Reject ties, missing candidates, and matches outside the validated tolerance. Define explicitly whether multiple results may refer to one image.

A timestamp assigned when the result callback runs describes completion, not capture. Matching it to the nearest captured image usually favors a newer, unrelated frame. Do not use that fallback.

Size a bounded buffer from retention time

7. Measure how long each accepted image must remain available. Include waiting time before processing, execution, result delivery, and any display ownership. Use that retention interval to size the pending store, then enforce admission and expiration policies.

A first capacity estimate is:

For an illustrative accepted rate of 40 frames/s, a retention interval of 0.35 s, and a margin of 4 frames:

Assume tightly packed monochrome images measuring 1920 × 1200 pixels, with one byte per pixel:

This excludes metadata, allocator overhead, processing outputs, acquisition buffers, and additional display copies. For padded rows, calculate storage from row stride multiplied by height. Packed pixel formats require their actual layout rather than a guessed byte count.

A latency percentile is useful for capacity planning, but it is not a maximum. Define what happens when a job exceeds the designed interval: reject its late result, mark it expired, or apply documented cancellation. Never substitute the newest frame after an expiration.

Removing an expired lookup entry does not authorize freeing pixels that an active worker still reads. Bound in-flight work separately and retain ownership until that worker actually finishes or cancellation is confirmed.

For queue behavior that blocks acquisition itself, follow multi-camera callback and buffer-queue troubleshooting. A larger pending store cannot repair a sustained processing-rate deficit.

Verify matching with deliberate completion reordering

8. Test identity independently of inspection accuracy. Feed known frames into the pipeline, introduce controlled processing delays, and verify that each result retrieves its own pixels.

Use a test source with an unmistakable visible identifier, and log its corresponding frame key. For example, assign three consecutive jobs delays of 120 ms, 30 ms, and 60 ms. With closely spaced submissions and sufficient processing concurrency, completions should arrive out of submission order. The image-result pairs must remain correct regardless of their completion order.

Record these fields for each job:

• Stream, session, frame key, and processing job identity.
• Capture timestamp, its units, and its clock identity.
• Host submission and completion times from the same monotonic clock.
• Match outcome, pending-record count, and expiration reason.
• Time when the final image owner releases its reference.

Repeat the test with a dropped frame, a duplicate callback, a processing failure, a reconnect, and a deliberately late result. Old-session callbacks must not match images from the new session.

Also choose a display policy. A completion-ordered audit view can show matched pairs as they finish. A live preview may suppress older completions to avoid moving backward in capture order. Suppressing a display update is different from pairing an old result with a newer image.

Measure durations with an appropriate host timer. The distinction in CPU time versus wall-clock pipeline benchmarking matters when a job spends time waiting rather than executing.

Check hardware and API fit before ordering

Frame matching is principally an integration requirement, not a reason by itself to replace a camera. Start with the existing SDK documentation and verify metadata availability, buffer ownership, request context, and restart behavior.

When selecting area scan cameras, use the camera finder to narrow the acquisition interface and imaging requirements. Then confirm the required software behavior separately.

The LS-CMR-0166 is a USB3 area scan camera with a rolling shutter. It is an acquisition candidate, not evidence that a particular asynchronous API preserves job identity.

For an integrated acquisition and processing platform, review vision controllers. The MV-VC3301H-128G66 is a vision controller with built-in light outputs. Confirm your camera interface, software dependencies, and licensing requirements before selecting a platform.

Neither product selection nor interface bandwidth establishes a result-to-image association guarantee. Request that guarantee from the documented software contract and verify it with the reordering test.

Acceptance checklist and next step

Before releasing the implementation, confirm the following:

• Every submitted job carries immutable identity from its captured image.
• Keys distinguish streams and acquisition sessions.
• Pixels and layout remain valid for every active reader.
• Pending records exist before completion callbacks can access them.
• Timestamp matching uses documented events and comparable clocks.
• Ambiguous, expired, and unknown results fail explicitly rather than selecting another frame.
• Pending images and active jobs have separate, enforceable limits.
• Reordering and reconnect tests preserve the correct image-result pairs.
• Display freshness rules remain separate from association correctness.

Do not replace an exact match with nearest-time matching merely because the callback is asynchronous. Likewise, do not clear acquisition buffers unless the documented recovery procedure calls for it; clearing them can discard useful evidence without fixing ownership or identity.

For a new acquisition setup, start with the camera configurator. If the API contract remains unclear, send the metadata fields, timing measurements, and ownership description through application support before choosing replacement hardware.