Image based control latency: budget the full feedback loop

A vision system can deliver a high frame rate while acting on information that is already too old. That distinction matters when images drive position correction, process regulation, or a time-critical reject decision. You need to know when the scene was sampled, when the command changed, and how much those intervals vary under load.

For image based control latency, start with a deadline tied to a physical event, not an acquisition benchmark. Then measure the complete path and identify its longest variable stage. The approach below separates sampling delay, image delivery, computation, and output timing so you can choose an architecture and validate it before committing to hardware.

1. Define the feedback deadline before choosing hardware

Write the requirement as an interval between two observable events. Examples include exposure start to electrical output, exposure midpoint to command acceptance, or a process change to actuator response. These are different requirements and need different measurements.

For each requirement, record:

• The starting event and its clock or electrical marker.
• The finishing event, including whether it means output transition or physical motion.
• The maximum permitted delay and variation.
• What the controller must do when an image or command arrives late.

A deadline of 800 microseconds from exposure start does not mean a disturbance will receive a response within that interval. With periodic acquisition, a disturbance may wait almost a full sampling period before the next suitable image.

Likewise, a command reaching an output terminal does not establish actuator response time. Keep actuator dynamics in a separate budget unless the requirement explicitly includes them.

The glass-to-glass latency measurement guide explains another end-to-end measurement boundary. For control, replace the display endpoint with the command or physical response you actually need.

2. Build a budget around the actual critical path

Separate fixed delay, variable delay, and overlapping work. Sensor readout and image transport may overlap, so adding their full durations can overestimate latency. Conversely, measuring only the processing function leaves acquisition queues and output scheduling invisible.

For an architecture that processes complete frames sequentially, use this initial budgeting model:

This is a budgeting model, not a universal camera timing equation. Obtain the actual timing diagram and measure the critical path. A streaming algorithm may begin before the complete image arrives, while another algorithm must wait for the final pixel.

For global exposure under approximately uniform illumination, the exposure midpoint is often a useful measurement-time approximation. Midpoint-to-output latency is then exposure-start-to-output latency minus half the exposure duration. Rolling-shutter images need row-dependent timing instead; one timestamp cannot describe every row's sampling instant.

Use GigE timing isolation when network arrival variation is obscuring the distinction between camera timing and host delivery.

3. Work through a deadline example

Consider a hypothetical complete-frame system operating with a 1,000-microsecond sampling period. These are application assumptions, not specifications for any catalog product.

The arithmetic is:

The nominal path meets the assumed exposure-start deadline. It does not yet establish performance under load.

If processing rises from 160 to 390 microseconds, the additional delay is 230 microseconds. Total latency becomes 850 microseconds, missing the deadline by 50 microseconds. The average processing time could still look acceptable if this happens infrequently.

For a disturbance that must wait for the next eligible exposure, allow up to roughly another 1,000 microseconds in this simplified model. The combined allowance becomes 1,000 + 620 = 1,620 microseconds. Disturbances occurring during an exposure also need a policy for interpreting partially integrated changes.

Delay affects control behavior as well as deadline compliance. A pure delay contributes phase lag according to:

That calculation is not a stability assessment. The complete model must include sampling, output hold behavior, plant dynamics, and controller design without counting the same delay twice.

4. Choose where processing should happen

Moving computation closer to acquisition helps only when it removes a meaningful part of the critical path. A fast host algorithm can outperform a complicated hardware pipeline if transport is already short and the algorithm changes frequently.

A programmable frame grabber can reduce latency when supported logic operates on incoming pixels and the control decision does not require a host round trip. Examples of suitable operations may include thresholding, region statistics, or feature accumulation. Availability depends entirely on the selected hardware and development environment.

Whole-image algorithms can still require frame storage. A result transferred back to the host and then sent through a scheduled output interface may retain much of the original delay.

For host accelerators, use the GPU processing bottleneck checks to distinguish computation time from transfers and waiting. Throughput improvements alone do not prove that an individual command arrives earlier.

5. Select acquisition and compute hardware against the budget

Start with the camera's documented exposure, readout, triggering, and interface behavior. The area scan cameras category and camera finder help narrow interface and shutter choices, but timing qualification requires more than those filters.

The MV-CU020-90GC is a global-shutter GigE camera example. Its catalog classification does not establish suitability for the worked deadline.

For acquisition interfaces, compare the frame grabbers category through the accessories finder. Match the camera interface, supported acquisition mode, host connection, and software environment before considering additional processing capabilities.

The MV-GC1102IOL is listed with a Camera Link interface. The MV-GX1008 is listed with a CoaXPress interface. Neither listing supplied here establishes programmable image processing or a bounded control-output delay.

The Camera Link and CoaXPress acquisition guide covers the underlying acquisition path. Use that compatibility work as a prerequisite, not as evidence that the feedback deadline is met.

For integrated acquisition and processing, consider vision controllers. The MV-VC2140-128G40-1T is a catalog vision-controller example. Its listed light outputs should not be assumed to provide a documented control-command interface or deterministic timing.

Before ordering any platform, request documentation for the exact signal path you intend to use. Ask where timestamps originate, which operations buffer complete frames, and how output timing is specified.

6. Measure exposure to output with correlated markers

Use a controlled test fixture and an approved commissioning procedure. Avoid exercising production motion merely to collect timing data.

1. Identify the exposure reference. Use a documented exposure-active signal where available. A trigger edge is not necessarily exposure start; include trigger-to-exposure delay when those events differ.
2. Observe the actual output. Measure the electrical command edge or documented communication endpoint. A software log written before transmission is not the same endpoint.
3. Capture both markers on a common timebase. A suitable oscilloscope or logic analyzer avoids subtracting unrelated clocks. Confirm signal levels and loading before connecting instrumentation.
4. Correlate images with commands. Record frame identifiers, acquisition timestamps, processing start and finish, and command identifiers. Count missing, duplicate, and late commands.
5. Exercise realistic workloads. Include image saving, communication, operator display activity, accelerator contention, and expected scene complexity.
6. Repeat through startup and recovery. Test acquisition restart, reconnect behavior, configuration changes, and fault handling separately from steady-state operation.

When an exposure marker is unavailable, an independently timed optical stimulus can help establish scene-to-output delay. Its uncertainty includes stimulus timing and the camera's integration behavior, so document those limits rather than claiming exposure-start accuracy.

For asynchronous software, follow the frame-to-result timestamp matching guidance. A perfectly timed output associated with the wrong image is still an invalid feedback result.

7. Report jitter, deadline misses, and image age

Record a latency distribution, not just a mean. Report the sample count, test duration, median, relevant upper percentiles, observed maximum, and number of deadline misses. Include measurement uncertainty and the exact operating conditions.

At 1,000 frames per second, a 10-minute run contains 1,000 × 600 = 600,000 frames. The slowest 0.1 percent represents 600 frames. A high percentile can therefore conceal many events that matter to a fast process.

An observed maximum is not a proven worst-case bound. A hard deadline needs evidence appropriate to the architecture and its failure consequences. Longer tests improve coverage but do not turn an unconstrained scheduling path into a guaranteed one.

Track image age when processing starts. A worker may take only 150 microseconds per image while processing an old frame from a growing queue. Bounded queues make overload visible; unbounded queues convert overload into increasing feedback delay.

Useful interventions include:

• Preallocating image buffers and avoiding unnecessary copies.
• Moving disk writes and display updates outside the control path.
• Separating acquisition from variable-duration processing.
• Fixing exposure when the application permits it.
• Logging queue depth, frame age, and late-result rejection.

If fewer pixels are sufficient, camera-side ROI cropping may reduce delivered data. Confirm its actual effect on readout and delivery timing; cropping after acquisition cannot remove an upstream delay already incurred.

8. Recognize when the usual latency fix is wrong

A latest-frame policy often reduces stale-data delay, but it is wrong when the control or inspection algorithm requires every sample. Dropping frames changes the effective sampling interval. The controller must account for that change or enter its documented degraded mode.

Shortening exposure can reduce integration delay, but insufficient signal can increase measurement noise. Raising gain does not restore missing photons. Evaluate the resulting control measurement, not only the brightness of the displayed image.

Global exposure avoids row-to-row sampling offsets, but it does not by itself establish fast readout or low transport delay. Similarly, a faster interface cannot remove computation or output scheduling that dominates the budget.

Hardware processing is also the wrong default when the required algorithm is unsupported, development overhead is disproportionate, or the command still waits on a slow downstream task. Choose the architecture that removes the measured bottleneck.

If the process is naturally described by sparse brightness changes rather than complete images, the event-based and frame-camera comparison offers another acquisition approach. It changes the measurement model and algorithm requirements rather than simply accelerating an existing frame pipeline.

9. Check the complete path before ordering

Use this checklist at the design review:

• Deadline boundary: The start and finish events match the process requirement.
• Sampling model: Exposure duration, sampling interval, and shutter timing are included.
• Critical path: Overlapping readout and transport are not counted twice.
• Acquisition compatibility: Camera, interface, card, driver, and host configuration are documented together.
• Processing capability: Any proposed hardware operations are explicitly supported.
• Output behavior: Command delivery and downstream scheduling are measured, not inferred from software completion.
• Overload policy: Queues are bounded and stale commands have a defined disposition.
• Validation evidence: Tests include realistic workloads, recovery cases, uncertainty, and deadline-miss counts.
• Control consequences: Delay variation and missing samples are addressed in the controller design.

Start with the acquisition accessory configurator when the interface or card is undecided. For a timing review, send your camera mode, proposed algorithm, output interface, and measured stage delays through application support. The next useful decision is the one supported by a complete timing trace, not the largest frame-rate figure.