Object detection without a GPU: CPU inference for inspection

A pretrained object detector can run without a GPU when its model operations and inference runtime support CPU execution. The practical question is whether it can inspect your actual images before each decision deadline, without missing relevant defects or accumulating a queue. For an inspection integrator, the useful starting point is a small, licensed model and a repeatable benchmark—not a processor specification alone. You will need to measure the complete acquisition-to-decision path, then compare speed and detection quality before selecting hardware.

What CPU-only inference actually requires

Training and inference are different workloads. Training updates model parameters using labeled examples. Inference applies already-trained parameters to new images. Running a pretrained detector does not require you to train it locally, and CPU execution is possible for many detector architectures.

However, a downloadable model is not automatically compatible with every runtime. Check these requirements first:

• The runtime provides a CPU execution backend for your operating system and processor architecture.
• Every operation in the exported model is supported, including any custom detection operations.
• Input dimensions, channel order, numeric type and normalization match the model's requirements.
• Output decoding and suppression of overlapping detections are implemented correctly.
• Model, runtime and deployment licenses permit your intended use.

Select the CPU backend explicitly where the runtime allows it. Inspect initialization messages and execution diagnostics rather than assuming that removing a GPU automatically produces a valid CPU configuration.

An exported model may return raw prediction arrays rather than finished boxes. Include decoding, coordinate conversion and overlap suppression in your application. A fast network with incorrect output handling is not a working inspection system.

Translate the inspection cycle into a deadline

Latency is the elapsed time for one image to produce a usable decision. Throughput is the number of images processed per unit time. Neither measurement can replace the other.

A system might sustain the required average throughput while occasionally delivering decisions too late. Conversely, it might process an isolated frame quickly but develop a growing queue during continuous acquisition.

For a triggered inspection, define the deadline from the physical event that starts the decision window. Include exposure, readout, transfer, preprocessing, inference, postprocessing and communication wherever they occur within that window.

The throughput estimate assumes one frame is completed before the next begins. Overlapped acquisition and processing require measurements of the actual pipeline. Shared CPU resources can prevent the expected overlap from improving performance.

Use elapsed time from a monotonic clock. Summed processor time across several threads does not represent the time a part waits for its decision. The distinction is explained in CPU time versus wall-clock time for vision benchmarking.

Worked example: an inspection with a reject deadline

Consider a hypothetical conveyor application. The inspection point is 150 mm upstream of a reject mechanism, and conveyor speed is 500 mm/s. Assume a 40 ms allowance for control and actuation, plus a separate 50 ms design margin.

These are illustrative application values, not performance claims for any product. A possible measured timing breakdown could look like this:

The arithmetic gives 20 + 8 + 92 + 10 + 5 = 135 ms. Against the 210 ms vision budget, the mean has 75 ms remaining.

If parts arrive every 200 ms, the arrival rate is 1 / 0.200 = 5 parts/s. A sequential service time of 135 ms suggests a theoretical capacity of 1 / 0.135 ≈ 7.4 frames/s. That comparison is encouraging, but it is not acceptance evidence.

Queueing and occasional stalls still matter. A measured maximum of 245 ms would miss the deadline despite the acceptable mean. Investigate late frames, define a response to missing decisions, and test at sustained production load. Ordinary inspection logic must not substitute for a validated machine safety function.

Choose input resolution from the smallest relevant feature

Reducing model input dimensions is often worth testing, but it changes the visual evidence available to the detector. Sensor resolution and inference input resolution are separate decisions.

Suppose a hypothetical image covers 240 mm horizontally across 1920 pixels. A target feature is 6 mm wide. Scaling the whole image proportionally to a width of 640 pixels gives:

Halving both input dimensions reduces the input pixel count to one quarter. It does not guarantee a fourfold latency reduction: model architecture, memory movement and fixed processing overhead also contribute.

There is no universal minimum pixel width that guarantees detection. Contrast, blur, orientation, training examples and the target's shape all affect performance. Test the smallest required feature after the exact resizing and padding used in deployment.

Use the camera-resolution selection guide to establish acquisition sampling. Its geometric approach helps you count pixels, but detector acceptance still requires labeled images. The object-space pixel-size explanation connects sensor sampling to physical feature dimensions.

The calculator addresses acquisition resolution, not neural-network latency. If the full image contains mostly irrelevant background, a fixed region of interest may preserve more target detail than aggressive whole-image reduction. Confirm that normal part movement never takes the target outside that region.

Compare optimization choices without assuming equal accuracy

Start with a lightweight detector that can represent your inspection task. A larger general-purpose model is not automatically the best production choice, especially when classes and backgrounds are tightly controlled.

Change one setting at a time so that timing and accuracy differences remain attributable.

Keep batch size at one for the initial latency benchmark. Test batching only when several images are genuinely available together and every associated decision still meets its deadline.

For quantization, use calibration data representative of production when the conversion method requires it. Verify support for the resulting operations and numeric types. Do not assume that a smaller model file means faster execution.

Thread settings deserve a measured sweep rather than a maximum setting. For example, compare 1, 2 and 4 inference threads while acquisition and communication remain active. Record processor load, queue growth and end-to-end timing for each configuration.

Build a reproducible benchmark

A useful benchmark separates model capability from integration overhead, then measures them together. Record the model version, runtime version, input shape, numeric precision, thread settings and operating-system configuration.

1. Create a representative image set. Include acceptable parts, required defects, empty scenes, position variation and difficult appearances. Keep final acceptance images separate from training and calibration data.
2. Establish a correct CPU baseline. Confirm output shapes, class mapping, image preprocessing and box coordinates before timing optimization.
3. Measure startup separately. Model loading and initial memory allocation matter after a restart, but should not be mixed silently into steady-state averages.
4. Warm up execution. An initial run of 100 frames is a reasonable starting experiment, not a universal requirement. Continue until timing behavior stabilizes.
5. Time each stage and the complete path. Retain per-frame records rather than only a running average. Confirm that inference timing ends when outputs are actually available.
6. Run continuous acquisition. Test the required arrival rate with production communication active. A suggested starting run is 30 minutes; extend it to capture relevant operating variation.
7. Report the distribution and failures. Record the median, 95th percentile, 99th percentile, observed maximum, dropped frames and missed deadlines.

A set of 10,000 timed frames provides more useful tail information than a short demonstration, but it cannot prove a worst-case execution bound. Report the sample count and test conditions alongside the percentiles.

Attach a frame identifier to acquisition, inference and the outgoing decision. When multiple stages operate asynchronously, follow the principles in matching asynchronous vision decisions to their frames. A correctly classified image is still operationally wrong if its decision is assigned to another part.

For inspection, silently discarding an old frame may discard an entire part's decision. Decide explicitly whether overflow stops acquisition, raises a fault or routes the affected part for separate handling.

Validate detection quality alongside timing

Pretraining is a starting point, not evidence that a model recognizes your defects. Check whether its class definitions and visual examples correspond to the actual task. A detector trained on broad object categories may locate a component without identifying a subtle manufacturing defect.

Measure the errors that matter to the process. For example, 480 correctly detected targets from 500 labeled targets gives recall of 480 / 500 = 96%. That still leaves 20 missed targets. Separately, rejecting 10 of 500 known-good parts gives a false-reject rate of 10 / 500 = 2%.

Those figures describe different consequences. Do not combine them into a single accuracy percentage that obscures missed defects. Establish acceptance limits with the process owner before changing thresholds to improve a headline score.

Keep confidence thresholds and overlap-suppression settings fixed when comparing runtime configurations. If optimization changes predictions, re-evaluate the complete inspection decision—not just whether boxes appear similar.

Include images containing no target. The guidance on background images in object-detection training explains why negative examples matter. For acceptance testing, they also expose false detections that a target-only image set cannot reveal.

Choose acquisition and controller hardware from evidence

Select area scan cameras for field of view, shutter behavior, exposure, triggering and interface compatibility. Use the camera finder to narrow acquisition requirements before treating camera frame rate as a system-speed target.

The LS-CMR-0053 is an acquisition candidate to evaluate for a GigE installation.

The LS-CMR-0166 is an acquisition candidate to evaluate for a USB3 installation.

Both examples use rolling shutters. Validate image geometry on moving parts; they are not automatic selections for conveyor inspection. A global-shutter option may be more appropriate when motion distortion affects the target. More acquisition resolution also provides little benefit if deployment resizing removes the relevant detail.

For processing, compare vision controllers against measured runtime requirements, camera connectivity, memory use and control integration. Neither a memory specification nor a processor-family label establishes detector throughput.

The MV-VC2140-128G40-1T and MV-VC5525-256G70-NN are controller candidates for an interface and resource comparison. Confirm runtime compatibility and benchmark access before choosing either for a CPU-only detector.

All 2 share: Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

These catalog entries do not establish inference latency for your model. Check software licensing separately from hardware selection, including any application or deployment entitlement your chosen workflow needs.

When CPU-only inference is the wrong choice

CPU-only operation is attractive when the validated model meets the inspection deadline with useful headroom and leaves resources for acquisition, communication and recovery. It also avoids requiring an accelerator merely to execute a compatible model.

Choose a different approach when acceptable speed requires input reduction that hides required defects. The same applies when simultaneous camera streams cause contention, a queue grows at the required arrival rate, or the validated model depends on operations unavailable in the intended CPU runtime.

Before moving to acceleration, check whether the task needs a general object detector at all. Stable position, controlled lighting and a simple dimensional or presence criterion may suit a rule-based inspection. Conversely, a difficult visual classification problem may need task-specific training before any hardware change helps.

If the model works but CPU performance remains inadequate, compare an accelerator-supported deployment against the same images, thresholds and complete timing budget. A faster network does not repair poor acquisition or incorrect part tracking.

Acceptance checklist and next step

Before committing to hardware, confirm the following:

• Compatibility: the exact model executes correctly on the intended CPU backend.
• Sampling: the smallest required target remains visible after deployment preprocessing.
• Detection quality: missed-target and false-reject limits pass on held-out production images.
• Latency: measured decision timing meets the physical deadline under sustained load.
• Throughput: the incoming frame rate does not produce persistent queue growth.
• Traceability: each decision retains its frame and part identity.
• Failure behavior: late frames, missing images and communication failures have defined responses.
• Repeatability: model files, preprocessing, runtime settings and benchmark conditions are recorded.

Start the camera configuration workflow once acquisition requirements are clear. For help narrowing controller candidates, send your timing budget, model input shape, camera interface and benchmark measurements through application support. Those details support a hardware decision far better than an isolated inference-rate claim.