Machine vision deep learning setup: choosing CPU or GPU

A deep learning application that opens on a vision controller has passed its easiest test. The failures arrive later: the inference engine never installed, the license lacks the deep learning tier, or the model runs on the CPU while the schedule assumed GPU speed. This guide gives you the decision as numbers — a frame-budget rule for CPU versus GPU, a worked example at 20 parts per second, the checks that prove which processor executed the model, and a controller for each path. Work through it before the camera arrives, and every later fault has one layer to live in.

Decide CPU or GPU from the frame budget

The line rate fixes the budget before any hardware does. Convert it to milliseconds and compare it with the completed inference time of the model:

R is the inspection rate in parts per second, and every T term is a completed stage time in milliseconds, measured from image ready to decision out. The GPU path adds a fourth term, T_transfer, for the host-to-device copy of each image.

The rule: ship the CPU path when T_frame stays at or under 80% of T_interval. The missing 20% absorbs acquisition, logging and the tail of the latency distribution — the 95th percentile, not the median, sets the deadline on a real line. Treat 20% as the starting margin and widen it until the measured 95th percentile plus acquisition fits inside the interval on your own sample run.

And when the CPU fits, stop. A GPU adds a driver to the compatibility set, a transfer term to every frame and another license line, and buys nothing the margin did not already give you. Three things push you to the card anyway: the rate rises, the model grows (segmentation replacing classification, for instance), or a second camera joins the station.

Deep learning earns its keep on the parts rule-based tools miss — varied textures, random orientation, low-contrast surface defects. That strength is why the runtime stack, not the algorithm, is where these integrations fail.

The backend comparison stays fair because the calling contract is standardized. IEEE 3110-2025 sets technical requirements for the application programming interfaces of deep learning algorithms in computer vision; a component built to it exposes the same calls across releases and backends, so you swap the backend, rerun the same test and compare numbers.

Lock the supported software set before you download

Compatibility is a set of versions, not a list of individually recent parts. Before downloading anything, write the record:

• Application edition, release and patch level, plus the deep learning component that release uses.
• Operating system edition, version and architecture — x86-64 or ARM64.
• CPU instruction-set requirement from that release's release notes.
• GPU model, its device memory, and the driver version the release names.
• Runtime or acceleration libraries that release pairs with that driver.
• Model format, export options, input shape and numerical precision.
• The license tier that unlocks inference on the deployment account.

Fill every line from the release notes and install guide for your exact application release. A driver version quoted for a newer release breaks the set even when the installer completes.

An architecture mismatch stops a runtime library from loading while the operating system boots normally, which is why the record starts with the architecture. The ARM64 acquisition compatibility guide walks the same application-to-library matching for capture components.

Keep the execution runtime and the development toolkit separate. Some applications ship their own inference runtime; others call for a separately installed package. Install the compiler-sized toolkit only when the deployment procedure names it.

Install through the vendor path and check the license tier

Get installers, drivers, model packages and manuals from the manufacturer or its authorized distributors, and follow the install sequence for your release. A controlled order isolates failures:

1. Back up the application configuration and record the running versions; book a maintenance window if the host already serves production.
2. Check each installer's signature or published checksum before running it.
3. Confirm the license covers the deep learning features on the account that will run them.
4. Install the application, then its inference component, in the order the install guide gives; add drivers and external runtimes only where that procedure names them.
5. Set library paths or environment variables only where the install guide requires them.
6. Restart the application or service, then record the versions it actually loaded.

Do not switch off endpoint protection to push an installer through. Route the blocked package through your security process with its checksum attached.

If activation fails, read the vendor's license diagnostics and use its transfer or recovery procedure. The main window opening reports the base application; the license page lists the deep learning features separately. For the matching split between activation and acquisition symptoms, see the camera SDK licensing troubleshooting guide.

Prove the stack with a known image

Keep the camera out of the first inference test. Run the vendor's example model on the vendor's example image, then on a saved image from your own task. Record the input contract both times: pixel dimensions, channel order, pixel type, resize method, normalization, and any crop or padding. An image that looks right on screen still builds the wrong tensor when the contract is off.

Check the output against the model type:

• Classification: the label map matches the exported model's class order.
• Segmentation: output dimensions and class indices match the training project.
• Anomaly detection: separate the score map from the acceptance rule — the inference output interpretation guide covers that step.

If the application starts from a compressed file, decode it before treating the bytes as pixels; the image bytes versus pixel data explainer isolates that mistake. For a complete capture-to-validation workflow on a real classification job, follow the tire classification with deep learning procedure.

Passing here says the model runs correctly under the recorded conditions. Production variation is the next test, not this one.

Worked example: 20 parts per second on a Core i3 station

Take a cap inspection at 20 parts per second with a compact defect classifier, one image per part, batch size one. Say the measured CPU stages on an Intel Core i3 class controller are 6 ms preprocessing, 28 ms inference and 5 ms postprocessing:

Both lines are milliseconds, batch of one, one 2 MP image per part. The CPU path uses 39 of the 50 ms interval and leaves 11 ms — 22% of the interval, just above the 20% starting margin. Ship it on the CPU.

The line then speeds up to 30 parts per second:

The GPU line inserts T_transfer = 4 ms for the host-to-device copy and cuts inference from 28 ms to 15 ms. The 39 ms CPU path now misses the 33 ms interval by 6 ms. The GPU path completes in 30 ms but leaves only 3 ms — 9% of the interval. The card alone does not close the gap: trim the 5 ms postprocessing stage toward 2 ms or export a smaller model, then re-measure before committing to the accelerator.

Measure the way you will deploy. Run 20 warm-up iterations, then 1,000 measured iterations, and record the median, the 95th percentile, the maximum latency and memory use as your first acceptance evidence. Those are starting counts — extend the run to a full shift before the station goes live.

Throughput and decision latency are different numbers. Batching four images raises images per second and quadruples how long the first image waits for its answer; on a 50 ms interval, batch only when the parts arrive together.

Confirm which processor executed the model

Device enumeration is the first check. Query devices through the application's diagnostics, select the GPU explicitly, and switch automatic fallback off for the test so a silent CPU path cannot pass as a GPU result. The runtime's execution report then names the device that ran each stage.

Corroborate the report with device activity. A single 18 ms inference escapes a monitor that samples once per second; the 1,000-iteration loop at 30 ms per frame keeps the card busy for about 30 seconds and shows up clearly. Utilization alone is weaker evidence — display rendering and other processes create activity too — so require the report, the activity trace and the expected output together.

Run the final check under the production launch method. A desktop session and a background service differ in permissions, library paths and license access; a station that passes interactively and fails as a service differs in one of those three.

Compare CPU and GPU outputs with a tolerance fixed before the test. Different kernels and precision modes produce small differences, so a practical starting point is to accept score differences under 0.01 across a saved image set, then review by hand every part within 0.01 of the threshold — for a 0.5 threshold that is the 0.49 to 0.51 band. Bigger gaps mean preprocessing, label mapping or output interpretation, not precision.

Once the backend holds, tune the transfer path with the GPU tensor input integration guide and manage competing loads with the GPU contention guide.

Choose the controller for the path you picked

Match vision controllers to the decision you already made, then check each line of the software record against the vendor's compatibility table for your release.

The MV-VC3301H-128G66 is the station behind the worked example: an Intel Core i3-8100 with 8 GB of RAM, a 128 GB SSD, 5+ LAN ports, 3-4 USB 3 ports, 4 built-in light channels and PCIe expansion supported. The CPU path runs on the Core i3 today; the GPU card goes into the PCIe slot when the line reaches 30 parts per second, so one box covers both halves of the decision.

The MV-VC2140-128G40-1T is the lighter CPU station: an Intel N97 with 8 GB of RAM, a 128 GB SSD plus a 1 TB HDD, 3-4 LAN ports, 2 USB 3 ports and 4 built-in light channels. Pick it for slower lines — at half the example's rate the interval opens to 100 ms — and let the 1 TB drive hold the image archive for audits.

For the wider trade-offs across the controller range, read the standalone controller selection guide.

After the backend, the timing and the restart test pass, add acquisition with area scan cameras. The LS-CMR-0166 USB3 area scan camera joins the station at this stage. Narrow the imaging choice in the camera finder, assemble the path in the camera configuration step, and check the acquisition software with the camera integration options guide.

Diagnose one layer at a time

When a test fails, keep the error text and find the earliest failing stage. Change one layer, rerun the same test, write down the result, then move on.

If the vendor's example runs and your model fails, the fault is in the export — input shape, precision or an unsupported operation. If both fail loading the same library, the fault is the shared environment. Never change the application release, the driver and the model in one sitting; you lose the evidence that explains the recovery.

Readiness checklist before the camera goes live

• Supported set recorded: application release, OS and architecture, driver, runtime and model requirements agree on one page.
• Installers from the source: every package came from the manufacturer or an authorized distributor, signature or checksum checked.
• License tier confirmed: the deployment account runs the deep learning features, on the license page, after a restart.
• Known image passed: preprocessing and output interpretation match the reference, on the vendor example and on your saved image.
• Backend named: the execution report names the GPU for the sustained run, with fallback off during the test.
• Timing banked: median, 95th percentile, maximum latency and memory use over 1,000 iterations, plus startup time.
• Service launch tested: the station runs after a restart under the production account, not only in a desktop session.
• Recovery in place: the version record and the approved installers rebuild the station without borrowing libraries.

Then send the compatibility record, the model's input requirements and the measured timing budget to applications support before ordering the controller configuration — the hardware discussion starts from a number, not from a guess about processors.