Why the newer detector model is slower: comparing versions on CPU

You swapped the older nano detector for the newest release and the live camera loop fell from five detections a second to four. The newer model finds more — that is what the release is for — but on the same CPU it costs more milliseconds per frame, and a loop that must answer every second cannot donate them. This comparison is for anyone running detection from a camera on CPU-only hardware: a laptop, a fanless vision controller, or a smart camera with onboard compute. It shows where the extra time comes from, how to benchmark two detector versions fairly at one input size, and how to turn a millisecond budget into a decision you can defend on the floor.

The decision, up front

Release date is not a speed specification; work per frame is. The newer model does more of it. YOLOv8n loads with 3,007,208 parameters and 8.1 GFLOPs at its 640×640 reference size, against 1,769,989 parameters and 4.2 GFLOPs for YOLOv5n. Measured on the same laptop CPU, at the same 416×320 input, on the same video, the newer model averaged 268 ms per frame (3.7 fps) and the older 194 ms (5.1 fps). The newer one returned a detection on every logged frame; the older came back empty on roughly one frame in three at the same conf_thres=0.25.

That gives four defensible calls:

• Presence check near 5 Hz: keep the older, lighter model. 194 ms fits with margin.
• Misses cost more than latency: take the newer model and buy the time back by shrinking the input or moving the inference.
• You must answer at the camera's 30 fps: neither CPU nano model qualifies. Go to on-device or GPU-class inference, or stride frames and accept a 300 ms worst-case result age.
• The same model ships everywhere: match runtimes before comparing any milliseconds at all.

The third line is the working estimate used below: to first order, per-frame time scales with input area.

What the load summary tells you before you run

Both detectors print one line when the weights load — layers, parameters, gradients, GFLOPs — and that line already holds the comparison. One catch: the GFLOPs figure is computed at the default 640×640, while this loop fed letterboxed 416×320 frames. Scale by area: 416 × 320 ÷ (640 × 640) = 133,120 ÷ 409,600 = 0.33. About a third of the arithmetic survives — roughly 1.4 GFLOPs per frame for YOLOv5n and 2.6 for YOLOv8n.

The newer model is costlier at the same input because its anchor-free, decoupled head runs more layers per pixel — 168 against 157 — not because of the year in its name. The load summary tells you the newer version will not be faster here before a single frame is grabbed.

The measured comparison

Conditions, so the numbers can be reproduced: Python-3.8.7 on a laptop CPU with a torch-1.9.0+cpu build, a 640×480 USB camera at 30 fps, frames letterboxed to 416×320, conf_thres=0.25, iou_thres=0.45, means over the logged steady frames with warm-up discarded.

Three ratios fall out of that table: the newer model does 1.93× the arithmetic (8.1 ÷ 4.2), carries 1.70× the parameters (3,007,208 ÷ 1,769,989) and needs 1.38× the time (268 ÷ 194). A cloud GPU rerun of the same pair kept the order intact, so this is a property of the two architectures' sizes, not one laptop's quirk.

Published speed tables for these families come from export-optimized models on data-center GPUs, which is why they quote single-digit milliseconds. A 2025 IEEE review of YOLO benchmark studies lists each result with its FPS, its mAP and the GPU it ran on — speed numbers are hardware-tagged, and none of them transfers to your CPU without a rerun.

Worked example: a 30 fps camera into a 3.7 fps loop

Follow one frame. The camera delivers 640×480 at 30 fps, one frame every 1000 ÷ 30 = 33.3 ms. The detector letterboxes it: longest side to 416, so the scale is 416 ÷ 640 = 0.65; 640 × 0.65 = 416 wide and 480 × 0.65 = 312 high; 312 pads up to the next multiple of 32, giving a 416 × 320 input of 133,120 pixels — about 0.13 MP. Padding is where camera pixels and detection pixels stop matching, and it is worth understanding once: see object detector input dimensions, height, width and padding.

The newer model needs 268 ms for that input — 1000 ÷ 268 = 3.7 fps. The camera still produces 30. If the loop queues frames, the backlog grows by 30 − 3.7 = 26.3 frames every second, about 1,580 after a minute, and the "current" detection is a minute old. That failure is structural, not a tuning problem. The fix is to grab and process in separate loops and always take the newest frame: camera stuck at low fps: separate acquisition from processing.

Newest-frame-wins bounds the result age at processing time plus one frame period: 268 + 33 = 301 ms for the newer model, 194 + 33 = 227 ms for the older. If 300 ms is too old, buy time back by shrinking the input. Target 100 ms with the newer model: the needed area ratio is 100 ÷ 268 = 0.37, the linear scale is √0.37 = 0.61, and 416 × 0.61 = 253, which rounds to imgsz 256. Check it: a 4:3 frame letterboxes to 256 × 192, the area ratio is 256 × 192 ÷ 133,120 = 0.37, and 268 × 0.37 = 99 ms — about 10 fps. The formula gives the estimate; a run gives the number.

Why GFLOPs alone do not give you milliseconds

The newer model does 1.93× the arithmetic but takes only 1.38× the time. The gap is the mix: convolution bodies are compute-bound and scale with area, while the head, the pre- and post-processing and every weight fetch are memory-bound and track parameters — 1.70× here. CPU time lands between the two ratios, shifted by whichever bound owns your layers and thread count. Two rules survive that. Within one detector family, GFLOPs rank the versions correctly. Milliseconds come only from a run at your input size on your device — and the same weights in a different runtime are not the same milliseconds: see the same YOLO model giving different results per runtime and ONNX inference speed by runtime.

Benchmark two versions fairly

An afternoon of method settles what a week of speculation cannot:

1. Fix the input. Same imgsz and same letterbox for both versions; log the resolved shape — the per-frame line prints 320x416, and that is the network's tensor, not the camera's 640×480.
2. Fix the scene. A recorded video repeats; a live camera does not. If it must be live, lock exposure and lighting so detection difficulty does not drift between runs.
3. Fix the runtime. Same machine, same build, same thread count, all written down with the result — here Python-3.8.7 and torch-1.9.0+cpu.
4. Discard warm-up. CPU inferences start slow while caches, threads and memory pages settle; those first frames are not steady-state samples of anything, and counting them skews every average that follows.
5. Average 30 or more steady frames and report the worst one too. The machine meets the worst frame, not the mean.
6. Keep thresholds identical. conf_thres and iou_thres move both speed and the empty-result rate, because the detection count feeds NMS.
7. Score quality with speed. Log empty results and misses on the same scenes; a fast model that returns nothing one frame in three has not won.

Choose by your frame budget

Where the loop runs: vision controller or smart camera

A laptop is a bench, not a plant. The plant version of this CPU is a vision controller — a fanless box with camera ports and light-output channels that runs the loop beside the line. Sizing one for detection rather than for camera count is its own exercise: object detection on an industrial controller: sizing a CPU vision loop.

The MV-VC3301H-128G66 pairs an Intel Core i3 with 8 GB of memory, 4 built-in light channels and five or more LAN ports, so one enclosure carries camera, light and loop. The MV-VC5525-256G70-NN steps up to an Intel Core i5 with 32 GB, PoE camera ports and 8 built-in light channels. Either way, the 30-frame benchmark runs on that exact box before a rate is promised: a fanless controller holds lower sustained clocks than a laptop's burst, and detector time does not divide cleanly by core count.

All 2 share: Humidity 20% RH to 95% RH (no condensation); Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

The other shape is a smart camera: sensor and inference CPU in one housing, no frame traffic on a cable. The LS-SMC-0017 is an 8.8 MP smart camera on an x86 platform, with a global-shutter sensor and a C mount. The loop-rate arithmetic above applies unchanged — the onboard compute is the CPU in the equation.

On the camera side, match pixels to the detector's input. This loop's network consumed 416 × 320 ≈ 0.13 MP while the webcam delivered 640 × 480 ≈ 0.31 MP; the surplus is downscaled away before the first layer. The MV-CU020-90GC is a 2.5 MP global-shutter GigE camera, a natural feed for a loop like this. Even that oversamples the 0.13 MP input about nineteen times (2.5 ÷ 0.13), so extra camera resolution buys nothing for this detector — spend it on the lens and the light. For sensor choice by pixel count: area scan cameras.

When the usual answer is wrong

"Newer is faster" comes from tables measured on data-center GPUs; your CPU margin is yours to measure, and here the newer model kept a 74 ms penalty (268 − 194). "Fewer GFLOPs wins in milliseconds" ignores memory-bound layers: 1.93× compute became 1.38× time. "Same weights, same speed" fails across runtimes and export formats, which move latency by multiples. "Dropping frames loses data" is backwards for detection — a queued frame is an old frame, and acting on it is acting on the past. "The lighter model wins" forgets the last table row: its empty frames arrived one time in three at the same thresholds, and a presence check that blinks is not a presence check.

Checks before you commit

• Mean and worst frame time at the exact input size, from 30 or more steady frames with warm-up discarded.
• Identical conf_thres and iou_thres across versions, with the empty-result rate logged on your own scenes.
• The loop takes the newest frame and drops the rest; result age equals processing time plus one frame period.
• The budget covers preprocess and postprocess, not inference alone.
• Anything the detection gates on the machine runs on safety-rated hardware.
• A rerun whenever CPU, thread count, export format or input size changes — the number moves with all four.

Pick the version that fits the milliseconds you have, then pick hardware that holds those milliseconds on the floor. More loop plumbing sits under the systems topic, and questions with numbers in them go to the contact page.