OpenCV ONNX confidence lower than training: match the preprocessing

You trained a small classifier, exported it to ONNX, and the image that scored 0.77 from the .pt comes back at 0.40 in OpenCV. The export did not damage the model: converting to ONNX translates weights and graph structure without retraining, and the same input tensor produces the same score through any runtime. What differs is the tensor itself — the training pipeline and your cv2.dnn code are not feeding the network the same pixels. This page names the five settings that drift, gives the size rule that costs the most, works the arithmetic on a real 0.77-to-0.50 case, and ends with a three-pipeline test that proves your runtime input matches training before you set any threshold on the line.

Why is the ONNX confidence in OpenCV lower than the training result?

Preprocessing is a practical first suspect: a different input tensor changes the logits and can change both confidence and the winning class. A score difference alone cannot establish that the export or runtime is correct.

Use the following reported case as a diagnostic example, not an accuracy guarantee for other models. Predicted from the .pt, the image gave neutral 0.77 with sad at 0.12 at shape (1, 3, 64, 64); the same model exported at 48×48 and predicted at 48×48 gave neutral 0.50 with sad at 0.32. The winner never changed, only the margin collapsed.

That behavior is consistent with a preprocessing mismatch. The network still recognises the part, but the tensor it receives is smaller, softer or differently scaled than anything it was validated on, so the logits spread out and softmax returns weaker peaks. Make the blob passed to net.setInput() carry the reference tensor, then test whether OpenCV evaluates the export within the release tolerance.

Which preprocessing mismatch costs the most?

Input size mismatch costs the most: in the case behind these numbers, running a 64×64 export at 48×48 alone pushed the top score from 0.77 down to 0.50 while the runner-up climbed from 0.12 to 0.32.

Use this checklist to isolate one change at a time; its order is a debugging sequence, not a ranking of effect size across models:

1. Input size. A fixed-shape export requires its declared dimensions. Dynamic-shape exports accept a range, but the comparison still needs the same dimensions and geometry as the reference. Test: print the model's input dimensions and compare them with your blob size.
2. Resize geometry. blobFromImage with crop=False stretches the image to the target size and distorts the aspect ratio, while training pipelines resize the short side and center-crop for classification or letterbox-pad for detection. Test: score a pre-squared image, where stretching and cropping coincide.
3. Channel order. OpenCV imread normally returns BGR; set swapRB=True if the reference tensor expects RGB. Test: score a channel-swapped copy; if the score jumps, the flag was wrong.
4. Scale. 1/255 maps 0–255 pixel values onto the 0–1 range the training data used. Test: print blob.min() and blob.max(); for a plain 0?1 pipeline, values above 1 flag a mismatch; mean/std normalization can legitimately produce values outside that range.
5. Mean subtraction. (0, 0, 0) is right when training normalised with mean 0 only; a model trained with a mean vector needs that vector here.

Several mismatches can interact. The remaining score near 0.40 in OpenCV still needs an input comparison before assigning its cause.

What image size should training, export and the runtime share?

One size end to end: read the input shape the .pt prediction prints, export the ONNX at that size and build the blob at that size, to reproduce the reference test. For a dynamic-shape model, validate each intended deployment shape separately.

The trap is that predict on the .pt does not always run at the size in the configuration. The reference run asked for 48 pixels and executed at 64×64, because the network downsamples by 32 and wants input dimensions in multiples of 32, and it printed what it really used: shape (1, 3, 64, 64). Exporting with imgsz=48 froze the ONNX at 48×48, a size the reference run never touched. One number broke the match, and exporting again with imgsz=64 restored it: the ONNX then returned the same scores as the .pt.

What does each blobFromImage argument actually do?

Five arguments control this simple blob construction: the scale factor, the output size, the mean subtraction, the channel swap and the crop flag, and their values come from the reference preprocessing pipeline. Per-channel standard deviation normalization or custom padding may require explicit preprocessing outside this call.

The call in the failing code read cv2.dnn.blobFromImage(image, 1.0/255, (64, 64), (0, 0, 0), swapRB=True, crop=False) — scale, mean and swap were right, the size had to follow the re-export, and geometry remained the open question. Use this table as the checklist:

For the classifier example, confidence is a softmax score: a relative measure across the trained classes, not a probability that the answer is correct. It only means something once the input matches training, which is also why the same value cannot be compared across models.

How do you prove the runtime input matches training?

Run the same image through three pipelines: reference framework, exported ONNX in a reference runtime, then OpenCV. Capture the actual tensors immediately before inference, after all transforms, and compare those before interpreting the output differences.

1. Predict from the .pt on a fixed image and record the actual input shape, dtype, class order and whole output vector.
2. Export at the intended shape (64?64 in this case). Compare reference and ONNX outputs using the same tensor and a tolerance established for this model and precision. If they differ, inspect export operators, evaluation mode and output processing.
3. Run the cv2.dnn code on the same image and capture the blob passed to net.setInput(). Record the complete output vector rather than only the argmax.
4. Check shape and dtype first. Minimum, maximum and mean are quick scale checks, but swapping RGB channels or shuffling pixels can leave all three statistics unchanged.
5. Convert both captured inputs to the same declared layout, such as NCHW, without changing channel meaning, and compare every element. Do not reshape a wrongly ordered tensor merely to make its dimensions match.
6. Change one preprocessing setting at a time until the input comparison passes. If equal inputs still produce different outputs, investigate the exporter, supported operators, backend, precision and post-processing. Do not keep adjusting the input to hide an output defect.

import numpy as np

def assert_same_input(reference, candidate, atol=1e-6):
    reference = np.asarray(reference)
    candidate = np.asarray(candidate)
    assert reference.shape == candidate.shape, "input shapes differ"
    assert reference.dtype == candidate.dtype, "input dtypes differ"
    assert np.isfinite(reference).all() and np.isfinite(candidate).all(), "non-finite input"
    np.testing.assert_allclose(candidate, reference, rtol=0, atol=atol)

Call assert_same_input(reference_tensor, opencv_blob) on the captured arrays. The absolute tolerance 1e-6 is a starting check for this float32, 0?1 preprocessing comparison, not an output-score acceptance limit. If it fails, report the largest absolute difference and its index, then inspect that pixel/channel in both pipelines. Use exact equality when the preprocessing is expected to be byte-identical.

For tracing transforms before the final blob, the guide on color normalization through the training pipeline explains where to capture intermediate arrays.

Once the vector matches, the same discipline carries downstream, where model output becomes accept and reject decisions; the ONNX inference walkthrough from maps to pass/fail decisions covers that mapping for map-style outputs.

Where should the corrected model run in production?

A classifier of this size is light — 5,084,167 parameters and 12.5 GFLOPs at 64×64 — so the host choice follows the camera count and the cycle time, not the model: one camera runs on a compact controller, a multi-camera stack wants more ports and memory.

The reference measured 2.8 ms of inference on a desktop GPU; the same 12.5 GFLOPs typically costs tens of milliseconds on a Core-class CPU, which fits most per-part cycle times, and it is worth timing once on the target with the golden image set.

For a single camera feeding one classifier, a compact controller keeps the cell on one device: the MV-VC3301H-128G66 carries an Intel Core i3, 8 GB of memory, 5+ LAN ports and 4 built-in light channels.

Multi-camera cells want headroom for several GigE streams plus the model: the MV-VC5525-256G70-NN adds PoE camera ports, 32 GB of memory, 8 built-in light channels and 5+ LAN ports.

Both parts come from the vision controllers category, which lists the full range of camera-port counts and memory sizes. Where the model must run on the camera itself, a smart camera carries inference on the device, and every preprocessing rule in this guide applies to its input pipeline exactly as it does to a controller.

How do you keep the score from drifting after the fix?

Pin the numbers in one place: keep the export size, the blob arguments and the class order in a single configuration the deploy step reads, and re-run the three-pipeline test on a fixed image set every time the model or the OpenCV version changes.

• Export at the size the .pt prediction prints, and record that number next to the file name in the release note.
• Generate the blob arguments from the same configuration the export used, instead of retyping scale, size, mean, swapRB and crop in each runtime.
• Keep a golden set of 20 production images with reference scores, and fail the deployment if any score moves by more than ±0.02; float noise between CPU and GPU sits below that, a preprocessing regression does not.
• Set the pass/fail confidence threshold after preprocessing matches, measured on the deployment hardware; the vision model task configuration guide covers that step.

The usual answer — retrain with more data — is wrong here. The model already reproduces its training behaviour through the correct pipeline, so no amount of retraining fixes a runtime that feeds different pixels. Fix the preprocessing first, then judge the model on real parts. When a retrained model does replace this one, the checklist in swapping a neural network model on edge vision hardware keeps the sizes and arguments pinned through the changeover.

Bring the score logs from your three-pipeline test to the contact page if the gap survives every check, and browse the Learn guides hub for more runtime checks like this.