ONNX classification confidence drop: fix the preprocessing mismatch

You trained a classifier in PyTorch, exported it to ONNX, and the deployed copy now disagrees with the original on half your images. The export itself is almost never at fault: the graph and the weights arrive intact, and the same float32 math runs on the other side. What breaks is the code between the image file and the input tensor. This guide is for engineers putting classification models onto factory inspection stations. It shows you how to read the exported model's input contract, reproduce the training preprocessing exactly, name the broken step from the score pattern alone, and prove parity with a one-image tensor diff before you trust a hundred-image agreement run.

Why does an exported ONNX classifier score so much lower than the model it came from?

Because the deployed code feeds the network different pixels, not because the export lost accuracy: an ONNX export copies weights and graph structure verbatim, so a collapse to agreement on 46 of 100 images points at preprocessing, the only stage your deployment code owns.

The export writes the same convolution weights, strides, padding and class order into the file. Two runtimes can differ in the last decimal of a float32 sum, never by whole percentage points. So list the stages between file and tensor — decode, resize, scale, channel order, layout, dtype — and expect exactly one of them to differ from what training used.

The symptom has a shape. Softmax scores stay confident, often 0.9 and above, but attach to the wrong classes, and the flips concentrate where the model separates classes by color. If the parent pipeline still reaches the accuracy you trained for, the network is fine and the feeding code is not. Detection models show the same mismatch as low confidence instead of wrong labels; the OpenCV ONNX confidence lower than training: match the preprocessing guide walks through that variant.

Which four preprocessing settings must match after an ONNX export?

Four: the input geometry together with its resize filter, the pixel scaling, the channel order and the tensor layout, and each must equal the training pipeline's choice exactly, because the exported graph cannot know or correct what you feed it.

A 224 × 224 RGB model consumes 1 × 3 × 224 × 224 = 150,528 float32 values per image, since 224 × 224 = 50,176 pixels per channel and there are 3 channels. Layout errors announce themselves as shape exceptions; channel-order errors stay silent, which is what makes them expensive. A grayscale camera adds its own twist, and the grayscale image into a 3-channel model input guide covers that conversion. If the training images were non-square and the export folded them into a square, the non-square YOLO training and ONNX export article explains what happened to the geometry.

How do I read the input contract out of the exported model file?

Ask the runtime session: its input descriptor carries the tensor name, the expected shape and the element type, and for a fixed-size export the shape comes back fully numeric, leaving nothing to guess.

import onnxruntime as ort  # documented inference package

session = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
inp: ort.NodeArg = session.get_inputs()[0]
out: ort.NodeArg = session.get_outputs()[0]

print(inp.shape, inp.type)  # [1, 3, 224, 224] tensor(float) for this export
print(out.shape)            # e.g. [1, 1000]: one score per class

# Test before building any tensor: a fully numeric 4-axis contract
assert inp.shape == [1, 3, 224, 224] and inp.type == "tensor(float)"

Every value in that printout is a decision: batch of 1, three channels, 224 × 224, float32. The output shape gives the class count, and the index order was frozen at export; the class index map article covers the scramble that follows when you rebuild that order by hand instead of reading the export metadata.

What does each mismatch look like in the scores?

The failure pattern names the wrong step: a shape error stops the run outright, a scaling error flattens or saturates the scores, and a channel-order error keeps the scores confident while moving them onto color-neighboring classes.

• Shape. The run call raises and prints the expected [1, 3, 224, 224] against what you passed. Transpose HWC to CHW and prepend the batch axis.
• Scaling by 255. Eight-bit values fed into an input that expects 0–1 arrive 255 times too large: mid-gray 0.5 shows up as 128. The first convolution saturates and the softmax collapses toward one arbitrary class.
• Wrong normalization statistics. If training used ImageNet mean and standard deviation, a white pixel's red channel is (1.0 − 0.485) / 0.229 = 2.25 after normalization. Feeding the raw 1.0 leaves it 1.25 low, and a black pixel sits at 0.0 instead of (0.0 − 0.485) / 0.229 = −2.12, so contrast is compressed in both directions.
• Channel order. A red part with RGB (200, 30, 40) fed to a BGR-trained model shows the first convolution 200 where training had 40, a difference of 160 / 255 = 0.63 in normalized units. The overall mean stays (200 + 30 + 40) / 3 = 90 either way, which is why whole-image statistics never catch this bug.
• Resize filter. Nearest-neighbor resizing from 1280 × 960 to 224 × 224 samples every 5.7th column and 4.3rd row (1280 / 224 = 5.7, 960 / 224 = 4.3), so a one-pixel scratch can land between samples and vanish from the tensor entirely.

How do I reproduce the training preprocessing step by step?

Diff one tensor rather than one score: run a single validation image through the parent pipeline, capture the tensor it feeds the network, and compare it element-wise against your deployed tensor, because the size of that difference names the broken step.

1. Print the input contract from the session (shape, dtype) and keep it beside your preprocessing code.
2. Capture one parent tensor: hook the export tool's predict path on a single validation image and save the array after preprocessing, before the network.
3. Build the same image through your deployment code, on the 0–1 float scale, and compute the per-channel maximum absolute difference.
4. Read the magnitude. A uniform factor of 255 anywhere is scaling. Differences that swap between channels are channel order. Scattered values of a few thousandths on 0–1 floats are the resize filter, as a rule of thumb. Zero means the pipelines match.
5. Fix that one step, re-run the diff back to zero, then re-run the hundred-image agreement test to confirm.

What changed in the worked example, number by number?

One 224 × 224 RGB classifier, one hundred validation images, figures illustrative and arithmetic exact: before the fix the deployed pipeline agreed with the parent on 46 of 100 top-1 picks, and after matching the resize, scaling and channel order it agreed on 100 of 100.

The model and the test. The export fixed the input at 1 × 3 × 224 × 224, so 150,528 float32 values per image, with one output score per class. Both models saw the same 100 JPEGs, and the check counted agreeing top-1 picks.

The mismatches found. The deployment script kept RGB where the training pipeline had converted to BGR, and it resized with the imaging library's bicubic default where training used OpenCV's bilinear filter. Scaling was already correct at 1/255.

The arithmetic that named them. For a part with RGB (200, 30, 40), the first convolution saw 200 where training saw 40: |200 − 40| / 255 = 160 / 255 = 0.63 normalized units too high. On a 1280 × 960 image, bicubic-versus-bilinear scatter of a few thousandths per pixel blurred one-pixel features. Scaling, checked the same way, differed by exactly 0.

The fix and the re-test. Convert to BGR, resize bilinear, keep 1/255, transpose to 1 × 3 × 224 × 224. The tensor diff returned 0.000 within float32 rounding, the agreement run went from 46/100 to 100/100, and the largest softmax gap between runtimes was 0.008 — summation order, not pixels. That residual belongs to the runtime, which the same YOLO model, different runtime results guide explains.

How do I keep training and deployment preprocessing in one place?

Write preprocessing once and import it from both sides: the export-time predict path and the deployment script call the same function with library versions pinned, so a changed filter default cannot drift between bench and line.

Concretely: keep a single preprocess(image) function in the training repository and have the deployment code import it, rather than re-typing the steps. Pin the imaging and runtime libraries to the versions you validated with; imaging libraries have changed their default resize filter across major versions, which silently changes your interpolation. Record the contract beside the model file in plain text: input 224 × 224, filter bilinear, scale 1/255, channels BGR, layout NCHW. An engineer joining the project six months later should find the contract, not archaeology.

When is the gap not preprocessing?

Only once the tensor diff reads zero: with bit-identical inputs and top-1 still flipping, the remaining causes are precision and metadata — an FP16 or INT8 export, an op-set mismatch between runtime versions, or a class-index map rebuilt in the wrong order.

FP16 conversion shifts scores in the third decimal and rarely moves top-1. INT8 quantization shifts them further and can flip borderline images, so treat any precision change as a reason to re-run the agreement test. Op-set differences between the exporting and running library versions show up as small systematic shifts; pin both ends. The class-index order is frozen at export, and a rebuild by alphabetical order produces confident, consistently wrong labels — a signature distinct from preprocessing noise, because the wrong answer is stable across lighting and viewing angle. The systems topic page collects the runtime-side guides once preprocessing is clean.

Where does the corrected pipeline run on a factory line?

Anywhere an x86 Python runtime lives, which on a line usually means a vision controller mounted next to the cell: the corrected script runs unchanged from bench to controller, and the same box drives the cameras and the lights through its own ports.

Keep validation and production on one code path. A controller sized like a small PC, such as the MV-VC5525-256G70-NN with 32 GB of memory and a 256 GB SSD, runs the validated onnxruntime session directly, and its 5+ LAN ports and 8 built-in light channels leave headroom for a second camera or a second light stage later.

If inspection belongs on the camera itself, browse the smart cameras category; the PC-class boxes live under vision controllers, and the camera finder filters by interface, resolution and shutter. Run the tensor diff first, though: hardware never fixes a feeding bug, and the contact page reaches the applications team when the contract itself looks wrong.