ONNX classification label mismatch: your class index map is wrong

You deployed an exported classification model with ONNX Runtime, and the line disagrees with the trainer: the trainer calls the part matte gray at 0.9849 confidence, your deployment code calls it aviation silver at 0.9850. Before you touch interpolation flags, channel order or the FP16 export, compare numbers, not names. When both frameworks return the same probability vector and only the labels disagree, the model, the preprocessing and the export are all proven correct — the fault is in the class index map your deployment code uses to turn output positions into names. This guide gives the one-test diagnosis, a worked example with a six-class model, the fix (one generated class order file, never typed by hand) and a startup self-test that catches this bug class on every machine.

First test: compare sorted scores, not names

Run one image through both paths: the trainer's own predict and your deployed pipeline. Strip the names off, sort each probability vector from highest to lowest, and diff the numbers.

The 0.01 tolerance is generous on purpose. This export kept a float32 input feeding FP16 weights; the runtime's cast showed up as a 0.0001 difference on the top score, and quantization noise of that order never approaches the tolerance. If both checks pass, the graph, the resize, the crop, the channel order and the quantization are exonerated in one step. What remains is the code that runs after inference: mapping an integer index to a name.

That is the decision point of this whole failure mode. Matching scores with different names means a label index mismatch, and you stop debugging the model right there.

Why the labels swap: two class lists

Classification training data follows the folder-per-class convention: one subfolder per class, the subfolder name is the class name, and the loader assigns integer targets from its directory listing — sorted alphabetically in the common ImageFolder-style loaders. The network never sees a name; it learns a position in the output vector.

Ultralytics classification reads the names from the dataset folder list at training time; there is no names YAML of the kind its detection models use. Deployment code, meanwhile, usually maps the output through a hand-typed list. Two lists own one model, and any difference in their order — a folder renamed, a class added later, six names copied from a spreadsheet — permutes names against indices while every probability stays the same.

The trainer's own log gives the order away. In this failure it printed the ids behind its top five as [1, 4, 2, 5, 3]: the winning class sits at output index 1, the runner-up at index 4. Nobody guesses that order; you export it or read it.

Causes in order of likelihood

Rank the suspects by what they do to the numbers:

Row 2 earns a number, because it is the one people suspect first. Applying softmax to a vector that already went through one — six classes, top value 0.985 — returns exp(0.985) / (exp(0.985) + exp(0.015) + 4 × exp(0)) = 2.678 / 7.693 ≈ 0.35. The winner's score collapses by two thirds, which the first test catches instantly. And because softmax is monotonic, even then the winning index would not change.

Worked example: six finishes, two transposed names

A painted-finish classifier trained on six classes. The training run's folder order produced the first map, the deployment programmer typed the second list, and the ONNX session returned the vector in the third row:

The argmax of the probability row is 1, the runner-up is index 4 at 0.0150, and the trainer's logged ids confirm both positions.

Through the hand-typed list, index 1 reads aviation silver and index 4 reads matte gray, so deployment prints "aviation silver: 0.9850" and "matte gray: 0.0150". Through the trainer's map, the same vector prints "matte gray: 0.9849" and "aviation silver: 0.0150". Two transposed entries reproduce the entire symptom: identical values, opposite top-1 names, and a 0.0001 difference that is nothing but FP16 rounding. Case closed — label index mismatch, model innocent.

The fix: one class order file, generated and never typed

1. In the training environment, read the ordered class names off the trained model object — every framework exposes them — and serialize that exact list to classes.json. Do not sort it again; do not retype it.
2. Ship classes.json beside the .onnx file and version the pair as one unit. Model-file and label-file mismatches are their own deployment failure mode, covered in machine vision model file paths: fix deployment errors.
3. Load the file at startup and assert its length against the model's output width. A width mismatch catches added or removed classes before the first part arrives.
4. Regenerate the pair together whenever a class folder is added, removed or renamed.

import json
import numpy as np
import onnxruntime as ort

# classes.json is generated by the training run, never typed by hand
with open('classes.json', 'r', encoding='utf-8') as f:
    class_names: list[str] = json.load(f)

session: ort.InferenceSession = ort.InferenceSession(
    'best.onnx', providers=['CPUExecutionProvider']
)
input_name: str = session.get_inputs()[0].name

# image_batch: float32, shape (1, 3, 640, 640), RGB, scaled 0..1
probs: np.ndarray = session.run(None, {input_name: image_batch})[0]

assert probs.shape[1] == len(class_names), 'model width and class list disagree'

top_index: int = int(np.argmax(probs[0]))
print(f'{class_names[top_index]}: {probs[0][top_index]:.4f}')

With the trainer's list loaded, the same session, same weights and same input that printed "aviation silver: 0.9850" prints "matte gray: 0.9850". That is the test that proves the fix: nothing changes except the file.

Prove it at startup: golden images

Keep one reference image per class — selftest/matte_gray.png, selftest/aviation_silver.png, and so on — and run them through the exact deployed path at every startup:

from pathlib import Path
import numpy as np

# run_pipeline(image_path) is your full deployed path: preprocess, session.run, argmax
for image_path in sorted(Path('selftest').glob('*.png')):
    probs: np.ndarray = run_pipeline(image_path)          # shape (C,), float32
    assert class_names[int(np.argmax(probs))] == image_path.stem, image_path

Six classes, six asserts, run on the deployment machine itself. This catches a stale class list, a width change and a preprocessing regression in the same few seconds, wherever the code lands — and it is the only check that survives a rename-and-retrain.

When it is not the label map

If the sorted scores disagree beyond that tolerance, or the argmax indexes differ, you have a real inference mismatch and the label list is innocent. Work the three classics in this order:

1. Resize behavior. This pipeline trained with a short-side-640 bilinear resize with antialiasing, then a 640 × 640 center crop. OpenCV's resize has no antialias argument, so downscaled edges come out different pixel by pixel. The exact reconciliations — resize, interpolation, dtype — are laid out in OpenCV ONNX confidence lower than training: match the preprocessing.
2. Channel order. OpenCV loads BGR and the training transform expects RGB; color parts drift, gray parts barely move.
3. Normalization. Here mean was [0, 0, 0] with std [1, 1, 1] — no normalization at all. If your trainer used ImageNet constants and deployment skips them, scores move far beyond rounding.

Feeding the trainer's own preprocessed tensor into your deployment code settles all three at once: if the outputs then agree, the mismatch was upstream of the model, and the label map was never the problem.

Keep the class list in one toolchain

The root cause is structural: a training environment that owns one class list and a deployment program that owns a second, hand-made one. Smart cameras delete the second list, because the classifier and its class names live in a single on-device project — nothing is retyped between training and the line. The MV-SC5060XM-08M-WBN is an SC5000X series smart camera delivering 6 MP mono images through an 8 mm focal length, M12-mount lens on a GigE interface.

When the decision belongs on a vision controller — your own model on a fanless industrial PC with PLC I/O — the controller's software project is the one place the class list lives. The MV-VC3301H-128G66 is a VC3000 series controller with an Intel Core i3 processor, 8 GB of memory, 4 built-in light channels and a VM6200Pro license included. The trade-offs between learned and rule-based inspection on one device are covered in AI vs rule-based smart camera inspection: how to choose.

The area scan camera feeding either box only owes the classifier sharp, evenly exposed frames; the label bug lives entirely downstream of the sensor. The MV-CU020-90GC is a 2.5 MP global-shutter GigE camera with a C-mount and a 1/1.7-inch sensor.

Checklist: a classifier whose labels cannot drift

• One classes.json generated by the training run, stored beside the .onnx and versioned with it.
• No class-name strings anywhere in deployment code.
• Startup assert: the model's output width equals the length of the class list.
• Startup golden-image run: one image per class, predicted name must equal the file name.
• Every rejected image logs its argmax index and top-5 raw values, not just the name.
• Any change to the dataset folders regenerates the model and the class file together.
• On any doubt, compare sorted score vectors against the trainer before touching preprocessing.

If the station is still on paper, size it from the smart cameras and vision controllers above, or send the part list, cycle time and class count through the contact page and we will lay out the options with you.