Same YOLO model, different runtime results: what to match

You trained one YOLO model, validated it on the bench in OpenCV, then loaded the same .cfg and .weights into the production inference pipeline — and the detections changed. Boxes vanish, confidences sag, a bus comes back labelled as a truck. The weights are fine; the wrapper around them is not. Three settings differ almost every time this happens: the input channel order, the confidence and NMS thresholds, and the anchors hard-coded in the custom output parser. This guide gives you a probe for each, the arithmetic that shows why a single number moves whole objects in and out of the report, and an acceptance band for the small residual difference that is normal between two image-processing libraries.

The weights are the same; the wrapper is not

A trained detector is a pure function. A fixed-size input tensor goes in, raw output tensors come out, and nothing inside the file knows whether channel 2 is red or blue. Everything else — video decode, resize, channel order, normalization, thresholding, NMS and box decoding — is implemented separately by every runtime that hosts the model. Two runtimes therefore mean two independent implementations of the code that feeds and reads your weights.

Moving between runtimes is routine, usually for speed, and exporting to ONNX does not make results portable: the file carries the network graph, while channel order, scaling and thresholds stay the runtime's job. The trade-offs of picking a runtime when ONNX inference runs slower in OpenCV DNN are a separate decision; here the goal is making two runtimes agree.

Fix the settings in order of how often each one is the culprit: channel order first, thresholds second, parser anchors third, input geometry and size gates last. The pattern — identical data, different result because a presentation setting changed — reaches beyond vision: two HMIs displaying different valve status colors from the same tag are the same class of bug in a different cabinet.

First suspect: BGR versus RGB at the network input

OpenCV delivers frames as BGR: cv2.imread and VideoCapture both hand your code blue-green-red. Deployment sample pipelines commonly default to RGB through a color-format key in the detector config — in the widely circulated samples the key is model-color-format, where 0 leaves RGB and 1 selects BGR. Train through OpenCV, deploy through an RGB default, and every frame reaches the network with channels swapped: scores drop unevenly, some classes fade, and the model performs as if you had shipped a worse one. Setting model-color-format=1 to match a BGR-trained model typically closes most of the gap in one line.

Do not assume the direction, though. Some recent YOLO stacks deliberately accept BGR arrays and convert internally, because they build on OpenCV conventions; others expect RGB. The correct order depends on the wrapper that produced the weights, so probe instead of guessing:

import numpy as np

w, h = 416, 416                        # use your cfg width/height here
probe = np.zeros((h, w, 3), np.uint8)  # all-black uint8 test image
probe[:, :, 2] = 255                   # BGR order: channel 2 is red

# Feed `probe` to both runtimes and print the first input pixel each reports:
# [0, 0, 255] -> that runtime kept BGR (OpenCV order)
# [255, 0, 0] -> that runtime converted to RGB

Match the pipeline to the training order: set the color-format key, or convert at the source with cv2.cvtColor(frame, cv2.COLOR_BGR2RGB). For clips recorded off a camera, run the same audit on the decode step — converting recorded camera frames between BGR and RGB for OpenCV covers that path.

Second suspect: confidence and NMS thresholds in two places

The OpenCV side usually states its gates inline: cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.3) applies a 0.5 confidence gate and a 0.3 NMS IoU gate. The deployment side hides its gates inside the custom parser as constants — kPROB_THRESH and kNMS_THRESH, plus a box-count constant such as kNUM_BBOXES = 3. Sample parsers circulate with their authors' values; one widely copied parser carries kPROB_THRESH 0.7 and kNMS_THRESH 0.5. Those numbers silently override whatever the training config implies, and nothing at startup prints them.

The gate that decides duplicate boxes is the IoU threshold:

The symptoms now map directly. Objects missing entirely point to a probability gate set above your training-time value; the same object reported twice points to an NMS gate raised above the overlap your boxes actually show. Match the values and the order of operations, because some parsers gate before NMS and others after, some per class and some globally. For the small-object end of this, confidence and IoU thresholds for small objects in tiled inference works through the trade-off.

Third suspect: anchors and masks inside the custom parser

During training, each [yolo] layer in the cfg carries an anchors= line and a mask= line naming the anchor subset that scale uses. A deployment runtime never reads those decode rules; the parser function you supply decides which output tensor maps to which anchor set and how raw values become boxes. Copy a parser written for the standard three-scale, three-anchor layout and deploy a custom tiny model with re-clustered anchors, and the decode scrambles: boxes appear at wrong scales, class scores attach to the wrong anchors, and results come out worse than either runtime alone.

The test is arithmetic, not vision. Print each output layer's name and shape, read the anchors= and mask= lines from the cfg, and compare them against the parser's constants: the box count per scale (kNUM_BBOXES = 3 in the samples) must equal the anchors behind each mask, and the scale order must match. The fix is to pass the cfg's anchors and masks into the parser, or to generate the parser's constants from the cfg at build time so the two files cannot drift apart.

Quieter mismatches: input size, scaling and size gates

Three smaller settings produce the same symptom with less noise. First, network input size: the width and height in the cfg (416×416 is typical for tiny variants) must equal what the pipeline requests, because an engine cache built at a previous size resamples every frame without a word — fixing an inference image size mismatch in YOLO walks that check. Second, the scaling scheme: stretching the frame to network size and letterboxing it with padding move objects differently, and thin or small parts lose the most.

Third, size gates. Pipeline configs carry keys such as detected-min-w, detected-min-h, detected-max-w and detected-max-h, which delete boxes by size after inference. A nonzero minimum removes every small part from that runtime's output only, which reads as a worse model rather than a filter.

The agreement test: still frames, dumped boxes, a diff

Freeze the test material. Pick 10 still frames that cover your part variants and edge cases, feed the identical files to both runtimes, and dump class, box coordinates and confidence as text from each. Sort both dumps and diff them. Still frames remove the video decoder from the equation; text dumps remove eyeball judgment from overlay images.

Then hold both runtimes to this acceptance band: the same object count per frame, matched boxes with IoU of at least 0.9, and confidences within ±0.03. The basis is that different resize and color-conversion kernels move scores by hundredths, which is the residual two image-processing libraries will always show — a 1:1 float match across them is not a realistic target, agreement on detections is. If confidences come back uniformly lower than training, the audit has caught preprocessing drift instead, and matching the preprocessing when OpenCV ONNX confidence runs lower than training is the same procedure applied at the training boundary.

Still different? Weights, caches and versions

If channel order, thresholds, anchors and input geometry all match and the dumps still disagree, work down this list. Checksum the weights file on both ends: a stale, retrained-over copy still loads without complaint, and md5sum on both sides settles it in seconds. Check the engine cache: pipelines that serialize the network on first run keep serving an engine built from the previous model until the cache is deleted. Check the decode path: if stills agree but video differs, the two stacks are decoding or color-converting the recorded stream differently.

Version drift sits on the same list. A pipeline or SDK update can change preprocessing defaults, so re-run the golden set after every update; the SDK and firmware version matching needed when deploying custom code to a smart camera is the same discipline one size smaller.

Where to run the comparison: one machine, one frame source

Pin both runtimes to one controller so decode, storage and file paths are identical, and run the comparison from recorded files rather than the live feed while you align settings. The MV-VC3301H-128G66 combines an Intel Core i3 processor, 8 GB of memory, 5+ LAN ports and 4 built-in light channels. That is enough machine to host the bench script and the deployment pipeline side by side.

The vision controllers range covers the fanless boxes built for this dual role, and the same one-machine discipline holds when the target is a smart camera: the on-device runtime replaces the pipeline, and the audit is identical, only smaller.

Keep it from coming back

Each of these mismatches takes minutes to prevent and hours to debug on a line that is already running:

1. Write the training channel order into a one-line file next to the cfg — BGR if your loader went through OpenCV — so deployment becomes a config check instead of archaeology.
2. Keep confidence and NMS thresholds in one config file read at startup, versioned with the weights, never as constants inside the parser.
3. Generate parser anchors and masks from the cfg at build time instead of copying floats by hand between files.
4. Keep a golden set of 10 still frames with expected detections and re-run it after every model, parser or pipeline change.
5. Checksum the weights file at every copy; md5sum on both ends takes seconds and catches the stale retrain.
6. Set size gates on purpose rather than by sample default, and record why each nonzero gate exists.

When a mismatch survives everything above, send both configs, the parser and one golden frame through the contact page and we will work through it with you. The systems topic hub collects the neighboring guides on runtimes, thresholds and deployment.