Your validation report says the custom model beats the baseline; on the line, its wrist keypoint lands two wrist-widths away while the baseline lands inside one. Before retraining anything, check one number: the image size the deployment call feeds the network. Pipelines routinely predict at imgsz=640 with a model trained at 1920, and every feature the network learned then shrinks to a third of its training size. This troubleshooting guide gives the symptom, the scale math behind a good mAP75 next to bad-looking pixels, a two-run test that settles it in minutes, and the fix that holds across training, validation and deployment.
Symptom: the metric and the picture disagree
On one custom pose model, validation against the pretrained baseline on the same test split looked convincing: the custom weights scored the higher mAP75. On single images the order flipped. The baseline put the wrist keypoint within about 10 px of the mark; the custom model sat about 40 px off — both runs at conf = 0.51. The split was small, 30 validation and 25 test frames, so the metric carried wide error bars, but that alone could not produce pictures this wrong.
Numbers fine, pixels bad is the signature of a pipeline difference between validation and deployment, not of a bad model. Metric-versus-intuition has its own article, Good val mAP but high YOLO loss values: why not to worry; here the difference had a name, and it was one parameter.
First suspect: predict does not inherit the training size
Training ran with imgsz=1920. The deployment call read model.predict(image_path, imgsz=640, conf=0.51) — 640 being the size quick tests default to, not the size the network was trained at. Nothing in that call looks up the training configuration: the network receives a letterboxed image at a third of the scale it expects.
Setting imgsz to 1920 — or removing the argument altogether — brought the visual results in line with the metrics in this case, and the 40 px wrist error collapsed to baseline-level accuracy. That is the whole fix in most cases shaped like this one. The dependable habit is to pass the number explicitly, read from the same configuration that recorded the training run, so nothing drifts between the bench and the line.
The scale math behind good mAP and bad pixels
Letterboxing preserves the aspect ratio, so the effective scale is close to input width divided by native width, and every feature scales by it. Validation never saw the problem because it ran at the training size; the deployment pipeline was the only one that ever executed at 640.
Worked example. Take a 12 MP USB3 camera with a 4000 × 3000 native frame — the LS-CMR-0166. Put 600 mm of field of view across that width:
- Pixels per millimetre: 4000 px / 600 mm = 6.7 px/mm.
- A 9 mm wrist spans 9 mm × 6.7 px/mm ≈ 60 px in the native frame.
- At the 1920 training input: 60 px × 0.48 ≈ 29 px wide. Comfortable.
- At a 640 inference input: 60 px × 0.16 ≈ 10 px wide. The whole wrist is ten pixels.
Read the errors at input scale. The 40 px deviation was measured in native pixels, so at the 640 input it is 40 px × 0.16 ≈ 6 px — against a 10 px feature, the predicted point lands over half a wrist-width away. The baseline's 10 px native error is 10 px × 0.16 ≈ 2 px, small against the same feature. Same network family, three times the feature scale.
To redo the first step with your own field of view and smallest feature, the resolution calculator returns the pixel budget:
Rule out the rest: causes in order of probability
When the sizes already match, or the fix only helps part of the way, work down this list.
| Rank | Cause | Test | Fix |
|---|---|---|---|
| 1 | Inference imgsz differs from training | Run the same frames through model.predict at both sizes and compare keypoint error | Pass the training size, or retrain at the deployable size |
| 2 | Preprocessing differs between validation and deployment: normalization, channel order, padding, resize | Run one frame through both pipelines and diff the arrays | Share one preprocessing function; OpenCV ONNX confidence lower than training: match the preprocessing shows the failure shape |
| 3 | Confidence threshold culls correct keypoints | Sweep conf from 0.1 to 0.6 and count detections per frame | Tune conf for recall, never for position |
| 4 | Split too small to trust | 30 validation and 25 test frames put wide error bars on mAP75; re-split and re-run model.val | Add varied frames before comparing models, per How many images do you need to train a custom YOLO dataset |
| 5 | Mixed training sizes or stretched aspect ratios | Check the resize behaviour in the training log | Pad, do not stretch; Mixed resolution images in object detection training: pad, not stretch has the method |
Rows 2 to 5 move metrics too. But when pictures look dramatically wrong while mAP looks right, row 1 is the first and usually the only answer.
The two-run test that settles it
Twenty frames and ten minutes settle the question.
- Read the imgsz recorded with the best training run, from the training arguments saved beside the weights.
- Run model.predict on the same 20 frames twice: once at the deployment size, once at the training size, with conf identical in both runs.
- Measure feature deviation in native pixels against the annotations, or simply overlay the two outputs.
- If error drops sharply at the training size, adopt that size; the mismatch was the cause.
- Re-run model.val on the test split at the deployment size and preprocessing, so the paper number and the line number describe one pipeline.
Expect the visual gap to close at step 4. The mAP75 recomputed at step 5 becomes the honest number to track from then on.
Size the input to the hardware, then train to match
Network cost scales with input area, not width: (640 / 1920)² = 1/9, so a 640 pipeline is about nine times cheaper per frame than a 1920 one. That is why deployments drift down to 640 quietly. The honest alternative is to choose the size the hardware sustains at line rate and train at exactly that size, so metric, picture and cycle budget all agree. Where acquisition or processing time binds first, Acquisition faster than processing: fix the frame rate gap and Full resolution image capture is slow or stalls: size the data first give the sizing method.
Downscale budget matters too. Taking a 4000 px native frame to a 1920 input is a 4000 / 1920 ≈ 2.1× width reduction; starting from a 20 MP class native frame is gentler, so small features keep more pixels at the input. The 20 MP mono USB3 option for that case:
- Sensor format
- 1"
- Lens mount
- C-mount
- Interface
- USB3
- Resolution
- 16-25 MP
Where the network runs matters as much as where the pixels come from. On a smart camera, the sensor's native frame is the only frame: train at what the device captures, because no PC step will rescue the detail afterwards. Where inference runs off-camera, a vision controller sized for the input class keeps 1920-class models inside the cycle:
- Interface
- GigE
- Resolution
- 6-10 MP
- LAN ports
- 3-4 LAN ports
- USB 3 ports
- 2 USB 3 ports
Among area scan cameras, match resolution to the feature budget first and imgsz second; that order keeps you from buying pixels the model never sees.
When the usual fixes are the wrong fix
- Lowering conf: the threshold decides which predictions are drawn, not where they sit. A wrist misplaced at conf = 0.51 stays misplaced at 0.2; you only add detections, including poor ones.
- More epochs or more frames: validation already said the model is fine. No amount of training repairs a deployment pipeline that feeds different pixels.
- Swapping in the pretrained baseline: it looks better only because its training size happened to match the deployment call, while on your parts it will miss what the custom model was trained to see.
- Raising imgsz past the training size: it cannot create detail the sensor never resolved, and the cost grows with input area.
Prevention: pin the size end to end
- Record imgsz beside the weights and have deployment read it from there; never let a predict call carry a hand-typed size that can drift from the training run.
- Share one preprocessing path — resize, letterbox, normalization, channel order — between validation and deployment, so the two pipelines cannot diverge.
- Export for a fixed runtime at the deployment input size; Non square YOLO training: image size, rect mode and ONNX export walks the size and shape decisions.
- Check pixels per feature at the training input before training: below about 20 px, tighten the optics or crop the region of interest rather than lowering the input size later.
- Keep 20 annotated frames as a visual regression set and re-measure keypoint error in px whenever camera, optics or field of view change — the scale moves even when imgsz does not.
Where to start
Start from the pixel budget, not from the model list: set your field of view and smallest feature, get the pixel budget from the resolution calculator above, then pick the camera and input size together in the camera finder or at the camera step of the configurator. If the size you need outruns the controller you have, send the cycle-time numbers through the contact page and we will size the controller and the training input together. More deployment-side checks live on the systems topic page.
Questions
Does the predict imgsz have to match the training size exactly?
Match it as closely as the pipeline allows. Predicting smaller than trained shrinks every feature by the size ratio: a model trained at 1920 and run at 640 sees features at one third scale, and box and keypoint quality drop with them. Predicting larger than trained cannot recover detail the sensor never captured. If the deployment budget forces a different size, fine-tune or retrain at that size, then re-run validation before trusting the numbers.
Why does validation mAP stay high if inference runs at the wrong size?
The val call sizes its input with its own imgsz argument, not with whatever the deployment passes to predict, so the metric describes a pipeline the line never executes. Re-run validation with the same imgsz and preprocessing as deployment. Once the two pipelines match, the mAP on paper and the predictions on your images move together, and future metric changes mean real changes.
Can I lower imgsz at inference to save processing time?
You can, but the price is accuracy. Input cost scales with area: dropping from 1920 to 640 cuts per-frame compute to about one ninth while shrinking every feature to one third. The cheaper honest options are cropping a region of interest so the input budget lands on the part, or retraining at the deployable size. Either way, re-validate at that exact size before shipping.
Does changing the confidence threshold move a keypoint's position?
No. The confidence filter decides which predictions are drawn, logged or counted; one that passes keeps the coordinates it already had. A misplaced keypoint is an input-size, preprocessing or data problem. Tune conf for the precision-recall trade-off the process needs, and hold it constant whenever you compare two models, or the comparison measures the threshold instead of the models.
How many pixels should the smallest feature span at model input?
Plan for tens of pixels, not single digits. In the worked example a 9 mm wrist spans about 60 px on a 4000 px native frame, about 29 px at a 1920 training input and about 10 px at 640, where localization visibly degrades. If your smallest feature lands below roughly 20 px at the training input, tighten the optics or crop before touching imgsz.