You trained a detector in Python, exported it to ONNX, and on the line the boxes sit too high or too low while confidence drops on parts the model found easily during validation. In most cases the weights are fine - the deployed resize is not the resize the model was trained with. This article is for engineers moving YOLO-style detectors onto a controller or smart camera. You will compute the per-axis scale factors, decode boxes correctly under letterbox and under direct stretch, run a five-minute synthetic test that proves the pipeline matches, and decide when a stretch resize is the right choice.
The rule: preprocessing is part of the model
A detector never sees the camera frame. It sees the resized, padded grid you feed the runtime, and during training it learned object sizes and positions on exactly such a grid. Change the grid geometry between training and deployment and you ask the trained weights about shapes they have never seen: the same part at different proportions, so lower confidence and displaced boxes.
That leaves exactly two working combinations. Train letterbox, deploy letterbox. Train with direct stretch, deploy the same stretch. Every mixed pair fails, and no export flag or post-processing trick repairs it, because the mismatch sits upstream of the first convolution. A pure confidence drop with no visible image change is the same mechanism - see OpenCV ONNX confidence lower than training: match the preprocessing.
What letterbox does to a frame
Letterbox scales the whole frame by one factor s, the smaller of the two axis scales, and centres the scaled frame on the input grid; the leftover bands get a neutral gray, value 114 in YOLO-style pipelines. That pad value is a deliberate choice, not a placeholder - Letterbox padding value 114: why neutral gray beats black borders explains why black borders hurt. One factor for both axes is the property that matters downstream: the decode needs only s and the two pads, it is exact, and every object keeps its labeled aspect ratio on the way in.
The cost is resolution spent on padding. On a 1920 × 1080 frame, s = 640/1920 = 0.3333, the content covers 640 × 360, and 280 of the 640 rows are gray - YOLO letterbox padding: what happens to a 16:9 image walks through that case row by row.
What direct stretch does to geometry
Direct stretch resizes each axis to fill the input completely: no padding, but the two factors differ unless the source and the input share an aspect ratio. A 4:3 frame in a square input lands as a 1:1 image - circles become ellipses and every box is 33% taller relative to its width than it was on the camera. That is harmless if the network trained on exactly those distorted images; it learned distorted shapes. It breaks the moment anything in the pipeline assumes isotropic scaling: a decode with a single factor, a minimum-size filter tuned on training-time box areas, or an aspect-ratio gate in post-processing.
Per-axis decode: what a stretch wrapper must compute
A wrapper that keeps a single scale factor - correct only while factor_w equals factor_h - displaces every box by (factor_w / factor_h - 1) times its distance from the frame origin: 33% on the 4:3 case below. Keep the two factors as separate, declared values and the arithmetic stays honest:
# illustrative decode for a stretch pipeline; sizes declared once, at wrapper start
input_w, input_h = 640, 640 # model input size, int
w, h = 4000, 3000 # source frame size, int
factor_w = w / input_w # 6.25, float
factor_h = h / input_h # 4.6875, float
x1 = x_model * factor_w # model box corners in, pixels out, float
y1 = y_model * factor_h
x2 = x2_model * factor_w
y2 = y2_model * factor_h
The box corner variables are illustrative; keep whatever names your post-processing already uses, and never apply factor_w to a y coordinate.
Worked example: 4000 × 3000 into 640 × 640
Take a 12 MP camera frame of 4000 × 3000, a 640 × 640 network input, and one part: box centre (2000, 1000), size 800 × 600.
Letterbox: s = min(640/4000, 640/3000) = 0.16, content covers 640 × 480, pad_y totals 160, so 80 top and 80 bottom. Model coordinates: x = 2000 × 0.16 = 320, y = 1000 × 0.16 + 80 = 240, size 128 × 96. Decoding back: x = 320/0.16 = 2000, y = (240 - 80)/0.16 = 1000 - exact.
Stretch: factor_w = 4000/640 = 6.25, factor_h = 3000/640 = 4.6875. Model coordinates: x = 2000/6.25 = 320, y = 1000/4.6875 = 213.3, size 128 × 128 - the same part, one third taller.
Decode the stretched box (320, 213.3) under three assumptions:
- single factor 6.25 on both axes: y = 213.3 × 6.25 = 1333, which is 333 px below the true 1000;
- letterbox decode with pad 80 subtracted: y = (213.3 - 80) × 6.25 = 833, which is 167 px above the true 1000;
- per-axis factors: y = 213.3 × 4.6875 = 1000, width = 128 × 6.25 = 800, height = 128 × 4.6875 = 600 - exact.
The direction of the error names the wrong assumption: pad subtraction on an unpadded image pushes boxes up; one factor on two different scales pushes them down. And note what decode cannot fix: if the model trained letterbox, the stretched input still shows it a 1:1 part where it expects 4:3, so detections can be missed outright even with perfect decode arithmetic.
| Letterbox (training) | Stretch (deployment) | |
|---|---|---|
| Scale, x and y | 0.16 and 0.16 | 0.16 and 0.2133 |
| Padding | 80 px top, 80 px bottom | none |
| Box centre, model space | (320, 240) | (320, 213.3) |
| Box size, model space | 128 × 96 | 128 × 128 |
| Decode needs | s and the two pads | factor_w and factor_h |
The synthetic-square test that catches the mismatch
Draw the test before the wrapper ships. Create a 640 × 480 image filled with gray 114, draw one black filled square of 200 × 200 with its top-left corner at (220, 200) - centre (320, 300), bottom-right (420, 400). Push it through the full deployed path: the same resize, inference and decode code that will run in production.
A matched pipeline returns the drawn square. A letterbox pipeline puts it at model-space centre (320, 380) - that is 300 plus the 80 px pad - and decodes back to (320, 300), 200 × 200. A stretch pipeline puts it at (320, 400) - 300 × 1.3333 - and decodes per-axis back to (320, 300), 200 × 200.
A mismatched one fails in a pattern you can read:
- box centre 20 px above the true 300, at (320, 320): a pad was subtracted from an image that was never padded - the worst case is a resize applied twice, as in Bounding box offset on new images: the double letterbox fix;
- box 267 px tall instead of 200, a 33% vertical inflation: one factor was applied to both axes;
- both axes correct but confidence far below validation: the geometry mismatch sits at the network, and only matching the training transform fixes it.
Accept nothing beyond 2 px on this image. It costs minutes, needs no labels, and it is the test to rerun after any framework, driver or runtime update.
Where the mismatch enters a vision line
The camera fixes the source size: an area scan camera such as the LS-CMR-0166 delivers 4000 × 3000 frames at 12 MP, the exact input of the worked example above.
- Sensor format
- 1/1.7"
- Interface
- USB3
- Resolution
- 12 MP
- Shutter
- Rolling
The runtime host applies the transform in hand-written code: on a vision controller hosting the inference service, a Python training script and a rewritten C++ deployment path diverge without anyone noticing.
- LAN ports
- 5+ LAN ports
- USB 3 ports
- 3-4 USB 3 ports
- Light outputs
- 4 built-in channels
- Memory
- 8 GB
When inference runs on the device itself, a smart camera such as the MV-SC5060XM-08M-WBN carries its own preprocessing; read which resize its runtime applies and train to match it.
- Interface
- GigE
- Resolution
- 6-10 MP
- Sensor format
- 1/2.4"
- Shutter
- Rolling
Pin the chain in the project notes before training starts: source resolution, resize rule, pad value, decode factors. That line is what an export review checks first when boxes drift after an update.
When direct stretch is the right choice
Compute one number first, at the production resolution: r = factor_w / factor_h.
- r between 0.98 and 1.02: source and input nearly share an aspect ratio, stretch and letterbox differ by under 2% of a coordinate value - 20 px at 1000 px from the origin - and either pipeline works as long as the decode matches.
- r far from 1: 1.33 for 4:3 into a square, 1.78 for 16:9. Default to letterbox unless all three conditions hold: one camera at one resolution, so the factors are constants; you own both the training transform and the deployment transform, ideally as the same function; and the decode applies factor_w to x and widths, factor_h to y and heights.
- Varying aspect ratios: mixed cameras, rotated mounts, crop-on-the-fly. Pad, do not stretch, so the model never sees proportions it was not trained on - Mixed resolution images in object detection training: pad, not stretch.
Under those three conditions a stretch pipeline pays: no rows spent on padding, and in the 4:3 example the part fills 128 × 128 instead of 128 × 96 in model space, 33% more pixels on the object. A rectangular native input - 640 × 480 for a 4:3 camera - delivers both benefits without distortion when export and runtime accept a non-square shape; Rectangular image input in deep learning training: letterbox or rect compares the two training routes.
Checks before you export
- Write down the training transform: scale rule, pad value, pad placement. The export is only as consistent as this note.
- Read the deployment resize: one scale factor or two, and is a pad subtracted? Both answers must match training.
- Compute r = factor_w / factor_h at the production resolution. Outside 0.98 to 1.02, single-factor decode is wrong by more than 2%; use per-axis factors.
- Run the synthetic-square test through the deployed path and require both axes within 2 px.
- Keep the transform in one shared function or config used by training and deployment, so the two cannot drift.
- Rerun the test after any framework or runtime upgrade; resize defaults change between versions.
Two numbers decide everything here: source resolution and network input size. With them, the factors, the decode and the choice fall out in minutes. If you are speccing hardware around the pipeline, the camera finder filters area scan and smart cameras by resolution and interface, or ask us through the contact page with those two numbers included.
Questions
Can I train on images resized offline and skip letterbox entirely?
Yes, and it is the cleanest stretch workflow when every production frame shares one aspect ratio. Resize the whole training set to the exact network input before the data loader runs, so no transform executes at train time, then deploy a wrapper that does the same plain resize and decodes with factor_w on x and widths and factor_h on y and heights. Mixed production aspect ratios break it, because the model then sees proportions absent from training - pad those instead.
How much of a square input does letterbox padding waste on a 4:3 camera?
For a 4000 × 3000 frame in a 640 × 640 input, the scale is 0.16, the content covers 640 × 480, and 160 of the 640 rows are gray - 25% of the input tensor. A 16:9 source is worse: 280 rows, about 44%. A rectangular native input that matches the sensor aspect removes both the waste and the distortion, provided export and runtime accept the non-square shape.
Does the padding color change detection results?
It changes the statistics the network sees. Gray 114 is close to the mean of typical training sets, so after normalization the padded bands carry little structure and cannot be mistaken for edges. Pure black padding draws a strong dark border that some models react to, especially around bright parts. Whichever value you pick, keep it identical in training and deployment; a mismatch acts like any other preprocessing change and moves boxes and confidence.
Boxes come back offset only vertically after ONNX export. What does that tell me?
A single-axis offset is the signature of an aspect-ratio mismatch, not of a broken export. Boxes sitting above the true position mean a pad was subtracted from an image that was never padded; boxes below mean one scale factor was applied to both axes. With a single-factor decode the offset grows with distance from the top of the frame, proportional to factor_w / factor_h minus one. Confirm with the synthetic-square test, then fix the decode or match the training transform.


