Tiled inference confidence and IoU thresholds for small objects

You trained on slices, you predict through the same slicer, and the small objects still come out missing - or twice, once per tile seam. In most cases the model is fine and the prediction-time numbers are wrong. A sliced-inference call filters every tile's detections through one confidence value before it merges anything, so a threshold that is sensible for whole frames, 0.6, discards exactly the low-scored small-object boxes that tiling exists to recover. The working numbers are simple: drop the per-detection confidence to 0.1-0.3, keep the per-tile NMS IoU near 0.5, and treat the cross-tile merge threshold as a separate decision with its own value. A worked frame - 12 MP, 1024 px slices - shows the tile count, the effective object size and the merge arithmetic, so you can predict what your settings will do before you run them.

What goes wrong, and where to look first

Two symptoms dominate, and a third hides underneath.

Small objects vanish. The class validated well on tiles, but production frames return nothing. The detector usually saw the object and the confidence filter threw it away.

Detections double near seams. Adjacent tiles each report the same part, the merge stage fails to recognise the pair as one object, and counts run high.

Everything runs slowly. A frame takes seconds instead of milliseconds, usually because the runtime fell back to the CPU. This one distorts every other measurement, so clear it first.

In order of probability: the prediction confidence is set for full frames (0.5-0.6) and needs to drop to 0.1-0.3; the two IoU settings have been conflated, so one of them is missing or wrong; the slicer is loading a detect checkpoint where the training produced segmentation weights; the PyTorch build is CPU-only; and, least often, training and inference tile differently. Each cause has a test in the sequence below.

Why sliced inference needs a lower confidence threshold

Slicing changes what the detector sees, and each change pushes small-object scores down.

The seam cuts the object. A part straddling a tile boundary is half-visible in each tile, and a half-evidenced box scores low. Scores of 0.3-0.4 for a badly clipped copy against 0.6-0.8 for the same part seen whole are typical.

Small stays small, by design. At full-frame inference the camera frame is resized to the network input before prediction: a 4096 × 3072 frame at imgsz 1024 scales by 1024 / 4096 = 0.25, so a 32 px feature becomes 8 px. Slicing feeds each 1024 px tile at native size, so the same feature stays 32 px and keeps (32 × 32) / (8 × 8) = 16 times the pixel area. That is the entire reason you tile. The catch is that detections recovered this way sit at the low end of the score distribution, because small and partly clipped examples were rare in training too.

The filter fires before the merge. The slicer applies model_confidence_threshold to each tile's raw detections, then merges duplicates across tiles. Merging only removes boxes; it never raises a score. At 0.6, every clipped small-object box dies before the merge can consider it.

So the starting value is low on purpose: set model_confidence_threshold to 0.2 and tune inside the 0.1-0.3 band on validation tiles. Below 0.1 you mostly import noise for the merge stage to clean up; above 0.3 you are back to losing the clipped copies that tiling exists to catch.

The two IoU numbers: per-tile NMS versus cross-tile merge

"IoU" does two different jobs in a sliced pipeline, and they live in two different places.

Inside each tile, the detector's own post-processing runs non-maximum suppression under its iou setting - the same iou you would set on a plain predict call. It removes overlapping duplicate boxes within one tile's output.

Across tiles, the slicer compares boxes that came from different, overlapping tiles. In the SAHI predict call these are postprocess_match_metric="IOU" and postprocess_match_threshold: when two boxes from different tiles overlap by at least the threshold, they count as one object and are merged. This is not the model's iou under another name, and it is not settable through the model object either - passing iou_threshold there fails outright with DetectionModel.init() got an unexpected keyword argument 'iou_threshold'. The working route for the per-tile settings is a small config file passed as model_config_path, carrying values such as imgsz and iou.

The confidence value has the same double life: the loaded model can carry its own confidence_threshold while the predict call takes model_confidence_threshold. Keep the two equal. A stale 0.45 left on the model object next to a careful 0.2 on the call is a common way to lose a day.

Worked example: one 12 MP frame in 1024 px slices

Take a 12 MP area-scan frame, 4096 × 3072 px, sliced at 1024 × 1024 with overlap_height_ratio and overlap_width_ratio both 0.35.

• Step per axis: 1024 × (1 - 0.35) = 665.6 ≈ 666 px.
• Columns: ceil((4096 - 1024) / 666) + 1 = ceil(4.61) + 1 = 6.
• Rows: ceil((3072 - 1024) / 666) + 1 = ceil(3.08) + 1 = 5.
• Tiles per frame: 6 × 5 = 30 forward passes.

The +1 counts the flush tile against each frame edge, which is why the arithmetic lands on 30 rather than 25.

Now the thresholds on this frame. A 32 px feature occupies 1,024 px² of a tile. Its contained detections might score 0.7 and its seam-clipped copies 0.35 - illustrative scores; the histogram habit below tells you where your own bulk sits. At model_confidence_threshold = 0.6 the clipped copies are deleted before the merge, and the part survives only where a tile caught it whole: good in the middle of the frame, gone at the edges. At 0.2 both copies survive, the displaced pair merges at postprocess_match_threshold = 0.5, and one box with one score comes out.

The cost side matters as much: 30 tiles is 30 times the single-frame workload, and that ratio decides where the inference runs.

Where the tiles run: let the tile count pick the box

Budget tiles against latency and the line rate. Planning ranges for a mid-size segmentation model: 0.1-0.5 s per 1024 px tile on a desktop-class CPU, so a 30-tile frame takes 3-15 s; roughly 5-20 ms per tile on a GPU, so the same frame lands in 0.15-0.6 s. If your measured numbers sit in the slow column, check the runtime before touching thresholds. A CPU-only PyTorch build - its version string ends in +cpu - cannot serve device="cuda:0", and the environment check (yolo checks) reports no GPU. Install the CUDA build that matches your installed torch version, restart the session, and re-check. The object detection without a GPU guide covers the CPU path in depth, and acquisition faster than processing covers the same gap from the camera side.

Where the loop then lives follows from the tile budget. Smart cameras are built for inspection on the device, with no PC in the loop - the natural home while tile counts stay low. The camera finder filters them by resolution and interface.

An x86-platform camera carries the whole tile loop on the camera itself:

The LS-SMC-0017 is an x86-platform smart camera with an 8.8 MP sensor and a C-mount.

For heavier tile counts, a vision controller sits beside the cameras and takes the inference load:

The MV-VC4712-256G10-2TH runs an Ultra 7 255H with PoE camera ports and PCIe expansion supported.

When tiled inference is the whole job, an AI inference terminal pairs camera ports with inference acceleration in one box:

The MV-VAC5724-256G02 pairs PoE camera ports with a dual-card AI inference layout and 8 built-in light channels.

Test and fix sequence

1. Baseline without the slicer. Run plain full-frame prediction at conf 0.25 on ten raw frames. If the results are wrong here, fix the model or the weights path first; no threshold setting can rescue this step.
2. Check the weights. Point the slicer at the best.pt inside the segmentation run (runs/segment/...), not at a detect run. A detect checkpoint served through a segmentation pipeline loses masks and shifts scores.
3. Set the three values once. model_confidence_threshold 0.2, postprocess_match_threshold 0.5 with postprocess_match_metric "IOU", slice_height and slice_width 1024, both overlap ratios 0.35. Change one variable at a time from there.
4. Confirm the compute. yolo checks should list the GPU. A version string ending in +cpu means the CUDA build is missing.
5. Sweep confidence. Run model_confidence_threshold from 0.10 to 0.30 in 0.05 steps over about 50 validation frames and record small-class recall at each step. Take the lowest value that holds recall flat.
6. Count duplicates. After the merge, flag any box that overlaps another with IoU above roughly 0.3. If more than a few percent of objects double, step postprocess_match_threshold down in 0.1 increments until they merge, and stop before distinct neighbouring parts start collapsing into one box.
7. Restore precision at the end, not the start. If the lower threshold admits false positives, add a single score cut on the merged output around 0.4-0.5 instead of pushing the slicer's filter back toward 0.6.

Keeping the settings true after the fix

• Tile the same way in training and inference, with each tile's labels remapped into tile coordinates. If the small class is thin in the dataset, how many images to train an object detection model gives the sizing arithmetic before you slice more data.
• Log the raw score histogram per class once. The threshold belongs just below the small class's bulk, and the histogram shows where that is - better evidence than any rule of thumb, including this one.
• Keep preprocessing identical across training, validation and the tile loop. When production confidence sits below training confidence for every class, matching the preprocessing is the usual cure, not the threshold. On the line itself, ambient light variation is the usual driver.
• Keep the task type consistent end to end. A segmentation model served through a detect-shaped pipeline loses masks and scores before any threshold matters; vision model task configuration covers the wiring.
• Re-check the thresholds after every retrain. They are properties of the model's score distribution, not constants. A retrain moves the distribution, and 0.2 moves with it.

When the usual answer is wrong

• Vanishes at every threshold. A class that never appears, even at 0.05, is a data or mapping problem: tile labels remapped wrongly, or weights from the wrong task. Fix the pipeline; stop lowering the threshold.
• Distinct parts collapse into one region. Neighbouring objects fusing means the match threshold is too low for your object spacing. Bring it back up in 0.1 steps until neighbours stay separate, or shrink the slice size so fewer objects share a tile.
• Confidence drops only in production. Every class scores lower on the line than in validation: that is preprocessing or illumination, not thresholds. Work the two links in the previous section.
• The rate cannot be met even on a GPU. Cut the tile count instead of tuning harder: larger slices where object size allows, or a coarse full-frame pass that gates a fine tiled pass to a small region. Coarse-gates-fine is the same idea behind pattern match score thresholds.
• One caveat worth keeping: thresholds tuned on a small validation set travel only as far as that data. Re-run the 50-frame sweep whenever the line, the lens or the lighting changes.

To size the hardware around your own tile count, the configurator narrows camera, lens and light options in one pass, and the systems topic page collects the rest of the inference guides. Send us the frame size, slice size and line rate via the contact page and we will help you match the box to the numbers.