Segmentation prediction offsets from contours: snap masks to edges

You trained a segmentation model, the classes are right, and the mask still sits a few pixels inside the wall. That gap — the offset between the predicted boundary and the image contour — is why floor-plan rooms stop touching their walls, why seam-width checks drift, and why thin structures fail tolerance on otherwise good hardware. This page gives you the numbers: where the offset comes from, what it costs in millimeters at your pixel scale, and the two fixes that hold — a full-resolution skip path in the decoder, or snapping the predicted boundary to the detected edges before you measure. Each section ends with its decision.

Why does a predicted mask sit a few pixels off the true contour?

The decoder predicts on a downsampled feature grid: at a 640 px input and stride 32, only 400 boundary cells exist, so a 3 px wall falls between cell centers and upsampling smooths the result without re-localizing it.

The arithmetic: 640 / 32 = 20, so the deepest stage works on a 20 × 20 grid of 400 cells, each covering 32 px of input. A wall 3 px wide occupies 3 / 32 ≈ 0.09 cells — under one quantum — so the boundary lands on whichever cell edge the encoder votes for, up to 16 px (half a stride) from the true contour before any upsampling runs.

Bilinear interpolation and transposed convolutions then fill in between the 400 predictions. They smooth the stair-steps; they cannot recover the gradient peak the encoder downsampled away. Boundaries are also a tiny share of the pixels: on that 640 × 640 image a 3 px wall contributes roughly 2 × 640 × 3 ≈ 3,840 boundary pixels out of 409,600 — under 1 % — so the loss has little to anchor to.

It is the same geometry behind teaching long thin parts by feature, not whole part: when the structure of interest is a few pixels wide, everything around it decides the measurement. Floor plans, seams, wires, gaskets and printed line widths all live in this regime.

Decision: the offset is a decoder property, not evidence of a bad training run — check the grid arithmetic (input px / stride) before blaming the labels.

How large is the offset in pixels, and what does it cost in millimeters?

Plan on 2-10 px of boundary shift for a stride-32 decoder on thin structures, and price it once at your scale: at 0.049 mm/px, a 6 px offset is 0.29 mm of wall error.

Worked example, machine part: the field of view is 120 mm across a 2,448 px sensor width, so mm_per_px = 120 / 2,448 = 0.049 mm/px. An unsnapped stride-32 mask drifting 6 px costs 6 × 0.049 = 0.29 mm and fails a ±0.2 mm tolerance. The same mask snapped to within 1 px costs 0.049 mm — a 4 × margin. To hold that margin by design, budget a pixel size of 0.05 mm/px or better: 120 / 0.05 = 2,400 px minimum, and 2,448 px leaves (2,448 − 2,400) / 2,400 ≈ 2 % headroom.

Decision: if offset_px × mm_per_px sits inside tolerance, a plain decoder is good enough. If it does not, shrink the output grid (next sections) or snap to edges. When the tolerance drives the sensor choice, the resolution calculator turns field of view and required accuracy into a pixel count.

Does input size or cropping make the offset worse?

Yes: train and infer at the same image size, and letterbox-resize instead of cropping, because a crop that clips part of an object deletes the contours the decoder needs and the mask ends wherever the crop did.

The classic failure: a model trained at 640 × 640 meets a 2,000 px image, the pipeline tiles or crops down to the training size, and a wall crossing the tile border is cut mid-shape. The mask stops a few pixels inside the border — it looks like a huge offset but is missing input. The symptom behind many large-imgsz reports is exactly this: the object grows past the crop and gets truncated. Published comparisons of cropping versus resizing for segmentation, including 2024 experiments on brain-tumor MRI, treat the choice as a measured performance factor rather than a neutral preprocessing detail — and cropping carries a risk resizing never has, the cut at the border.

If you must tile for resolution, overlap tiles by at least the widest object you segment and stitch before you measure, so every structure appears whole in at least one tile. Decision: same imgsz for training and inference, letterbox instead of crop, tile overlap at least the largest object width.

Can more training fix it, or is the architecture the problem?

Boundary-weighted losses and consistent annotation widths shave the shift at the margin, but no loss can move a boundary by less than half its prediction cell — 16 px at stride 32 — so the durable fixes are a finer output grid or post-processing.

Training-side moves that help: weight the boundary class so the sub-1 % of pixels that decide the measurement are not drowned out; annotate thin structures as regions of constant width (2-3 px) instead of 1 px lines, giving both sides of a wall a target; train on tiles at the resolution you will deploy. Each reduces bias. None changes the quantum: the mask grid is still 20 × 20, so the error floor stays at half a cell.

Decision: spend a day on boundary weighting and annotation width, re-measure the offset on a holdout, and if the tolerance still fails by more than about a pixel, move the budget to the decoder or the snap step. Retraining cannot recover information the encoder discarded.

What does a full-resolution skip path actually change?

Skip connections feed full-resolution encoder features to the decoder so the mask predicts at stride 1-2 instead of 32; on the 640 px input that turns 400 output cells into 409,600, a 1,024 × jump, and that is why thin walls land on the contour instead of the nearest cell.

The arithmetic: a stride-1 head outputs 640 × 640 = 409,600 cells against the stride-32 head's 400, a factor of 32² = 1,024. At stride 2 it is 320 × 320 = 102,400 cells, 256 ×. Output cells scale with the square of the stride ratio, and memory and inference time move the same direction — which is why stride-1 decoders on multi-megapixel images belong on a controller-class machine rather than a small on-device sensor.

A middle path exists for exactly this problem: predict the boundary class at stride 1-2 through the skip path and the interior at stride 32. Thin structures get full resolution where it matters, and the cell count rises only across the boundary band.

Decision: if the tolerance needs the boundary inside 1-2 px by construction, ask for the finer head. If the tolerance sits at 5 px or more in image terms, the snap step is cheaper than the compute.

How do you snap a predicted mask back to the image contours?

Detect the image edges, keep only edge pixels inside a band around the predicted boundary, and pull each boundary point to the strongest gradient within its search radius; set that radius from your measured offset, never wider than half the nearest parallel edge.

Variables: p95_offset_px is the 95th-percentile boundary-to-edge distance measured on labeled data, in pixels; d_min_px is the distance in pixels to the nearest parallel edge the snap could wrongly capture — the opposite wall of a corridor, the second seam of a gasket; |∇I| is the gradient magnitude; r is the search radius in pixels.

1. Run an edge detector over the image. A Canny hysteresis pair near 50 / 150 is a workable start; tune until every structure you measure yields a connected edge, keeping the low threshold around a third of the high one.
2. Build the band: dilate the predicted mask boundary outward and inward by r.
3. Move each boundary vertex to the strongest gradient pixel inside its band; if no edge pixel lies in the band, leave the vertex in place and count it as unsnapped.
4. Re-close the polygon and re-measure the boundary-to-edge distance.

Active contours do the same job continuously: the curve balances internal stiffness against image-gradient attraction, and constraint markers seeded from the predicted mask keep it in the right neighborhood. The division of labor is always the same — the network decides which edge, the image decides where exactly.

Decision: derive r from measurements, never from preference; a band that keeps coming back empty points to a contrast problem upstream, not to the radius.

How do you verify the snap grabbed the right edge?

Measure boundary-to-edge distance again on a labeled holdout and gate the result: accept a snapped mask when the median distance is 1 px or less and the 95th percentile is 2 px or less, with no boundary moved farther than its search radius.

Compute it as a chamfer distance: for every mask-boundary pixel, take the distance to the nearest detected edge pixel, and report median and 95th percentile per structure rather than per image. A second mode in the distribution at d_min_px is the signature of cross-snapping — boundaries jumping to the parallel edge — and the cure is a smaller radius or a tighter band, not more training.

Watch the snap live by drawing the corrected mask over the image: the methods in image overlays with transparency and masks transfer directly, and line-side displays that highlight defects in red use the same overlay configuration (a knowledgebase walkthrough covers an overlay for red defect masks).

Decision: pass the gates on the holdout, then freeze them. The gates, not the architecture diagram, are the spec you hand to production.

Should inference and snapping run on a smart camera or a vision controller?

Run it on the device when one field of view, a few megapixels and a stride-2 output head cover the tolerance, and move it to a vision controller when you need dense stride-1 masks, several cameras, or GPU-class inference with headroom for the snap.

On-device keeps capture, inference, snap and result inside one housing — the working premise of the smart cameras category; the camera finder filters them by resolution, shutter and mount. Which steps live on the camera and which belong downstream is the subject of on-camera image processing: what you can offload — localization-critical steps such as the edge snap want the pixels the device already holds.

The on-device case in practice:

A 5 MP color vision sensor such as the MV-SC3050XC-16M-ATN-H — global shutter, Fast Ethernet, M12 mount — carries capture through result on the device for a static field of view.

The MV-SC5060XM-08M-WBN adds 6 MP mono resolution over GigE on the same M12 mount, for fields of view that need the extra pixels.

The multi-camera, dense-mask case:

The MV-VC5525-256G70-NN vision controller pairs PoE camera ports with 32 GB of memory and 8 built-in light channels, and its PCIe expansion accepts a GPU for stride-1 decoders. The vision controllers category lists by LAN port count, PoE and memory, so match camera count first.

As a working rule from the tolerance math above: one camera, up to about 5 MP, stride-2 head → on-device sensor; two or more cameras, 12 MP-class dense masks, or a stride-1 decoder → controller with PoE.

Before you order, four numbers decide it:

• field of view and tolerance, in mm → the pixel-size budget;
• thinnest structure, in px → whether it survives stride 32 at all;
• measured p95 offset, in px → the snap radius r;
• camera count and output stride → on-device or controller.

When is edge snapping the wrong fix?

Edge snapping fails when the image, not the network, causes the drift: defocus and motion blur move the gradient peak on their own — 50 mm/s across a 2 ms exposure smears 0.1 mm, about 2 px at 0.049 mm/px — and auto exposure moves it frame to frame.

Motion blur is arithmetic, not mystery: blur_px = part_speed × exposure_time / mm_per_px, so 50 mm/s × 0.002 s / 0.049 mm/px ≈ 2 px, and a snap chasing a smeared peak inherits the smear. Shorten the exposure and let the lighting carry the signal; on a moving line, choose a global-shutter sensor and filter for it in the finder.

Auto exposure is its own trap: gain and exposure time change between frames, the gradient peak moves with them, and a snapped mask inherits the wobble. That mechanism is the subject of contour position jitter under automatic exposure; lock exposure and gain before trusting a 1 px gate.

Low contrast defeats the snap outright: if the edge detector cannot find the wall, there is nothing to snap to. Raise contrast at the source with lighting and exposure before adding post-processing.

Decision: fix optics and exposure first, snap second. A snap step cannot outrun a blurred image, and a mask that only passes on a sharp image needs its radius re-measured whenever the line speeds up or the lighting changes.

If those four numbers are on your desk — field of view, tolerance, part speed, camera count — the build-a-system configurator sizes camera, lens and light around them, or contact us and we will check the decoder-and-snap plan against your parts.