Draw and fill segmentation label polygons on training images

Segmentation training fails quietly: the loss goes down, the polygons in the training montage look roughly right, and nobody notices the boundaries sit two pixels off the true part edge until the deployed model labels good parts as defective. The fix is to draw and fill the label polygons on the raw images yourself, with your own code, and read the geometry directly. This guide gives you that code, the denormalization math behind it, and the three failure modes that produce offset contours when the bounding boxes are fine.

This matters most on high-resolution capture, where a one-pixel annotation slip is invisible on a laptop preview but is real training data. If you capture with a 20 MP camera such as the MV-CS200-10UM, checking labels at native resolution is the only reliable check.

What the label file actually contains

Segmentation label formats store polygons as normalized coordinates: every value divided by the image width or height, giving numbers between 0 and 1. A typical line holds a class index followed by a flat list of pairs, where the first of each pair divides by image width and the second by image height. The file says nothing about the resolution it was annotated at, so the consumer must reconstruct pixel coordinates with the dimensions of the image it pairs with.

That asymmetry is the whole source of offset bugs. Get width and height swapped and every polygon is stretched or squashed. Scale with the wrong resolution and the error grows with the resize factor.

Draw and fill polygons with OpenCV

The core routine reads a label file, denormalizes, and draws. Outline and fill do different jobs, so do both:

import cv2
import numpy as np

def draw_label_polygons(image_path, label_path, class_colors):
    img = cv2.imread(image_path)
    h, w = img.shape[:2]
    overlay = img.copy()

    with open(label_path) as f:
        for line in f:
            vals = line.split()
            cls = int(vals[0])
            coords = np.array(vals[1:], dtype=np.float64).reshape(-1, 2)
            coords[:, 0] *= w   # x was normalized by width
            coords[:, 1] *= h   # y was normalized by height
            pts = coords.round().astype(np.int32)

            cv2.fillPoly(overlay, [pts], class_colors[cls])
            cv2.polylines(img, [pts], isClosed=True,
                          color=class_colors[cls], thickness=1)

    blended = cv2.addWeighted(overlay, 0.35, img, 0.65, 0)
    cv2.polylines(blended, [pts], isClosed=True,
                  color=class_colors[cls], thickness=1)
    return blended

Use a thickness of 1 for the outline. At thickness 3 the outline itself covers a two-pixel boundary error, which is exactly the defect you are hunting. Keep polygon coordinates as float until the final round() before drawing: rounding earlier bakes the error into the annotation permanently.

Diagnose an offset contour

If the contours shown during training are offset but the bounding boxes are correct, test the three usual causes in this order.

1. Swapped width and height. For a 4000 × 3000 image, a swap scales x by 0.75 and y by 1.33. Draw a square part that is square in the image; if the filled region is visibly a rectangle, dimensions are swapped in the denormalization. Non-square sensors make this obvious; square crops hide it.
2. Wrong resolution for denormalization. If images are resized to 640 × 640 for training but the polygons were denormalized against the original 4000 × 3000 (or the reverse), all coordinates shift proportionally. Print the shape used for scaling and the shape on disk; they must match.
3. Rounding too early. If the annotation tool saved integer pixel coordinates that were normalized and the trainer multiplies back, the error is under a pixel and rarely visible. But if rounding happened before normalization at a different resolution, the shift can reach several pixels. Compare a polygon vertex against the same corner in the image at high zoom.

A fourth, rarer cause: the polygon winding order or a self-intersecting polygon makes fillPoly fill an unexpected region, while the outline still traces the vertices. The outline-plus-fill pair catches this immediately, which is why you draw both.

Verify against the geometry, not the eye

Eyeballing catches gross errors; for pixel-level truth, measure. If your parts are square, the contour geometry behind square detection applies to labels too: extract the polygon's corners, check the angles are 90 degrees and the sides equal. A filled label over a square part that measures 1.33:1 tells you instantly which denormalization bug you have.

For area checks, compare the polygon area against the bounding-box area. A sane polygon for a compact part covers 60–90% of its bounding box. A coverage above 100% is impossible and means coordinates exceed [0, 1] — usually a normalization that divided by the wrong dimension.

Compute the polygon area with the shoelace formula on the denormalized points. The same pixel-to-real-world mapping that bounding box depth mapping covers lets you convert the discrepancy into millimetres on the part, which is the number that decides whether the label is acceptable.

Build the check into your workflow

Run label visualization on a fixed sample — say 20 images per class — before every training run, and save the overlays as artifacts. Two practices keep them useful:

• Native resolution only. A downscaled preview hides one-pixel errors. Open the original files the trainer will read.
• Toggle the overlay. Save both the blended image and the raw image; flipping between them at 100% zoom exposes sub-pixel boundary drift that a static blend conceals.

If your defects are themselves thin structures, the same native-resolution discipline governs training data quality — the crack segmentation guide shows why thin-class labels are the least tolerant of boundary error.

Capture quality still bounds label quality

Precise polygons are only possible on sharp, well-exposed images. A blurred edge has no true boundary, so annotators guess and offsets become unmeasurable. Set exposure and gain deliberately — the trade-offs are covered in the industrial camera image quality settings guide — and confirm the saved settings actually applied at startup, which the camera startup frame rate verification guide walks through. A monochrome sensor simplifies edge judgment on machined parts; browse the area scan cameras or use the camera finder to filter by resolution and shutter. For compact inspection cells that run inference on the device, the smart cameras category covers integrated options, and the camera configurator narrows by interface and sensor format.

Quick checklist before you train

• Denormalize x by width and y by height, never the reverse.
• Scale against the exact resolution of the file on disk.
• Round to integers only at the draw or mask-conversion step.
• Draw outline and fill together; each catches errors the other hides.
• Check polygon-to-box coverage lands in a plausible 60–90% range.
• Review at native resolution, toggling the overlay on and off.

When the usual answer is wrong

Occasionally the labels are perfect and the montage is lying: some training pipelines render the label preview after their own resize, so the preview offset does not exist in the data. Test by running your own visualization on the untouched files. If your polygons land on the boundary there, the training itself is fine and the preview code has the bug. Fix the preview, not the dataset.

Once your overlays look right, the next step is a training run you can trust: pick a camera through the camera finder, size the lens with the lens focal length calculator, and contact us through the contact page if you want the capture chain reviewed before you annotate a thousand images on top of it.