Mask out background regions for stable pattern matching scores

A pattern match that scores 0.85 on one frame and misses an 0.80 threshold on the next is rarely a part problem. The matcher reads every pixel inside the search region, and the background around the part — belt texture, a neighboring casting, a shadow that drifts — changes between frames while the part does not. This guide shows you how to mask that background out with irregular polygons and painted ignore zones, how to check which edges the matcher is really using, and how to set a threshold the masked scores actually fit.

Why background pixels destabilize the score

Correlation-based matching compares your trained template against candidate positions across the whole search region. Edge-based variants extract feature points from the training image and score how many reappear, at the same relative positions, in the run image. Either way, the pixels you leave inside the region are not neutral. They vote.

The vote works against you twice. Train with the belt inside your region, and the template now carries belt edges that must reappear in every run image; the part shifts against the belt at each index, those edges land in the wrong places, and the score drops. In the run image, clutter adds false hits that inflate or deflate the score depending on how the tool normalizes it. Identical parts end up tenths of a score apart.

The familiar symptom is a part that finds nothing at an 0.80 threshold, then reports 0.85 once the threshold is relaxed to 0.70. Two mechanisms produce it. The coarse search can reject the true position before the fine scorer sees it; that gating is covered in pattern match score threshold: the coarse search gates the result. The other mechanism is score drift from background pixels, which is what masking fixes. How a reported score relates to the threshold you set is its own question too — template matching score thresholds: was the match found? works through it.

What to mask, what to keep

Mask anything whose pixels change while the part stays identical:

• Belt and chain texture, including the belt joint line that passes through the region once per belt revolution.
• Neighboring parts that wander into the search region on a busy conveyor.
• Date and lot codes. A printed mark that changes every shift is worse than clutter: trained into the template, it has to match today's code tomorrow.
• Tooling that enters the frame: grippers closing on the part, retracting part-present flags, fixtures that swing in.
• Wandering shadows and glare from windows, sun angle and overhead lights that switch.

Keep everything the match needs to pin the part's position and rotation: at least two independent feature clusters, such as two corner arcs and one hole edge. Draw the polygon 2–3 px clear of the edges you keep, so ±1 px of edge localization on the run images does not cut a feature in half.

Then decide where the mask lives. An ignore zone saved with the template travels with every match and keeps covering the same spot on the part wherever the part goes. A mask painted into the run image stays at fixed pixel coordinates; that suits stationary clutter such as a machine-frame edge — the same job a trapezoid mask does before road-crack detection, as in masking a trapezoid ROI before crack detection in road images. If your tool takes a mask image instead of drawn shapes, a 0/255 image does the same work: 0 is ignored, 255 is inspected, and any drawing routine can produce one.

How to mask out the background, step by step

1. Log images first. Save 30 or more run images spanning the part's full position range, with the scores your current setup reports. The span, not the average, is the number that hurts: max minus min across good parts.
2. Measure the position range. Read the minimum and maximum x-y match positions from that log. At an example scale of 0.1 mm/px, a ±2 mm indexing wander is ±20 px per axis; the search region and any image-fixed masks must cover it.
3. Draw the polygon. In the training image, draw an irregular polygon around the part, 2–3 px clear of kept edges. Follow the real silhouette and include the cut-outs where the belt shows through; a rectangle over such a part just re-admits the background.
4. Paint the ignore zones. Inside the polygon, paint out the date code, glare patches and everything else that changes while the part is identical. If your editor only offers primitive shapes, several small rectangles beat one large one.
5. Anchor the zones. Save ignore zones with the template so they travel with the match. If your tool only has image-fixed masks, draw them for the far end of the position range from step 2.
6. Retrain and replay. Run the 30 logged images again on the new template. The good-part span should fall from tenths to a few hundredths.
7. Check the features. Turn on the found-feature overlay on the training image and on run images from both position extremes. Any feature point landing off the part means background still votes: paint it out, retrain, replay.
8. Black-out test. On five logged images, fill the masked area with solid black and compare scores. A filled polygon mask is a few lines of code — see create a black mask image with a filled circle ROI and OpenCV polygon mask: crop with a transparent background. A shift above about 0.03 means clutter still contributes.
9. Set the threshold from the spread. Place it between the worst good and the worst bad score; the worked example below runs the arithmetic. From then on, log results per program, so one recipe's scores never mix with another's.

Worked example: a bracket that missed at 0.80 and found at 0.85

Inputs. A stamped bracket on an indexing conveyor, camera overhead, 1.6 MP frame — 1,600,000 px. The search region is an 800 × 600 px rectangle and the polygon around the bracket face accepts 300,000 px. Score logs: before masking, 30 good parts scored 0.58–0.85 and brackets with a missing locating hole scored 0.55–0.71; after masking, the same 30 good parts scored 0.84–0.88 with the bad brackets unchanged at 0.71 or below. The pixel counts are example values for this class of set-up; the 0.80, 0.70 and 0.85 figures are the symptom this guide set out to fix.

Rule. On the masked template, place the threshold midway between the worst good score and the worst bad score:

Arithmetic. Region: 800 × 600 = 480,000 px, which is 480,000 ÷ 1,600,000 = 0.30 of the frame. Mask: 300,000 ÷ 480,000 = 0.625 accepted, so 1 - 0.625 = 0.375 removed — 180,000 px, 37.5% of the region. Spans: 0.85 - 0.58 = 0.27 before, 0.88 - 0.84 = 0.04 after. Threshold: 0.84 - (0.84 - 0.71) ÷ 2 = 0.84 - 0.065 = 0.775, set to 0.80.

Decision. The 0.80 threshold that missed parts before masking now passes every good part with 0.04 of margin and rejects the worst bad bracket by 0.09. Masking also cut the evaluated pixels to 62.5%, so each candidate costs about 0.625 of before — roughly 1.6x the match throughput on the same processor. Shrinking the search window itself buys more, as pattern matching low fps: shrink the search to speed the update rate shows.

Verify the mask before you trust the threshold

Four checks, in this order:

• Overlay, training image. Every found feature sits on the part — nothing on the belt, the fixture or a neighboring part.
• Overlay, position extremes. Run images from both ends of the position range. Anchored ignore zones must still cover the same part features; image-fixed masks must still clear the part path.
• Black-out delta under 0.03. If blacking out the mask area moves the score by more than 0.03, the mask is not covering the clutter — redraw it.
• Separation of at least 0.10. Worst good minus worst bad on the masked template; the example above ends at 0.84 - 0.71 = 0.13. Below 0.10, fix lighting or features before you trust any threshold.

That re-check is the same discipline as validate a trained pattern matcher when features stay hidden: a new template means a new score scale, so the acceptance set has to be new with it.

When masking is not the fix

Masking removes pixels that should never have voted. It does nothing for:

• A score that tracks ambient light. If the spread follows the time of day or a light bank switching on, the part's own appearance is changing. Fix exposure and shielding first, or move to edge-based matching, which holds results when lighting changes — edge based pattern matching for stable results when lighting changes — and control the environment per ambient light variation in pick and place vision: control it.
• A find that fails while the score looks healthy. If relaxing the threshold reveals a good score, the coarse search rejected the true position; the fix is in the coarse pass, covered earlier, not in another repaint.
• A mask that starves the template. Tools refuse templates below a minimum area or feature count; the answer is more of the part inside the region, as pattern match error template image too small: fix ROI size explains.
• Clutter that is actually a mating part. A component that legitimately sits in the frame is not background. Exclude it from training as an ambiguous region, per exclude ambiguous regions from detection training with a validity mask.

Where to run it: smart camera or vision controller

A tight, masked single-part match is light work, and today it runs on the camera itself. The LS-SMC-0063 pairs a 1.6 MP sensor with a fixed 16 mm lens and runs inspection on device at 60 fps. The MV-SC5060XM-08M-WBN adds a 6 MP mono sensor behind an 8 mm M12-mount lens on GigE, for more pixels on the same part. Both keep the whole pipeline — image, masked match, decision, digital outputs — inside the smart cameras and vision sensors line:

Cells with several cameras, several recipes and per-program image logs want a controller. The MV-VC3301H-128G66 ships with 8 GB of memory and a 128 GB SSD, and its four built-in light channels drive the illumination directly:

Browse the vision controllers category, filter by ports and resolution in the camera finder, and lay the station out in the build-a-system configurator. The learn hub and the systems topic page collect the surrounding guides, and contact us with a few logged images if you want a second opinion on a template that refuses to hold its score.