Template matching score thresholds: was the match found?

Template matching in the classic C vision API always returns a best position. The comparison runs at every offset, the score map always has an extreme somewhere, and the location search hands it back whether the part is in view or not. The question on a production line is different: did the match find the template — yes or no — and at what score do you commit? This walkthrough gives the decision rule for both method families: the squared-difference score, where you read a minimum, and the normalized-correlation score, where you read a maximum. It shows the arithmetic that turns a raw minval into a gray-level threshold, a worked example with real numbers, the tests that prove the code, and the imaging conditions that keep one threshold value valid in production.

Why the matcher always reports a position, even with no part in view

Every offset is scored: with an image of W × H pixels and a template of w × h, the result map holds (W − w + 1) × (H − h + 1) scores. The cvMinMaxLoc call returns one of them whether or not the template exists in the scene.

The comparison writes a dense map of scores, one per tested position. The point it reports is simply where the score is most extreme — the least-bad position when the part is missing. That is why drawing code always draws a rectangle, and why a position alone can never be the found/not-found signal. The signal is the score at that position, normalized so it is comparable from frame to frame. Normalizing the result map for display with cvNormalize and CV_MINMAX is for viewing only; it rescales the scores to 0–1 and destroys the absolute values your threshold needs. The names here come from the classic C interface; the current C++ interface carries the same methods, and the same decision applies.

Which extremum to read — the minimum or the maximum?

Read the minimum for squared-difference methods, where a perfect match scores 0, and the maximum for normalized-correlation methods, where a perfect match scores 1.0. The two families measure opposite quantities — error and similarity — and reading the wrong end accepts the worst position on the image.

The third family subtracts the template and patch means before correlating, so its score runs from −1 to 1 and ignores overall brightness. Reach for it when the background is bright or uneven; the first two rows are the decision rules worked through below.

What threshold separates found from not found with CV_TM_SQDIFF?

Normalize minval into a per-pixel RMS gray-level error and accept at or below 8 gray levels on a 0–255 image: rms = sqrt(minval / (w × h)). At a lit, fixed station, present parts land between 2 and 6 and empty scenes sit above 15.

The division by pixel count is what makes the number usable: a raw minval grows with template area, so it cannot be compared between templates, or reused on the same template after a resize. RMS error in gray levels is size-independent — 8 means an average per-pixel deviation of 8 levels whatever the template size.

The decision in code, with the API calls and your own variables:

int tplW = 120;              /* template width, pixels */
int tplH = 90;               /* template height, pixels */
double thr = 8.0;            /* RMS accept limit, gray levels */
double minval = 0.0, maxval = 0.0;
CvPoint minloc, maxloc;

cvMatchTemplate(image, templ, res, CV_TM_SQDIFF);
cvMinMaxLoc(res, &minval, &maxval, &minloc, &maxloc, 0);

double rms = sqrt(minval / ((double)tplW * (double)tplH));
bool found = (rms <= thr);

Two tests prove the chain: crop the template out of the search image itself and rms must print 0.0; run an image of the empty fixture and rms must exceed 8. If either test fails, the template dimensions or the result buffer are wrong, not the threshold.

Worked example. Search image 1600 × 1200 pixels, template 120 × 90 pixels, so N = 120 × 90 = 10,800 pixels.

• Part present: minval = 432,000 → 432,000 / 10,800 = 40.0 → sqrt(40.0) = 6.3 gray levels. 6.3 ≤ 8, found at minloc.
• Fixture empty: minval = 3,240,000 → 3,240,000 / 10,800 = 300 → sqrt(300) = 17.3 gray levels. 17.3 > 8, not found, and the rectangle stays undrawn.
• Runner-up position: minval = 1,728,000 → 160 → 12.6 gray levels, four times the best minval, so the part appears once and the position is unambiguous.

The two groups sit at 6.3 and 17.3, so any cut between 8 and 12 separates them; 8 leaves headroom for lamp aging and dust, which push the present-part score upward.

What score to require from CV_TM_CCORR_NORMED

Accept the peak only at 0.90 or above on 8-bit images. A true match under stable lighting reaches 0.95 and higher; a background-only scene stays below 0.85 in a 50/50 log at a fixed station. The value 0.90 sits in the gap between them.

The normalization removes the overall brightness scale between template and patch, but it does not remove the mean. A bright, flat region — white conveyor, bare metal — correlates strongly with any template and produces a high peak with no part in view. That is the classic false accept of pure normalized correlation. When the background is bright, switch to the mean-subtracted coefficient family from the table above, or stay with squared difference, and re-derive the cut from your own score log. On a smart camera the same accept score is set in the device's match tool; configuring golden template match results in a vision VI walks through those result settings.

How to code the found/not-found decision

Work it as seven steps in this order — convert, match, read, normalize, threshold, check uniqueness, publish. Each step feeds the next, and skipping the normalization steps is what makes a threshold drift between templates.

1. Make the images comparable: same bit depth, grayscale, exposure locked. A mismatch here fails before any score exists — template matching incompatible image type covers that failure.
2. Run the comparison into a 32-bit float result buffer sized (W − w + 1) × (H − h + 1).
3. Read the extremum your family defines: minimum for squared difference, maximum for normalized correlation.
4. Normalize: RMS gray-level error for squared difference; the score is already 0–1 for the normalized correlation family.
5. Threshold: rms ≤ 8 gray levels, or peak ≥ 0.90.
6. Check uniqueness: the runner-up must sit far from the winner — four times the best minval in the worked example. Two close peaks mean two parts, a symmetric feature, or periodic texture; for nested or repeated features, child template matching configuration covers the multi-match set-up.
7. Publish the decision stamped with the frame it came from; asynchronous vision result frame matching keeps a pass tied to the right part.

How to prove the threshold before release

Log the normalized score for 50 known-present and 50 known-absent scenes and demand complete separation between the two groups, with the threshold placed near the middle of the gap. Overlapping distributions mean no threshold can work and the imaging must change first.

Plot both groups as a histogram. In a healthy set-up the present group clusters tightly — the 2-to-6 band of the worked example — and the absent group sits several times higher, with empty space between them. Place the cut in the middle of that empty band, not at its edge: a cut hugging the present group false-rejects the day the light dims or dust settles on the lens. If the groups overlap, fix the light, the focus or the template choice before touching the threshold; no score rule separates what the image did not separate.

When does a passing score still lie to you?

Four classic ways: bright flat backgrounds fool pure correlation, rotationally symmetric parts match flipped 180 degrees, a template nearly as large as the search image leaves too few offsets to trust, and drifting focus or scale quietly walks a present part's score across the line.

• Bright background: pure normalized correlation can peak near 1.0 with no part present. Use the mean-subtracted family or squared difference.
• Symmetric parts: a part that is its own 180-degree rotation passes at the flipped position. Pattern match flips 180 degrees gives the tolerance settings that stop it.
• Template size: a template too small for the part's texture gives noisy, unstable scores — the template image too small error case — while a part that overfills the view raises the opposite problem covered in template matching a part larger than the field of view.
• Drift: scale and focus change slowly, so the present-part RMS rises a little each week until good parts cross the cut. Re-run the 50/50 log after any mechanical or optical change.

Once a match is accepted, refine its position rather than re-matching: subpixel peak localization explains the fitting step.

What hardware keeps the threshold stable?

A fixed-focus, fixed-aperture lens on a mono camera under constant light holds the present-part RMS in a narrow band, so one threshold value stays valid in production. Automatic focus, iris and exposure each walk the score from week to week.

The camera only needs to be repeatable, not exotic: a mono GigE area scan camera with a rolling shutter costs nothing here, because the part and camera stay static during exposure. In front of it, the LS-FFL-0679 is a 35 mm 2/3" 6 MP C-mount machine vision lens — teach it once and lock the focus ring at that distance. Browse the camera finder, area scan cameras and machine vision lenses for the pair that fits your field of view.

Matching can also run on the device, with no PC in the loop: the LS-SMC-0017 is an 8.8 MP smart camera with a global shutter on the X86 platform, and the accept threshold then lives in the camera while the controller receives only the result bit. What the operator sees on the panel is a separate matter — smart camera image in a PLC HMI explains what actually updates live — and the wider smart cameras range carries the same decision logic in a fixed housing.

If your 50/50 log shows overlapping groups and the light, focus and template all look right, send the log and two sample images to the applications desk; a look at the score distribution usually finds the imaging step that moved.