Pattern matching overlay not showing: draw the results on the image

You configured the match, it runs, the score looks right — and the output panel shows a clean image with no box anywhere. This page is for anyone who has moved a vision job from a point-and-click configuration tool into code, or who is wiring match results to a display, an HMI or a robot. In ten minutes you will know where the results actually live, the one drawing step that puts them on the image, the two geometry bugs that misplace boxes, and the test that proves the overlay before anything moves.

Why does a good match still draw nothing on the image?

Because the matching step writes its hits — position, angle and score — into a separate results list, while the image you display is still the untouched pixel buffer, and no library draws on an image unless a later overlay or annotation step does it.

The pipeline makes two different outputs on every cycle. Pixels go into the image buffer; the matcher writes its hits into a results list: centre, angle, score, and the template size it matched. The display, the file writer and the network stream all consume pixels only. A score of 0.94 sitting in that list proves the match ran, and proves nothing reached the picture. Read the list before you touch anything else:

Where did the overlay go after exporting the job to code?

Nowhere in your code: a point-and-click vision tool draws those boxes inside its own viewer while you configure, and the exported program reproduces only the processing steps, so the display-only overlay is dropped and you add it back yourself.

Configuration viewers are debug tools. While you build the job, the tool's own viewer draws match rectangles on every test run so you can see what you are doing. The exported program is a description of the processing only — acquire, match, output — and the viewer stays behind with its drawing. Two things therefore go missing at once.

1. The draw step. Nothing in the exported code ever touches the image for display, so you add the overlay call yourself (next section).
2. Sometimes the match step's helper library. If the runtime cannot find the algorithm module the tool shipped, the match step errors out at load and the results list never fills. Check for load errors on the matching subroutine before you chase the overlay.

Install or link the library first, then add the drawing step.

How do you draw match results onto the output image?

You add one drawing step after the matching step: read the match list, build a rectangle from each hit's centre, angle and template size, draw it in pixels on a copy of the image, then display, stream or archive that copy.

Five calls, in this order:

1. Acquire one frame into an 8-bit monochrome buffer — 2448 × 2048 px in the worked examples below.
2. Run the match with a minimum acceptance score; 0.80 is a workable start on a clean part. Keep the returned list.
3. Copy the frame. Draw on the copy, never on the acquisition buffer.
4. For each hit, compute the axis-aligned box from the template size and the hit angle (next section), then call your library's rectangle overlay. Every machine-vision library has one, whether it is named overlay, annotation or shape drawing.
5. Display, stream or archive the copy, and clear the overlay layer at the top of each cycle if your library keeps layers between frames.

# Illustrative pseudocode: your library's function names differ.
img: uint8[2048][2448] = acquire # one mono frame
hits: list[Match] = match_pattern(img, template, min_score=0.80)
disp = copy(img) # draw on the copy, never on img
for m in hits: # m: centre x, y in px; angle deg; score 0.0-1.0
 bw = abs(m.w * cos(radians(m.angle))) + abs(m.h * sin(radians(m.angle)))
 bh = abs(m.w * sin(radians(m.angle))) + abs(m.h * cos(radians(m.angle)))
 draw_rectangle(disp, m.x - bw/2, m.y - bh/2, bw, bh, 'green')
display(disp)

Why is the drawn box too small for a rotated part?

Because a match at 32° needs an axis-aligned box of about 150 × 140 px for a 120 × 90 px template, not 120 × 90 px: the rotated footprint is larger than the template, and the overlay rectangle must be computed from the angle, never copied.

Overlays draw axis-aligned rectangles in image coordinates, and the match angle rotates the part's footprint under that fixed frame. The box you must draw is the rotated template's bounding box, from template width w_t, height h_t and hit angle θ:

Worked example, with the 120 × 90 px template from the code above matched at θ = 32°. cos 32° = 0.848, sin 32° = 0.530:

• w_box = 120 × 0.848 + 90 × 0.530 = 101.8 + 47.7 = 149.5, so draw 150 px
• h_box = 120 × 0.530 + 90 × 0.848 = 63.6 + 76.3 = 139.9, so draw 140 px

Reuse the unrotated 120 × 90 box and it covers 10,800 px² of the rotated footprint's roughly 21,000 px², just over half the part. The match was never wrong; the box was. If your library offers a rotated-rectangle primitive, draw that and skip the arithmetic. And when hits alternate between θ and θ + 180°, the angle convention is the fault, not the overlay: fixing 180° flipped pattern matches covers that case.

Why does the box land far from the part on a calibrated set-up?

Because the match list and the overlay function are working in different units: with a 0.05 mm/px scale, a hit at pixel 1800 sits 28.8 mm from the calibration origin, and 28.8 drawn as pixels lands the box at pixel 29, 1771 px from the part.

Two coordinate spaces meet at the overlay. A calibrated job often reports hits in millimetres from the calibration origin, while the rectangle overlay consumes pixels. Convert once, with one pair of formulas:

Worked example on the same 2448 px-wide frame. The calibration maps a 122.4 mm field of view across the sensor, so the scale is s = 122.4 mm / 2448 px = 0.05 mm/px, origin at the image centre, cx = 1224 px. A hit at pixel x = 1800 converts as world_x = (1800 - 1224) × 0.05 mm/px = 576 × 0.05 = 28.8 mm. Hand 28.8 to a pixel-space overlay and the box draws at pixel 29, pinned at the frame's left edge and 1771 px from the part (1800 - 29).

Convert back first and the arithmetic closes: px = 1224 + 28.8 / 0.05 = 1224 + 576 = 1800, and the box lands on the part. Label which space each list carries — pixel lists from uncalibrated jobs, millimetre lists from calibrated ones — and route both through one conversion function. The wider camera-to-robot chain is chaining pixel-to-world transforms in eye-in-hand set-ups.

Why does the overlay blink, vanish or drag the score?

Because one of three buffer mistakes is active: the box was drawn once on an overwritten frame, it was burned into the image the matcher reads, or the display layer is repainting against the wrong buffer — each with a one-line test and a one-line fix.

• Drawn once, gone next frame. You drew on the acquisition buffer and the driver overwrote it on the next grab. Test: freeze the trigger and the box persists; release it and the box vanishes. Fix: redraw every cycle on a display copy.
• Boxes flicker and scores wobble. The annotation is burning into the image the matcher reads, so its synthetic edges feed back into the next match — the same feedback family as auto-exposure ROI instability caused by detection feedback. Fix: copy first, draw on the copy; image bytes versus pixel data covers the buffer discipline.
• Flicker on the live view only. Some runtimes repaint the display layer against the wrong buffer when the window resizes or the frame rate changes; blinking-image-overlay failure and its fix walks through that display-layer case.

Speed is rarely the problem: a few dozen 2 px rectangles cost far less than the match itself, which takes tens of milliseconds on a 6 MP frame. If cycle time grows when you add overlays, suspect the full-image copy or the display transfer first.

How do you verify the overlay before the robot uses the coordinates?

Present the template part at a known pixel position and check three numbers: the results list count equals the drawn box count, each box centre sits within 2 px of the known position, and the archived frame carries the boxes, not just the live screen.

Run this five-minute test on the bench:

1. Place a part with its centre at a measured pixel — read it from the live crosshair, for example (1800, 900).
2. Trigger one cycle and log two numbers: hits in the results list and boxes drawn. They must be equal.
3. Log the drawn box centre and accept it only within 2 px of (1800, 900). At the 0.05 mm/px scale above, 2 px is 0.1 mm, comfortably inside a ±0.5 mm pick tolerance.
4. Rotate the part by about 30° and repeat. The box must grow to the rotated size and stay centred on the part.
5. Save the annotated frame to the archive and open the file. The boxes must be in the pixels, not only on the screen.

Leave these running in production: results count and drawn count in every log line, one annotated frame per shift or per rejected part, and the score and angle printed beside each box. When the coordinates move on to motion, using eye-to-hand calibration results to compute robot-base picks continues at the transform, and debugging object detection output coordinates covers the logging side.

When is a blank display a real matching failure instead?

When the match list itself is empty: scores below the acceptance threshold, a template that flipped 180°, or lighting drift produce no hits to draw, so no overlay call can help and the fix belongs to the matcher, not to the display.

Print the results count; that one number settles it.

For the empty-list case, work the matcher. Drop the acceptance score from 0.80 to 0.60 and watch what appears: weak hits arriving means the template or the lighting is marginal, not the threshold. Glare and contrast drift are the usual root cause, and edge-based pattern matching for stable results when lighting changes is the durable fix.

Should overlays run on a smart camera or on a controller?

It runs wherever the matching runs: on a smart camera the job's own display step draws the boxes as a setting, while a camera-plus-controller build draws them in your process, right after the match call, on hardware you specify.

For a single fixed camera, an on-device smart camera is the shorter path: the job already ends in a display step, so the result overlay is a setting you enable rather than code you maintain.

When the station grows to several cameras, custom geometry or a calibrated hand-off to a robot, the matching and the drawing move into your own process on a vision controller — filter the candidates by LAN port count, PoE camera ports and light-output channels to match your camera count.

Whichever side you choose, the pipeline is identical — match, copy, draw, output — and so is the debugging. Starting from the camera side, area scan cameras with C-mount lenses keep the pixel geometry predictable, and the systems topic page collects the rest of this failure family. For an unusual set-up, describe the job on the contact page and we will check the overlay and calibration plan with you before you order.