Pattern matching low fps: shrink the search to speed the update rate

A positioning loop that reports one update per second turns live tracking into a slideshow, and the first instinct — buy a faster camera — is usually the wrong one. In a slow pattern matching loop the camera sits idle for most of that second while the matcher sweeps a template across the full image, row after row. The fix is to measure where the second actually goes, then cut the area the matcher searches. If you are wiring an area scan camera to a positioning or tracking task, run the five-minute triage below, apply the two search fixes, and learn the one case where hardware really is the answer.

Split the loop first: grab time versus search time

Every loop iteration does three things: acquire a frame, find the pattern, run your code. The total per frame sets the update rate, so measure the pieces instead of guessing. Run acquisition alone — grab and discard, no matching — and log the rate over a few hundred frames. Then run the full loop and log that rate. The difference between the two rates is search time.

The split decides everything. If search time is more than 80% of the loop, software is the problem and the two fixes below will move the number. If grab time alone approaches the loop period, you are acquisition-limited and only hardware helps. Take a loop that reports 1 fps while the grab-only loop runs 30 fps: the grab costs 1/30 = 0.033 s, so search owns 0.967 s of every 1.000 s. That is 97% software, 3% camera.

Exposure sits inside grab time: at a 30 fps cap the exposure cannot exceed 33 ms, and shortening it costs no frame rate. Measure with timestamps around each stage rather than by reading a reported frame-rate property, which can return zero or a stale value — reading frame rate reliably shows why. Keep the camera streaming continuously; a one-shot grab per iteration re-pays set-up latency every loop, and trigger call versus exposure time shows how the trigger and the exposure window interact. If grabs stall outright, start with diagnosing a camera frame acquisition timeout.

Likely causes, ranked by how often they explain 1 fps

Ranked by how often they explain a one-update-per-second loop:

1. Full-frame search. The matcher sweeps the template across every position in the image. Test: restrict the search region to a quarter of the frame; loop time should fall to roughly a quarter.
2. Search step too fine. Testing every pixel position when every fourth would do. Test: set the step to 4 px and re-time; the sweep cost drops 16-fold.
3. Oversized template. Match cost scales with template area as well as search area, and whole-part templates carry background instead of features. Test: re-teach on the smallest distinctive patch — the template-too-small error and match region sizing article covers that balance.
4. Full-resolution handling upstream. Decode, colour conversion and buffer copies on a 12 MP frame each cost tens of milliseconds. Test: bin or subsample to half resolution; the data drops fourfold.
5. Acquisition ceiling. Rare on an industrial camera at 1 fps, common with webcam-class sources or a 12 MP sensor on GigE. Test: the grab-only rate from the triage; USB3 low frame rate and latency fixes covers the USB side.

Fix 1: search a region of interest around the last position

The target barely moves between two frames of a fast loop, so search near where it was. Hold the last matched position, build a search region centred on it, and match only inside. Size the region in pixels: the template width, plus twice the largest motion you expect per frame, plus a margin of about half a template width per side. Convert motion to pixels first:

with v the part speed in mm/s, t_frame the loop period in seconds, and s the image scale in px/mm. A part moving 150 mm/s, looped at 30 fps on a 10 px/mm image, moves 50 px per frame. The speedup from restricting the search is the area ratio:

Two guards keep it honest: reacquire over the full frame whenever the match score drops below your threshold, and run a full-frame search every Nth frame — every 30th is a workable compromise — so an undetected miss cannot latch. The frame-rate arithmetic for multi-ROI capture applies the same area logic when the camera crops several regions itself.

Fix 2: step the first pass and shrink the template

Two parameter changes multiply with the ROI. First, the search step: most match tools test every pixel position by default and can be set to test every second, fourth or eighth. A step of 4 px in both axes evaluates 16× fewer candidates; the tool then refines around the best candidate in a small window. Pyramid or coarse-to-fine matching packages the same idea — a half-resolution first pass costs fourfold less and lands within a few pixels, then the fine pass confirms at full resolution.

Second, teach the template on the smallest region that is still distinctive: an edge junction, a corner, a printed mark. Match time scales with template area, and large low-texture templates run slow and drift when lighting shifts. Patches of roughly 64-200 px on a side are the usual working range; below that, false matches climb and scores turn noisy.

Worked example: ball tracking on a 400 mm field, from 1 fps to 28 fps

Inputs, with their source:

• Field: 400 mm × 300 mm white board, target a 30 mm ball — the bench case of a ball moved by hand over a paper field.
• Camera: 12 MP, 4000 × 3000 px, 30 fps ceiling over USB3 — catalog figures for the 12 MP USB3 camera.
• Measured loop: 1.00 s per update — the reported symptom.
• Grab-only rate: 30 fps, so t_grab = 1/30 = 0.033 s.
• Ball speed: up to 150 mm/s, moved by hand.
• Template: the 30 mm ball, 300 px across at the scale computed below.

The rule: search only inside a ROI around the last position, sized as template + 2 × per-frame motion + margin, then step the first pass.

1. Scale: 4000 px / 400 mm = 10 px/mm, so the 30 mm ball is a 300 px template.
2. Motion per frame at the current 1 fps would be 150 mm/s × 1.00 s = 150 mm — wider than any sensible ROI, which is why a ROI designed at the current rate cannot work. Design at the target rate: at 15 fps, motion is 150/15 = 10 mm = 100 px per frame.
3. ROI side = 300 (template) + 2 × 100 (motion) + 2 × 50 (margin) = 600 px, rounded to 640 × 640 px.
4. Area ratio: (4000 × 3000) / (640 × 640) = 12,000,000 / 409,600 = 29.3. Search time falls from 0.97 s to 0.97 / 29.3 = 0.033 s. New loop: 0.033 + 0.033 = 0.066 s, about 15 fps.
5. Step the first pass every 4 px: candidate positions drop 16-fold, so the sweep costs 0.033 / 16 ≈ 0.002 s, and the refine pass over ±4 px around the winner adds about 0.001 s. Search is now roughly 0.003 s.
6. New loop: 0.033 + 0.003 = 0.036 s → 1/0.036 ≈ 28 fps.

Decision: grab time is now 0.033 / 0.036 = 92% of the loop. The loop is acquisition-limited, so more code tuning buys nothing: 28 fps is the ceiling until the camera changes, and 30 fps is that camera's listed rate. If updates beyond 30 fps are required, the next move is the hardware section below, not the search parameters.

Two footnotes. The first frame after a missed match pays the old 0.97 s full-frame search — fine for rare escapes, intolerable as a routine. And the ROI was sized for 15 fps motion, so at 28 fps the same 150 mm/s hand moves only 150/28 = 5.4 mm ≈ 54 px, comfortably inside.

When the camera really is the limit

When the triage shows grab time filling the loop, or the task needs a shorter exposure than the sensor allows, hardware is the lever. Pull it in this order: cut the pixels you move, change the interface, then move the match onto the camera.

Cut the pixels the task really needs

Position to ±1 mm over a 400 mm field needs 1 px/mm at minimum — 400 px across a 400 mm field. Add a 2× factor for robust matching and 800 × 600 px does the job: about 0.5 MP, not 12 MP. Frame rate on a given interface scales roughly with inverse pixel count, so matching on a reduced or binned stream multiplies the ceiling before any purchase. The resolution calculator below turns a precision target into a pixel count.

Change the interface before the sensor

Two 12 MP cameras in the catalog make the point. The LS-CMR-0053 moves 12 MP frames over GigE; the LS-CMR-0166 moves the same frames over USB3, both on the IMX226 sensor with a rolling shutter. The catalog lists 9 fps for the GigE model and 30 fps for the USB3 model at 4000 × 3000.

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

Same sensor, 3.3× the rate: the interface alone buys it, before any software change.

Move the match onto the camera

For single-target tracking at modest resolution, a smart camera runs the match on the device and publishes coordinates, so transfer, decode and operating-system scheduling leave the loop entirely. The LS-SMC-0063 pairs a 1.6 MP sensor with a 60 fps rate, an integrated 16 mm lens and integrated white light over GigE. Its 1.6 MP is roughly three times the 0.5 MP the worked case needs, so the same optics cover the field with sampling to spare.

Check exposure against speed

A second ceiling hides in the shutter: motion blur must stay under a pixel.

The worked ball crosses 1500 px/s (150 mm/s × 10 px/mm), so exposure must sit at or below 1/1500 s ≈ 0.67 ms. That fits easily inside a 16.7 ms frame at 60 fps, but it rules out any camera that cannot shorten its exposure — which is why tracking fast parts pushes you toward high frame rate and short exposure together.

When the usual answer is wrong

ROI tracking fails in three predictable ways, and each has a counter-move.

The target jumps further than the ROI covers — pick-and-place handoffs, occlusion, start-up. The matcher misses, or worse, returns a confident match on background texture. Widen the ROI for those moves, reacquire on low score, and never publish a low-score position without flagging it.

Fast lateral motion meets a rolling shutter. The sensor reads row by row, so a fast target skews: the ball's circle becomes an ellipse and scores fall. Track fast parts on a global shutter sensor.

The template drifts. Lighting shifts lower the score gradually until the matcher latches onto whatever resembles the template. Set a score floor that gates the output and re-teach when scores trend down across shifts.

Downstream, the position loop cares about a steady rate more than a fast one. Bursty or stale feedback causes overshoot and hunting: choosing the update rate for a cascaded position loop treats that choice from the controls side, and what a slow PV update does to a tuned PID loop shows the failure mode when feedback crawls. A fixed-period publisher that drops late frames beats a variable-rate one that never drops any.

Prevention: run the loop to a budget

Pick the update rate from the motion requirement, then budget backwards. A loop that must publish every 50 ms gets: grab ≤ 20 ms, match ≤ 20 ms, code ≤ 10 ms. Log grab and match times every frame with a monotonic clock, and alarm when the match exceeds its share for more than a few frames in a row — that is how template drift and ROI escapes announce themselves.

Keep the checks that catch the problem in the first place:

• Grab-only rate and full-loop rate recorded after every template or ROI change.
• ROI sized from template + 2 × maximum speed at the target rate + margin.
• Full-frame reacquire on low score and every 30th frame.
• Search step chosen and timed, never left at defaults.
• Score floor gating the published position.
• Continuous acquisition confirmed — no one-shot grab per iteration.

If the triage lands acquisition-limited, the camera finder filters by resolution, interface and shutter, and the camera step of the configurator pairs the body with a lens. If it lands search-limited, no purchase fixes it — the fixes above move the number on the camera you already own. More loop-timing faults are collected on the systems topic page.