Fiducial detection FPS drops with tag in view: profile the decode step

A frame rate that holds steady on empty frames and collapses the moment a marker enters the field of view is not a camera problem. It is the signature of a fiducial detector sitting inside the acquisition loop: empty frames never reach the expensive code, tagged frames pay for it on every capture. This page is for engineers wiring marker detection into a vision cell on an embedded ARM board or a compact x86 host. You will time each stage, separate camera limits from decode limits, and size the compute from a number you measured instead of a guess.

Why the FPS falls only when the marker is in view

Marker libraries decode in two stages. The detector first finds candidate quads using an adaptive threshold and connected components; the decoder then samples each quad and runs error correction. On an empty frame there are no candidates, so the decoder never runs and the loop cost is grab plus preprocessing. The moment a tag enters the field, quad detection and decode execute on every frame, and the loop time jumps by tens or hundreds of milliseconds.

That work sits in the same thread as the grab, so the camera stalls while the CPU decodes. It is the same main-thread blocking that freezes GUIs during image capture, wearing a detector instead of a display.

The frame budget is arithmetic:

A 190 ms decode path caps one core at 1000 / 190 = 5.3 detections per second, whatever the camera streams. This is the gap between acquisition and processing, with the numbers moved to the decode stage.

Rule out the camera link first

Before profiling code, confirm the camera still delivers its rate with the detector disabled. Measure the delivered frame rate over a few thousand frames with detection commented out; how to measure camera frame rate in Python shows the timing loop. If the delivered rate already misses target, the detector is innocent and the table points elsewhere.

Auto-exposure is the classic impostor: exposure priority trades frame rate for scene changes, and the mechanism is in auto exposure priority: why your frame rate drops to half, with the GigE variant in why your GigE camera frame rate drops at longer exposure times.

Link problems sit on the other side of the table: a USB hub that drops the camera frame rate and BGR versus Bayer pixel format on GigE cap the stream before any code runs, and sustained drops after minutes of load point at heat, as in why sustained recording FPS drops on an embedded vision box. The systems topic page collects the rest of the frame-rate fixes.

Profile the decode step stage by stage

1. Save one reference frame. Capture a frame with the marker centered at the working distance and feed every stage from that saved image, so camera jitter cannot pollute the timings.
2. Time each stage separately. Wrap grab, color conversion, preprocessing, quad detection and decode in timing prints. Take the median over 100 frames and discard the first ten, which pay cache and allocator warm-up.
3. Build the library on the target with debug symbols. A wrapper compiled on a laptop can carry different optimization flags and SIMD paths than the build on the board, so benchmark the build you ship.
4. Attribute the time with a sampling profiler. On Linux, run the perf profiler against the process; the hot-spot report names the function that owns the milliseconds.
5. Compare traces between machines. Produce a single-frame debug trace on the fast host and on the slow board, then check that both decode the same tags from the same image. Different results mean a broken build, not a slow one.

If the wrapper is Python, watch for preprocessing written as per-pixel Python loops; the rewrite in slow Python OpenCV code: fix per-pixel loops that kill FPS often recovers more time than the decoder ever cost.

Worked example: a 5 MP frame on an embedded ARM board

Setup: a 5 MP camera delivering 2592 × 1948 frames, requested at 30 FPS. The frame budget is 1000 / 30 = 33.3 ms. Median stage times on the board, marker in view:

A 200 ms frame gives 1000 / 200 = 5 FPS, which is the collapse. With no marker in view, detection and decode are skipped: 10 + 12 + 3 = 25 ms, so 1000 / 25 = 40 FPS. The camera never changed; the code path did.

The decode stage alone breaks the budget. At 110 ms it caps the loop at 1000 / 110 = 9.1 FPS even if every other stage cost nothing, so no camera or cable change reaches 30 FPS while decode runs inline.

The remaining fix moves decode out of the grab path. Run detection on a worker thread that always processes the newest queued frame: the grab loop falls back to 10 + 12 + 3 = 25 ms, under the 33.3 ms budget, so the camera keeps its 30 FPS, while detections complete at 1000 / 190 = 5.3 per second. For a pick cell that triggers on part arrival, 5.3 Hz guidance is comfortable. For a 30 Hz tracking loop it is not, and no thread trick changes that.

Fixes in the order that pays

1. Downscale before detection. Half linear scale removes three quarters of the pixels; quarter scale removes fifteen sixteenths. Confirm decode still succeeds on your smallest tag at the working distance before you commit.
2. Crop on camera. If the marker lives in a known region, set sensor ROI or decimation and the pixels never travel the link at all. The sensor side of that trade is in 4x4 binning vs 2x2: sensor readout decides your frame rate.
3. Decode off the capture thread, newest frame only, so the camera runs at its delivered rate.
4. Decode every Nth frame when guidance tolerates it: at a 30 FPS stream, decoding every 6th frame issues 5 attempts per second, just inside a 5.3 per second decoder's capacity.
5. Change silicon when the rate requirement exceeds the decode ceiling; that decision is the next section.

Camera choice carries the same logic. Fiducial guidance rarely needs more than about 2.5 MP, and any motion demands a global shutter; the MV-CU020-90GC is a 2.5 MP GigE camera with a global shutter, a 1/1.7" sensor and a C-mount. Half the pixels of the worked example means roughly half the conversion and search time before any downscaling.

Size the front end from the area scan cameras list, filter by shutter and interface in the camera finder, or let the camera configurator walk sensor, resolution and interface in order.

Size the compute from the measured decode time

At 30 FPS the budget is 33.3 ms, so decode must measure at or below 16.7 ms to leave half a core free for the grab loop and the rest of the inspection.

The decision for the worked example: the board decodes a 1/2-scale frame in about 110 / 4 = 28 ms, which clears the 100 ms limit of a 5 Hz guidance loop with room to spare. Keep the board, keep the downscale, keep the worker thread, spend nothing. A 30 FPS tracking loop needs 16.7 ms, and no low-power ARM core reaches it on this workload; that case moves to x86.

Two tiers cover the controller range. The MV-VC3301H-128G66 pairs an Intel Core i3-8100 with 8 GB of memory, 5+ LAN ports, four built-in light channels and a VM6200Pro license included, the tier for a decode-heavy cell that also drives lights. Where decode shares the core with several cameras or an inference model, the MV-VC5525-256G70-NN brings a 12th-generation Intel Core i5, 32 GB of memory, PoE camera ports, 8 built-in light channels and PCIe expansion supported.

Expect a desktop x86 core to run a single-threaded C decoder several times faster than a low-power ARM core at the same image size; the gap comes from clock and IPC, not from the library. Run the same stage-timing benchmark on the controller before you commit the number to the cell. If the board choice itself is still open, choosing an ARM board for USB3 camera acquisition weighs the USB3 and CPU budget first, and the vision controllers range starts where the boards stop.

When this diagnosis is wrong

Three patterns break it. The rate falls on empty frames too: decode is not the bottleneck, and the exposure, bandwidth and throttling rows of the table above are the suspects. Both machines decode the same tags but at different speeds after an onboard rebuild: the compilers used different flags, so compare profiler hot-spot lists, not frame counts. The profiler points at your own preprocessing rather than the library: vectorize that code, using the per-pixel loop rewrite linked in the profiling section, before you buy silicon.

Prevention checklist

• Lock exposure and gain to manual values in any triggered loop; auto modes trade your frame rate for scene changes you did not ask for.
• Measure the delivered camera rate with the detector disabled before you promise a rate to the line.
• Benchmark the decoder on the target hardware with debug symbols, never on a development laptop.
• Downscale or crop to marker scale; never decode pixels you do not need.
• Keep detection off the grab thread and queue only the newest frame.
• Leave half of one core free at the target rate so the loop survives a busy system.
• Re-run the stage timings after every library or OS update; hot spots move with the build.

The decisions, one line each: guidance at 5 Hz with a measured 28 ms half-scale decode stays on the ARM board with the downscale and the worker thread. Tracking at 30 FPS needs decode at 16.7 ms or less, which means x86; start at the Core i3 tier and step to the Core i5 tier when decode shares the core with more cameras or inference. If you want a second pair of eyes on the numbers, the contact page reaches the applications team.