Live edge detection from a video stream: wiring the grab loop

You snapped a single frame from the camera, tuned an edge tool on it, and it found the transition every time. Wired to the live grab, the same tool returns nothing, throws an error, or freezes the program — and the instinct is to fight the algorithm. That instinct is wrong almost every time. The edge tool is fine; the wiring is not: which image reference the tool receives, which rectangle it searches, and where the call sits in the loop. This guide walks those faults in probability order, gives a loop skeleton that works with any camera or frame grabber, and runs one worked example with real numbers — pixel scale, rectangle geometry and the per-frame time budget.

The answer: one frame, one wait, one edge call

Run the acquisition continuously, and inside one per-frame loop do three things in order: wait for exactly one frame, pass that frame's image reference and one correctly ordered search rectangle to the edge tool, then draw the result on a copy. Every failure below is one of those three inputs being wrong.

An edge tool is a simple consumer. It projects the pixels inside a search rectangle onto one axis and reports the strongest polarity transition along that profile — dark-to-light or light-to-dark — with a position in pixels and a strength score. It holds no state and keeps no time: it processes whatever image and whatever rectangle arrive on its inputs, once per call. That is why the tool can be flawless on a snapped frame and useless live. Live, the inputs come from your loop, and your loop is the bug.

Symptoms and causes, in probability order

Work this table top to bottom. Stop at the first test that fails, apply the fix, and re-test before moving on.

Rows 1 to 3 account for nearly all first-integration failures, and they share one root. The classic debug path — snap one frame, tune the tool in an assistant, generate the code — proves the tool and never writes the loop: the generated code starts from a filled buffer, your live code starts from an empty one. For the drift in row 4, contour position jitter under automatic exposure works through the exposure half of that fault.

The loop that works

The skeleton below fixes rows 1, 2 and 6 at once. It assumes continuous acquisition into a ring of buffers, which every major camera SDK and frame grabber supports.

setup: open device, set exposure and trigger, start continuous acquisition
loop:  frame = wait for next frame   (timeout = 2 x frame period)
       if frame is empty: log it, continue
       edge = edge tool(frame, search rectangle)
       copy = duplicate(frame); draw(copy, rectangle, edge)
       publish edge position and strength
exit:  stop acquisition, then close the device

In wiring order:

1. Initialize once, outside the loop. Opening the device, setting exposure and starting acquisition belong before the first wait; repeating them per iteration stalls the stream and leaks handles. The pattern is the one described for continuous camera acquisition in live image loops.
2. Wait for exactly one frame per pass. A second wait steals the following frame and produces the classic timeout — one wait call per frame covers the diagnosis.
3. Hand the tool that image reference untouched — no crop, no copy, no display buffer between them.
4. Hand it one rectangle: left smaller than right, top smaller than bottom, inside the frame, in pixels.
5. Gate on strength. Log the strength of a known good edge across 100 frames and accept only edges above half the lowest value logged.
6. Draw on a copy and publish the position. The copy, never the grab buffer.

The wait call is the one-shot of this pattern: the tool fires once per new frame, not on every processor cycle, the same way a PLC action is gated to a rising edge so it runs once per scan — the discipline behind positive edge detection that stops continuous PLC execution.

The loop shape does not change with hardware: a GigE camera SDK, a USB3 camera and a Camera Link or CoaXPress grabber all expose the same wait-for-frame primitive, as frame grabber image acquisition for Camera Link and CoaXPress shows. For a multi-camera GigE cell, one 4-port GigE card covers the cell:

Browse frame grabbers by interface when the camera side is already fixed.

Worked example: an edge position from a 4000 × 3000 stream at 30 fps

Inputs. The camera is the LS-CMR-0166, a 12 MP USB3 camera delivering 4000 × 3000 pixels at 30 fps. The part sits in a fixture, 60 mm wide, under a lens set for a 64 mm horizontal field of view. The edge of interest is a single dark-to-light transition crossing mid-frame.

Rules. Pixel scale = field of view ÷ pixel columns. Position on the part = (edge x − rectangle left) × pixel scale. The loop holds rate while the edge call plus overlay stays under the frame period.

Arithmetic:

• Pixel scale = 64 mm ÷ 4000 px = 0.016 mm/px, or 16 µm per pixel.
• Rectangle: left 800, top 1450, right 3200, bottom 1550 — 2400 px wide, 100 px tall, ordered correctly, one transition inside.
• The tool returns x = 1234.6 px. Inside the rectangle that is 1234.6 − 800 = 434.6 px, and on the part 434.6 × 0.016 mm = 6.95 mm, so 7.0 mm from the rectangle's left edge.
• Frame period = 1000 ÷ 30 = 33.3 ms. The projection covers 2400 × 100 = 240,000 samples; a compiled edge tool clears that in single-digit milliseconds on one x86 core, so budget 8 ms for the call and 2 ms to draw and publish.
• Total 10 ms against the 33.3 ms budget, so the loop sustains 30 fps with headroom for the PLC handshake.
• Repeatability: a clean transition localizes to 0.1–0.3 px on this sampling, 2–5 µm at the part; edge position accuracy from image sampling derives the bound.

Decision. With 10 ms of a 33.3 ms period spent, free-running continuous acquisition is the right mode for this station. If a larger rectangle or a slower CPU pushes the edge call to 50 ms, the ceiling drops to 1000 ÷ 50 = 20 fps and frames back up: shrink the rectangle or accept every second frame, the trade laid out in acquisition faster than processing and ROI versus binning versus subsampling.

The search rectangle: five rules

The rectangle is where live integrations die, because a malformed one fails silently — most tools return an empty edge list, not an error.

1. Order the coordinates: left strictly below right, top strictly below bottom. If the call takes x, y, width and height instead, width and height must be positive, never zero.
2. Think in pixels. Calibrated millimetre coordinates belong to a transform applied after the tool returns its pixel position.
3. Keep it inside the frame. Clamp right and bottom to the image width and height minus one pixel; a rectangle one pixel over the border can return nothing.
4. Put one transition in it. Set the polarity — dark-to-light or light-to-dark — and the projection axis explicitly; leave either on auto and a second transition wins on some frames.
5. Build it once. Compute the rectangle before the loop for a fixtured part, or from a locate step for a moving part. Per-frame arithmetic on a stationary part is how drift starts.

When a grab loop is the wrong answer

Three situations send you elsewhere:

• The output is a position or a pass/fail for a PLC, not images for an operator. A device from the smart cameras range runs the edge tool on the sensor and reports the result over Ethernet; there is no grab loop, no image reference and no timeout to wire.
• Parts arrive one at a time at low rate. Trigger per part instead of free-running — one frame per trigger, one edge call — and hardware trigger that stops after one frame covers the acquisition-mode setting that makes it work.
• The task is finding shapes, not measuring transitions. Thresholding followed by contour analysis fits that job better; contour analysis of a binary image and live binary preview for defect inspection walk that loop.

The on-device route replaces the whole loop with one part:

That model is a 6 MP mono smart camera with an 8 mm focal length in an M12 mount, on GigE — the edge tool, the display and the Ethernet result all live in the housing.

Checks before you call it done

• Save and open one image exactly as the tool receives it.
• Block the lens and watch the saved frames go black.
• Overlay the rectangle on a saved frame; it covers exactly one transition.
• Timeout equals two frame periods; one wait per pass; three consecutive skips raise an alarm.
• Strength gate set from 100 logged frames of a known good edge.
• Display draws on a copy; the grab buffer is untouched.
• Acquisition stops and the device closes on shutdown.
• Ten minutes at rate with zero timeouts in the log.

Once the loop runs clean, the station is ready to grow — the same skeleton takes a second tool or a better sensor without redesign. Compare sensors in the camera finder, pick from area scan cameras when the task outgrows this one, or start a build at the camera step.