Interactive ROI selection: find the region a user clicked

Your viewer control draws regions over the live image, lets the user edit them, and then stays silent: there is no event that reports which region the user selected. The fix takes about thirty lines of code. On every mouse-down, convert the click from the control's client pixels to image pixels, test the point against each region in your own list, and apply one rule for overlaps. You created the regions, so the geometry is already in your program; the viewer owes you nothing more than the click point. This guide gives the conversion, the three containment tests (box, rotated rectangle, polygon), a worked example with the arithmetic shown, and the four mistakes that put a selection on the wrong region. The same loop works whether you draw the overlay yourself or use the viewer's region tools.

How do you know which region a user clicked when the viewer has no selection event?

You hit-test the click yourself: take the mouse-down point from the viewer, convert it to image pixels, and run it against every region in your own list — the first region that contains the point is the selection, however many regions you have.

A selection event is a convenience, not a prerequisite. The viewer already gives you the two inputs the test needs: a mouse-down event carrying the point in client coordinates, and the region definitions you supplied. If you already split one camera image into multiple ROIs for display, that region store is the input list. The work is two stages — a coordinate conversion, then a containment test per region — and neither touches the image data.

How do you convert a mouse click to image pixel coordinates?

One scale and one offset do it: image_x = (click_x - offset_x) / scale, where scale is displayed width divided by image width — at the 4:1 shrink used below, a click at control pixel (311, 206) lands on image pixel (1220, 800).

Scale is display pixels per image pixel; offset is where the image's top-left corner sits inside the control. Both live in the viewer's zoom-and-pan state, and if you draw the image yourself you set them. The subtraction removes the pan; the division undoes the zoom:

Keep the two scales separate if the control can stretch the image; with the aspect locked, scale_x equals scale_y. Click positions are pixel centres at integer coordinates and regions are defined the same way, so selection needs no half-pixel correction.

Worked example, carried through this guide: a 2448 x 2048 image shown in a 612 x 512 client area gives scale = 612 / 2448 = 0.25 in x and 512 / 2048 = 0.25 in y, with the image drawn at offset (6, 6). A mouse-down at (311, 206) converts as (311 - 6) / 0.25 = 1220 and (206 - 6) / 0.25 = 800.

How do you test whether a point is inside a box, a rotated rectangle or a polygon?

An axis-aligned box needs four comparisons (x_min ≤ x ≤ x_max and y_min ≤ y ≤ y_max); a rotated rectangle needs the point rotated into the rectangle's own frame by minus the rectangle angle; a polygon needs a ray-cast crossing count, where odd means inside.

The rotated-rectangle test is the box test after a change of frame: express the click relative to the rectangle centre, rotate by minus the angle, and compare against the half-width and half-height. Ray casting is the standard point-in-polygon test and costs one pass over the edges. Run the box test first as a prefilter; only regions whose bounding box contains the point earn the exact test.

What happens when a click lands in two overlapping regions?

Both containment tests return true, so a selection rule decides: take the smallest containing region — 10,000 px² beats 120,000 px² in the worked example — or take the region drawn last, which is the one the user sees on top.

Smallest-area matches what users mean. Nested regions usually mark a detail zone inside a parent, and clicking the child should select the child. Draw order works too, if the overlay paints regions in list order: the last one painted sits on top, the same stacking you control when you draw overlay lines at your own thickness. Define the rule once and use it for clicks and drag-starts alike, or the two will disagree on the same geometry.

What does the complete hit-test routine look like?

Wire six pieces together: a region store in image pixels, the mouse-down client point, the scale-and-offset conversion, box tests before exact tests, one overlap rule, and a returned region id that the display code formats.

1. Store every region in image pixel coordinates in one list, whatever the viewer draws on top.
2. On mouse-down, read the point in the control's client coordinates, not desktop coordinates.
3. Convert it with the scale and offset from the section above.
4. Loop over the regions: bounding-box pretest, then the exact test for the shape.
5. Collect the hits and apply the overlap rule: smallest area, or draw order.
6. Return the winning region id. To print that region's numbers, see how to display region width and height in one string.

// Illustrative names for your own code; the region store stays in image pixels.
struct Region {
    int Id;
    double Cx, Cy;     // centre, image pixels
    double W, H;       // full width and height, image pixels
    double AngleDeg;   // rotation in degrees
}

bool HitRotated(Region r, double px, double py) {
    double rad = r.AngleDeg * Math.PI / 180.0;
    double dx = px - r.Cx;
    double dy = py - r.Cy;
    double lx =  dx * Math.Cos(rad) + dy * Math.Sin(rad);
    double ly = -dx * Math.Sin(rad) + dy * Math.Cos(rad);
    return Math.Abs(lx) <= r.W / 2.0 && Math.Abs(ly) <= r.H / 2.0;
}

Proof with the worked numbers, where region A is a 400 x 300 rectangle centred at (1200, 800) and rotated 30°, and region B is the box x 1150-1250, y 750-850:

• Box test on B at (1220, 800): 1150 ≤ 1220 ≤ 1250 and 750 ≤ 800 ≤ 850, so B hits.
• HitRotated on A at (1220, 800): dx = 20, dy = 0; with cos 30° = 0.866 and sin 30° = 0.5, lx = 20 x 0.866 = 17.3 and ly = -20 x 0.5 = -10.0; |17.3| ≤ 200 and |-10.0| ≤ 150, so A hits too.
• Overlap: area A = 400 x 300 = 120,000 px² (rotation preserves area), area B = 100 x 100 = 10,000 px². B is 12x smaller, so B is the selection.

How fast is the hit test as the region count grows?

A plain loop stays well under a millisecond into the thousands of regions: 500 regions with 4 vertices each cost about 20,000 floating-point operations per click — roughly 20 microseconds at 10⁹ operations per second — so no spatial index is needed at click rates.

The arithmetic is short: a box pretest is four comparisons, a four-edge polygon test about 40 operations, so 500 regions cost 500 x 40 = 20,000 operations. Tracking the hit continuously during a drag at 60 updates per second raises that to 1.2 million operations per second, still trivial for the PC. Consider a uniform grid only beyond roughly 10,000 regions, or when single regions carry hundreds of vertices, and let the box pretest do the sorting even then.

Where should the overlay and the hit test run in a vision system?

On the PC side, next to the viewer: a vision controller runs your application, draws the regions and executes this hit test, while the camera delivers images and a smart camera keeps its own on-device region tools for stand-alone inspection.

That is the normal split in vision hardware. Smart cameras run the inspection on the device, with regions configured in their own tools; vision controllers are fanless computers with camera network ports and built-in light-control channels, sized to run your application, its overlay and this hit test in one box. Two current examples:

The MV-VC3301H-128G66 provides 5+ LAN ports, 3-4 USB 3 ports, 4 built-in light channels, 8 GB of memory and an Intel Core i3 processor.

The MV-VC2140-128G40-1T provides 3-4 LAN ports, 2 USB 3 ports, 4 built-in light channels, 8 GB of memory and an Intel N-series processor.

Either drives the cameras, fires the lights and renders the overlay, so the whole coordinate pipeline in this guide runs on the controller.

What mistakes put the selection on the wrong region?

Four causes cover nearly every wrong selection: the pan offset applied without the zoom, one uniform scale used while the control stretches one axis, desktop screen coordinates read instead of the control's client point, and a y-axis that points the wrong way.

• Half the transform. Applying zoom but forgetting the pan, or the reverse, shifts every click by a constant. Log the converted image point on each click; it must land where you clicked at every zoom level.
• Non-uniform stretch. A fit-to-window mode can scale x and y differently. Separate scale_x and scale_y absorb it.
• Display scaling. At 125% or 150% operating-system scaling, the desktop cursor position differs from the control's client point by a factor of 1.25-1.5.

• Y direction. Image row 0 is at the top; some display stacks measure from the bottom. One sign flip in the conversion fixes it, and the click log finds it in seconds.
• Stale geometry. Regions changed programmatically but drawn from an old copy produce selections that look wrong yet are correct for what is on screen. That mismatch is the same family as ROI image source binding errors, where a region belongs to a different image source than the one displayed, and its cousin, cropping an ROI when the corner coordinates are unknown: map every point between frames you fully control.

Once the hit test returns the right id, the rest is presentation: highlight the region, print its parameters, pass the id to your recipe. If you are specifying the machine this overlay will run on, the configurator call-out below starts at the camera step and walks the rest in order.