Crop an image ROI when the corner coordinates are unknown

An inspection needs a region of interest, but the part's corner coordinates are not yet known. Start by deciding whether an operator should select a fixed window or whether software must locate the feature in each frame. Both approaches can produce the same crop rectangle. This guide uses OpenCV and NumPy, keeps one coordinate convention throughout, and includes a mask-based crop function with explicit handling for empty detections and image boundaries.

Choose a fixed window or a moving detection

Use a fixed rectangle when the fixture holds the feature in a repeatable place. Establish it during setup and revalidate it after optical, mechanical or image-format changes. A rectangle is not automatically valid for the life of a station.

Use a detected box when the feature moves. First isolate the intended feature with segmentation, an instance mask or a suitable component-selection rule. A raw Canny image can contain background texture, disconnected edges and several parts; taking its global minimum and maximum coordinates does not establish which feature you found.

A useful combination is a generous fixed search window followed by detection inside it. When detection uses that subimage, add the search window's origin to the detected coordinates before cropping the full frame. The ROI image-source binding guide covers another common failure: applying a valid rectangle to the wrong image buffer.

Draw a rectangle with OpenCV

For a desktop setup tool with a GUI-enabled OpenCV build, cv2.selectROI returns four values: x, y, width and height. The HighGUI API reference describes the selection controls and the empty rectangle returned on cancellation. This example expects an existing NumPy image named frame:

import cv2

x, y, w, h = map(int, cv2.selectROI(
    "Select ROI", frame, showCrosshair=True, fromCenter=False
))
cv2.destroyWindow("Select ROI")
if w <= 0 or h <= 0:
    raise ValueError("ROI selection cancelled or empty")

crop = frame[y:y + h, x:x + w].copy()
assert crop.shape[:2] == (h, w)

1. Display a representative frame and draw around the feature with the required surrounding context.
2. Confirm the rectangle with Enter or Space; use c to cancel.
3. Inspect the returned crop and its dimensions before saving the rectangle.
4. Record the image geometry along with x, y, width and height.

Do not assume every GUI tool uses these conventions. Some tools report spatial axes coordinates, one-based pixel locations or floating-point bounds. Verify a known landmark. A resized display is acceptable when its ROI tool maps pointer positions back to the original image; a manually resized input array needs the coordinate conversion described below.

Keep inclusive pixels and exclusive slice bounds separate

In this guide, (x0, y0) is included and (x1, y1) is excluded. NumPy slices use rows before columns, so the crop is frame[y0:y1, x0:x1]. This follows NumPy's indexing rules.

The maximum occupied pixel is different from the exclusive boundary. Add one when converting a mask's maximum row or column to a slice stop. A single selected pixel at column 9, row 7 must produce frame[7:8, 9:10], with shape (1, 1), rather than an empty array.

Compute a crop from an isolated mask

Supply a two-dimensional mask with nonzero pixels only for the intended feature. It must have the same height and width as the source image. The image can be grayscale or carry colour channels. The function returns exclusive bounds and an independent copy of the crop, or None when nothing was detected:

import numpy as np

def crop_from_mask(image, mask, margin=0):
    if image.ndim < 2 or mask.ndim != 2:
        raise ValueError("Expected an image and a two-dimensional mask")
    if mask.shape != image.shape[:2]:
        raise ValueError("Mask and image dimensions differ")
    if not isinstance(margin, (int, np.integer)) or margin < 0:
        raise ValueError("Margin must be a nonnegative integer")

    rows, cols = np.nonzero(mask)
    if cols.size == 0:
        return None

    height, width = image.shape[:2]
    x0 = max(int(cols.min()) - int(margin), 0)
    y0 = max(int(rows.min()) - int(margin), 0)
    x1 = min(int(cols.max()) + 1 + int(margin), width)
    y1 = min(int(rows.max()) + 1 + int(margin), height)
    return (x0, y0, x1, y1), image[y0:y1, x0:x1].copy()

Handle None as an explicit inspection outcome. Do not silently reuse the previous frame's rectangle after a failed detection. The copy also prevents later changes to the source array from changing the saved crop; for high-throughput acquisition, include that memory copy in the processing budget.

If several candidates remain, select the intended one before calling the function. Connected components describe connectivity in a mask, not physical object identity. One object can produce disconnected edge fragments, and touching objects can form one component. Use the application's position, area and shape constraints, and retain a failure state for ambiguity.

Verify the worked bounds and choose the margin

Take a synthetic 4000 by 3000 image and a mask whose occupied columns run from 812 through 1390 and rows from 590 through 1040. These are example inputs, not a camera performance claim. With zero margin the result is 579 by 451 pixels. With a 20-pixel margin the exclusive bounds become (792, 570, 1411, 1061) and the crop is 619 by 491 pixels.

That crop contains 303,929 pixels. A fixed 800 by 600 window contains 480,000, so the smaller box reduces the pixel count by about 36.7%. Processing time need not fall by the same percentage: detection, copying and fixed overhead still cost time.

Choose padding from the feature context and measured uncertainty. For example, if an established motion allowance is 8 pixels and an established detection-error allowance is 2 pixels, their sum is 10 pixels; choosing 20 adds a further 10-pixel reserve. Those allowances must come from the application. A short trial is not proof of the maximum possible drift.

Clamping keeps array indices valid but cannot recover content outside the captured image. If a required margin is clipped at the frame edge, flag that condition and decide whether the inspection can proceed. For barcode work, preserve the symbol's required surrounding quiet zone; see cropping barcode images safely.

Map preview coordinates back to the source

For an image resized without padding, use separate horizontal and vertical scale factors. A 4000 by 3000 frame resized to 1000 by 750 has a factor of four in both directions. Preview position (160, 128) therefore maps to source position (640, 512).

If the preview is letterboxed, subtract the displayed image's offset before undoing the scale. Clamp selections to the displayed image, excluding the padding. Round lower bounds down and upper bounds up when a selection crosses fractional source pixels. If the preview is rotated or undistorted, use the corresponding geometric transform; multiplying by a width ratio alone is insufficient.

Store whether the rectangle is in preview, full-frame or cropped-sensor coordinates. When a camera changes, verify field of view, pose, sensor crop, binning and distortion before reusing it. To convert positions into part dimensions, use a calibrated mapping such as the one discussed in locating image defects with pixel coordinates and calibration.

Test before using the crop in an inspection

Test an empty mask, one pixel, an interior rectangle and a feature touching every image boundary. Check grayscale and colour images, a mismatched mask shape, and invalid margins. Confirm the crop contains the original intensity or colour values; use the binary mask only to locate the bounds unless the next algorithm explicitly needs a mask crop.

Save one annotated full frame together with the crop, bounds, frame identifier and geometry settings. Inspect known landmarks and expected dimensions. Then exercise part movement and missing or ambiguous detections with representative images. The numerical examples establish indexing behavior, not detection accuracy on your production parts.

For a fixed fixture, start with a validated interactive rectangle. For moving parts, use a selected feature mask and explicit empty-detection handling. The guides hub collects related setup and calibration workflows; use the camera configurator when the required field of view or sampling calls for a hardware change.