Extract a region of interest from an image after detection

The detection is done: the model has drawn its box around the part, and on screen that box is four numbers. What you need next is the region itself — the pixels inside the rectangle — so you can run blob analysis on it, archive it, or hand its position to the line controller.

No second pass and no extra hardware are involved: the box is x, y, width and height in pixels, and every vision library turns those four integers into a crop in one line. This how-to walks the extraction end to end: clamping the rectangle to the frame, cropping or reducing the image domain, a worked example with the arithmetic shown, the blob pass on the crop, and the mistakes that cost an afternoon. It is written for whoever writes inspection code on smart cameras and vision controllers.

What the detector already gives you

Every common detector returns its result as a rectangle in frame pixels. The usual format is four integers: x and y for the top-left corner, width and height for the size. Some libraries return the opposite corners — x_min, y_min, x_max, y_max — instead. Either way, the four numbers are the region of interest, and they come free with the detection.

The one convention to check is the far corner. Array libraries end a slice exclusively, so a slice from 1240 to 1800 spans columns 1240 through 1799, which is 560 of them. Formats that treat x_max as inclusive need width = x_max − x_min + 1. When nobody hands you a rectangle at all, the crop an image ROI when the corner coordinates are unknown guide finds the corners first. The same four numbers feed code downstream, too: passing bounding box coordinates to a depth function shows the identical hand-off into 3D height data.

What you need

• The frame in memory as an array whose first index is the row (y) and whose second is the column (x)
• The detector's box: four integers in frame pixels
• A threshold gray level for the blob pass, chosen on a known-good part
• A lossless format such as PNG, if you archive the crop
• A calibrated millimetre-per-pixel scale, if any result leaves the vision system in millimetres

Extract the region in six steps

1. Read the box as integers. The example values used throughout this guide are x = 1240, y = 830, width = 560, height = 420 pixels.
2. Clamp the rectangle to the frame so an edge-touching detection cannot overrun the array:

3. Crop rows first, then columns: crop = frame[y0:y1, x0:x1]. Writing frame[x0:x1, y0:y1] is the classic silent error on a non-square frame.
4. Check the crop before using it: its shape must read (height, width), and an 8-bit crop must hold exactly height × width bytes.
5. Run the blob pass on the crop, then add the crop origin back to every centroid: frame_x = crop_x + x0, frame_y = crop_y + y0.
6. Save the crop losslessly and log x, y, width and height beside it, so the region can be audited months later.

Worked example: a 560 × 420 crop out of a 6 MP frame

Take a mono 8-bit frame of 3072 × 2048 pixels, a 6-megapixel class size used here as the example. The full frame holds 3072 × 2048 = 6,291,456 bytes, which is 6.0 MB. The box sits fully inside, so clamping changes nothing: x1 = min(3072, 1240 + 560) = 1800 and y1 = min(2048, 830 + 420) = 1250.

The crop holds 560 × 420 = 235,200 bytes, about 230 kB. That is 235,200 / 6,291,456 = 3.7% of the frame, and because thresholding and labelling cost time roughly in proportion to pixels, running them on the crop cuts that stage's work about 27-fold. A detection filling a quarter of the frame instead gives a 4-fold cut.

import numpy as np

# Illustrative frame and box values, not any camera's real output
frame = np.zeros((2048, 3072), dtype=np.uint8)   # frame[rows, columns]
x, y, w, h = 1240, 830, 560, 420                  # detector box, px

x0, y0 = max(0, x), max(0, y)
x1 = min(frame.shape[1], x + w)                   # 1800
y1 = min(frame.shape[0], y + h)                   # 1250
crop = frame[y0:y1, x0:x1]                        # rows first, columns second

crop[99:220, 279:400] = 200                       # example part, 121 x 121 px

mask = crop >= 128                                # threshold for the blob pass
rows, cols = np.nonzero(mask)
print(crop.shape)                                 # (420, 560)
print(crop.nbytes)                                # 235200
print(int(mask.sum()))                            # 14641
print(cols.mean() + x0, rows.mean() + y0)         # 1579.0 989.0

The example paints a part into the crop first: a filled 121 × 121 pixel square at gray level 200 on a black background. The threshold at 128 with an 8-connected count returns exactly 121 × 121 = 14,641 blob pixels. The centroid sits at column 339, row 159 inside the crop, and adding the origin gives 1240 + 339 = 1579 and 830 + 159 = 989 in frame pixels. The four printed values are the test that the crop is right: shape (420, 560), 235,200 bytes, 14,641 blob pixels, position 1579.0 and 989.0.

Crop the pixels or reduce the domain

Two standard operations restrict processing to a region. Cropping copies the pixels out into a new, small image: light to save, trivial to hand to another tool, but its coordinates restart at zero. Reducing the domain keeps the full frame in memory and marks the region every later operator may touch: blob centroids come out in frame coordinates with no offset, and the only new thing to store is the region description.

Choose the crop when the region leaves the pipeline: archived, rendered, stored in a database. Choose the domain reduction when the whole inspection runs in one library and you need features, not pictures.

Run blob analysis on the crop

Three short operations. Threshold the gray levels; in the example, mask = crop >= 128. Label the connected pixels — 8-connectivity is the usual choice for solid markers on a clean background, and the connectivity 4 or 8 for circular markers guide shows when 4 wins. Filter by area and measure; the blob area by pixel counting guide walks the counting itself.

Set the minimum-area filter from the part. Measure the smallest true feature on a known-good crop, say a 15 × 15 pixel mark at 225 pixels, and reject blobs below about half of that. Everything smaller is noise or a clipped fragment.

Save the crop and its coordinates

Archive analysis crops losslessly, as PNG or raw bytes. JPEG compression leaves artifacts around every edge, and at a working threshold those artifacts become phantom blobs — the too many blobs from JPEG artifacts and thresholding guide shows that failure. Put the origin in the file name, for example region_x1240_y830_560x420.png, and log the same four numbers with the result.

For traceability, store the crop as a binary object in a database next to its coordinates and pass/fail result. The guide to saving and retrieving inspection images from a SQL database walks that pattern end to end.

Where the loop runs: on the camera or on a controller

On a smart camera, detection, crop and blob analysis all run on the device, so the only traffic leaving it is the result, plus the small crop if you archive one.

Both of these GigE models run the inspection on the device. The LS-SMC-0063 pairs a 16 mm lens with an integrated white light and captures at 60 fps; the MV-SC5060XM-08M-WBN pairs a 6 MP rolling-shutter sensor with an 8 mm lens.

When several cameras feed one decision, or the boxes must reach the line controller, move the loop to a vision controller. The MV-VC3301H-128G66 has five or more LAN ports, four built-in light channels, an Intel Core i3 processor and 8 GB of memory, and it includes the VM6200Pro license.

Common mistakes

1. Off-by-one at the far edge. Most array libraries end a slice exclusively: frame[830:1250, 1240:1800] spans exactly 420 × 560 pixels. Inclusive formats need width = x_max − x_min + 1. One painted square settles the convention in a single run.
2. Rows and columns swapped. The first index is the row, the second the column. On this frame, frame[1240:1800, 830:1250] runs without an error and returns the wrong rectangle.
3. A box that overruns the frame. An overlong slice truncates silently in some libraries and raises in others. Clamp x + w to the frame width and y + h to the height before slicing.
4. A rotated box. The detector returns an upright rectangle around a tilted part. Handle it after the crop: rotate the cropped pixels back by the box angle, or mask with the four corner points, so the blob pass sees the part axis-aligned.
5. Coordinates from a resized input. Detectors often run on a downscaled copy of the frame. Scale every coordinate by frame width divided by input width before cropping.
6. Overlays saved into the image. Keep display drawings out of the crop that the analysis sees.

Verify it works

• Assert the crop's shape and byte count at start-up; any other value means the clamp or the slice order is wrong.
• Push 20 known-good and 20 known-bad parts through the full box-to-blob chain; the pass and fail counts must match the bench.
• Watch the borders: if a true blob ever touches the crop edge, widen the box by a 10 to 20 pixel margin so no blob sits on the boundary.

When the usual answer is wrong

If the part always arrives inside a known window, skip the detector: a fixed region with a plain blob pass is faster and has nothing to retrain. Detect-then-crop earns its keep when position or orientation varies — the 2D vs 3D camera ROI: where to set it and which one wins guide compares the ROI strategies side by side.

If blobs under-segment because the region holds too few pixels per feature, the fix sits upstream of the code: the camera resolution calculator sizes the sensor from field of view and smallest feature.

Report positions in millimetres only with a calibration taken on the installed lens, working distance and resolution, because the millimetre-per-pixel scale moves with all three.

To pick the hardware for this loop, start from the smart camera and vision controller categories above, then let the system configurator walk the chain end to end. A specific box-to-blob question is best asked through the contact page, and the systems topic page gathers the neighbouring pipeline techniques.