Find rectangle edges and corners in an image

A rectangular part has to report four edges, four corners and a center point to the robot or the reject logic, and the classic failure is one edge — usually the bottom one — that never comes out clean. That failure is almost always optical, not algorithmic. This procedure takes you from raw image to four corner coordinates and a centroid, with the numbers to hold against your tolerance: how many pixels the part needs, how much contrast the light must deliver, and how to prove repeatability before the first production cycle. It is written for line engineers and integrators setting up pick-and-place, presence and orientation checks.

What you need

The fit itself is arithmetic on a binary image, so the camera only has to deliver the pixel scale from the next section. Where the check must run without a PC, smart cameras execute the whole chain — threshold, region selection, rectangle fit — on the device. The LS-SMC-0063 is a 1.6 MP smart camera with a 16 mm lens, white light and 60 fps on GigE. The MV-SC5060XM-08M-WBN is a 6 MP mono smart camera with an 8 mm M12-mount lens on GigE; its rolling shutter suits static stations, since a moving part skews under rolling readout.

On a PC-based line, an ordinary area scan camera does the same job. The MV-CU020-90GC is a 2.5 MP color camera with a 1/1.7" global-shutter sensor on C-mount, which also suits fast conveyors.

On the software side you need four primitives every vision library carries: a threshold, a morphological close, connected-component labeling, and a minimum-area rotated-rectangle fit.

Size the sensor to your tolerance

Two rules fix the pixels before anything else. Keep the part between a third and a half of the field-of-view width: enough margin for position wander without wasting pixels on background. Then size the pixel scale so a 0.1-pixel edge fit stays under a third of your tolerance.

Worked example — an 80 × 60 mm bracket whose center must repeat to ±0.10 mm:

• Field-of-view width: 3 × 80 mm = 240 mm.
• Sensor: 2448 × 2048 pixels at 3.45 µm pitch, so 2448 × 3.45 µm = 8.45 mm wide.
• Magnification: m = 8.45 / 240 = 0.0352.
• Working distance 300 mm: f = 300 mm × 0.0352 / 1.0352 = 10.2 mm, so the nearest standard lens is 12 mm.
• With the 12 mm lens: field of view ≈ 8.45 mm × 300 / 12 = 211 mm; pixel scale 211 mm / 2448 px = 0.086 mm/px.
• Part coverage: 80 mm / 0.086 mm/px = 927 pixels, a 38% fill — above the one-third floor.
• Edge repeatability at 0.1 px: 0.086 × 0.1 = 0.0086 mm ≈ ±0.01 mm.

Motion sets the exposure. At 50 mm/s conveyor speed, a 0.5 ms exposure smears the edge by 50 mm/s × 0.0005 s = 0.025 mm, or 0.29 pixels at this scale — inside a 0.5-pixel budget. Past half a pixel the gray ramp widens and the sub-pixel fit degrades.

Shortlist sensors with enough pixels in the camera finder.

Light for a two-valued image

Edges come from contrast, and the most reliable contrast is a silhouette. Backlight an opaque part and it goes dark against a bright panel: silhouette near gray 40, background near gray 220, a step of 180 levels. A fixed threshold at 125 sits far from both populations and holds all shift. For thin or translucent material, front-light bright field or dark field instead — the lights finder groups the shapes.

Color helps when the part has it: threshold the single channel with the largest step at the edge, red for a red part on gray. A straight grayscale mix can halve that gap, which matters when the budget is 50 levels.

Strobe through one of the light controllers when ambient light wanders or the part moves: the flash freezes motion and darkens room light within the same exposure. The LS-CTL-0070 is a 4-channel, 200 W digital control box with strobe output and RS-232 on 24 V constant-voltage channels. Where ambient shifts dominate a cell, the ambient light article orders the countermeasures.

The seven-step workflow

1. Pick the channel. Convert to gray or take one channel — whichever shows the largest step across the rectangle's boundary.
2. Threshold once, fixed. Mid-gap between part and background: 125 against the 40/220 backlight example. If shading varies across the part itself, adaptive thresholding handles that; the dark-region extraction article walks through it.
3. Close and fill. A 3-5 pixel morphological close bridges speckle along the edge; fill holes so engraved text or punched holes inside the part cannot fragment the region.
4. Label and select. Run connected-component labeling and keep the largest region — the same region math as measuring blob area by pixel counting. Gate on area: reject the frame when the region leaves ±10% of nominal, because a clipped part or an overlapping neighbor shows up here first.
5. Fit the minimum-area rotated rectangle. One call returns center, angle, length and width. Multiply pixel results by the 0.086 mm/px scale to reach millimeters.
6. Take the corners. Read the rectangle's four corners directly, or — for rounded or chamfered corners — fit lines to the straight sections and intersect adjacent pairs, as the next section shows.
7. Reduce to centroid and angle. The center is the mean of the four corner coordinates; the angle is the arctangent of a long edge's direction. Mind the half-pixel coordinate origin before converting to robot coordinates (why blob centroids carry a half-pixel offset); the same four-point reduction extends into a full ground-plane pose from four known points.

Corners and centroid to sub-pixel accuracy

Binary edges sit on the pixel grid, so their reported position wanders by tenths of a pixel between frames. A 1-D sub-pixel fit tightens that to about 0.1 pixel: walk a short profile perpendicular into each edge, mark the 50% crossing of the gray step, and gather 200 or more points per edge. Discard outliers more than 2 pixels from a first-pass line, then least-squares fit each edge as a1·x + b1·y = c1. Adjacent edges intersect where the determinant is nonzero:

At the example's 0.086 mm/px, 0.1 pixel per edge is ±0.01 mm; a corner, where two edges meet, repeats near ±0.015 mm; the center averages four corners down to about ±0.01 mm. The same 50%-crossing reduction on a filled circle is worked through in the grayscale centroid article. If the corners feed a controller that also needs the rectangle's rotation, the formulas for computing rectangle rotation from corner coordinates take this section's output directly.

Prove repeatability before production

Park the part untouched, run 30 images through the full pipeline, and compute the 3σ spread of the center and each corner. The decision rule: a measurement serves a tolerance when its 3σ stays under a third of that tolerance. Against ±0.10 mm you need 3σ ≤ 0.033 mm, and the example's ±0.01 mm center clears it threefold. Repeat after an hour of warm operation: a creeping mean usually means auto-exposure or auto-gain is still on, which the brightness-drift article dissects — lock every automatic mode off and re-test.

When the usual answer is wrong

• Rounded or chamfered corners. A minimum-area rectangle cuts inside a corner radius — 2 mm is 23 pixels at the example's scale, and the reported corner sits roughly that far inside the true one. Fit lines to the straight midsections and intersect them; the line fit never sees the radius.
• One edge refuses to appear. Hunt the optics before the code: a shadow band from the light, the light's own reflection, the edge hugging the image border, or fixture teeth over the silhouette. Move the light or widen the field first.
• Edges near the image border. Lens distortion bends them: at 1% distortion a point 1,224 pixels from center pulls about 12 pixels, about 1 mm at the example's scale. Keep the rectangle inside the central 60% of the frame or calibrate the distortion out.
• Parts touching or cluttered. Erode 1-2 pixels to separate regions before labeling, or confine the pipeline to a per-nest ROI.
• No backlight possible. Under a directional front light, expect ±0.5-1 pixel edge repeatability rather than 0.1 px; re-check against the one-third rule and shrink the field of view if it fails.

Checks before you order

• Part fills 33-50% of the field-of-view width — 38% in the example.
• Pixel scale puts 0.1 pixel of edge fit under a third of your tolerance.
• Global shutter whenever the part moves under exposure; exposure short enough that blur stays under 0.5 pixels.
• Light step of at least 50 gray levels at the edge, threshold fixed mid-gap.
• Strobe controller channels matched to the lights in the cell.
• Rectangle inside the central 60% of the frame, or distortion calibration in hand.

Size the whole chain from your own part in the camera configurator, or send drawing and tolerance through the contact page and we will return the field of view, lens and lighting that meet the numbers.