You have a rectangular mask on the image and a detected edge outside it, and you need the gap between the two. Most vision libraries compare like with like: point to point, line to line, rectangle to rectangle. A rectangle has no single distance to a line unless the two are parallel, so the ready-made distance function refuses the pair. The fix is to define the measurement yourself: fit a line to the detected edge, reduce the mask to fixed reference points, and take the perpendicular distance from each point to that line. This guide gives the definition, the point-to-line formula, a worked example with the arithmetic shown, and the checks that make the number trustworthy on the floor.
Pick the distance definition before you measure
For non-parallel lines the minimum, maximum and midpoint gaps all differ, and they drift apart as the tilt grows. There is no single number until you fix the reference points, so pick the definition that matches the function of the check and write it into the test specification:
- Position check. The part must sit at a nominal gap from the mask boundary: use the perpendicular distance from the midpoint of the mask edge that faces the part.
- Clearance limit. The part must never cross the mask boundary: use the minimum distance from the mask corners to the fitted line.
- Zone report. The image is split into left, top and right regions and each must report a gap: use one reference point per zone, the midpoint of that zone's inner edge, and return three numbers per cycle.
For corner references, the two-line corner construction in corner position measurement with two edge lines gives a repeatable vertex; for gaps between two points, measuring crack endpoint distance is the point-to-point variant. Everything below uses the midpoint definition.
What you need
Five things, all in pixels except the last:
- The mask as coordinates: the four corners of the rectangle, for example (x1, y1) and (x2, y2).
- Edge points along the detected edge at subpixel precision, from calipers or scan lines placed across the expected edge.
- A least-squares line fit through those points.
- The pixel scale in mm/px, from a calibration target or computed from the optics (Step 4).
- Light that repeats every cycle: a constant-current or strobed light controller holds the gray level the threshold lands on, and that gray level fixes the edge position. If brightness wanders, the measured edge wanders with it; closed-loop lighting control for repeatable vision images shows how to pin it down.
Step 1: fit a line to the detected edge
Place 20 to 50 calipers perpendicular to the expected edge direction; each returns one subpixel (x, y) point. Reject outliers before fitting: drop any point more than about 2 px from the median of its neighbors, because one smeared caliper tilts the whole line. Fit y = mx + b by least squares, then write the line as a·x + b·y + c = 0 with a = m, b = −1 and c equal to the fitted intercept. A clean, well-lit edge fits with residuals under 0.3 px; residuals above that point to lighting or search-direction trouble, not to the math.
When the feature is a cluttered contour rather than a straight edge, a line can be the wrong model; Hough line detection versus contour statistics helps you decide which to fit.
Step 2: reduce the mask to reference points
A rectangle is four corners, so the reduction is exact and costs one line of code. For the midpoint of a mask edge, average that edge's two corners: a left edge with corners (x1, y1) and (x1, y2) has its reference at (x1, (y1 + y2) / 2). For the left-top-right split, compute one midpoint per zone and report one distance per zone. Clearance checks skip the averaging and use the four corners as they are. The definition, not the code, is what needs care: once fixed, it stays fixed across parts, operators and software versions.
Step 3: compute the perpendicular distance
Take the reference point (x0, y0) and the fitted line a·x + b·y + c = 0:
The numerator is the vertical gap rescaled by the line's slope; the denominator turns it into the true shortest length from the point to the line. Keep the sign of a·x0 + b·y0 + c if you also need to know which side of the line the point is on.
Worked example. The fit returns m = 0.05 and intercept 350, so the line is 0.05x − y + 350 = 0, and the mask-edge midpoint is (800, 600).
d = |0.05 × 800 − 600 + 350| / √(0.05² + 1²) = |−210| / √1.0025 = 210 / 1.00125 = 209.7 px
The vertical gap is 210 px, but the edge is tilted by θ = arctan(0.05) = 2.86°, so the perpendicular distance is shorter by cos θ: 210 × 0.99875 = 209.7 px. The tilt costs 0.12 % at this angle. A vertical reading on a 5° tilted edge overstates the gap by 0.4 %; at 10°, by 1.5 %.
Step 4: convert pixels to millimeters
Pixels are done; millimeters need the scale s. Calibrate it from a glass target or a gauge block, or compute it from the optics:
The computed scale assumes the pinhole model and inherits the working distance directly: a 1 % working-distance error moves the scale by about 1 %. When the tolerance is tighter than 1 %, calibrate from a target; calibration target field of view and coverage margins sizes the target, object-space pixel size from magnification gives the magnification route, and the working distance and field of view formula walks the same geometry from the imaging side.
Worked example. Take a typical 6 MP 2/3" sensor: 3072 × 2048 pixels at a 2.86 µm pitch, so the sensor width is 3072 × 2.86 µm ≈ 8.8 mm. Lens: 35 mm focal length. Working distance: 350 mm.
m = 35 / (350 − 35) = 35 / 315 = 0.1111 field of view = 8.8 / 0.1111 = 79.2 mm s = 79.2 mm / 3072 px = 0.0258 mm/px
The Step 3 gap becomes 209.7 px × 0.0258 mm/px = 5.4 mm.
The LS-FFL-0679 is a 35 mm C-mount lens built for 2/3" 6 MP cameras.
- Type
- Machine vision fixed
- Sensor format
- 2/3"
- Mount
- C-mount
- Resolution
- 6-10 MP
Subpixel edge finders repeat to 1/10-1/5 px with clean contrast. At 0.05 mm/px that is 0.005-0.01 mm of edge accuracy; the 0.0258 mm/px of this example puts the same behavior near 3-5 µm. Plan on ±0.01 mm for a production rig once threshold drift, vibration and distortion have had their say; subpixel edge measurement to micrometer precision shows what a disciplined set-up achieves.
Step 5: verify the measurement
Five checks separate a number you can ship from one that drifts next week:
- Repeatability. Acquire 20 images of a stationary part. The spread (max − min) of the distance should stay under 0.1 px; more means threshold or trigger jitter.
- Scale. Insert a 1.00 mm gauge block under the lens and re-measure: the distance must change by 1.00 mm within tolerance. This catches working-distance and calibration errors at once.
- Tilt. Rotate the part by ±2° about a known point on the edge. A reference 400 px along the edge from that point should move by 400 × sin(2°) ≈ 14 px; the numbers must move with the geometry, not with the algorithm.
- Fit health. Log the line-fit residual every cycle and alarm above 0.3 px, so a smudged part cannot quietly feed the distance a bad line.
- Light. Log LED drive or strobe width per cycle; brightness drift shows up first as a slow creep in distance. The ambient light rejection specification puts numbers on the acceptance test.
Common mistakes
- Squeezing the edge into a thin rectangle to feed the rectangle-distance function. The result quantizes to that rectangle's width and biases the answer by up to half of it. Fit a line instead; it costs one step and removes the bias.
- Counting pixels in a band instead of locating the boundary. Blob area by pixel counting answers how many pixels are on, not where the edge sits; a threshold that drifts one gray level moves the counted boundary with it.
- Vertical calipers on a tilted edge. The vertical reading overstates the gap by 1/cos θ: 0.4 % at 5°, 1.5 % at 10°. Use the perpendicular formula or aim the calipers perpendicular to the fitted line.
- Reusing a scale after moving the camera. The scale tracks the working distance: a 10 mm shift at a 350 mm working distance changes this example's scale by about 3 %, which turns the 5.4 mm gap into a 0.17 mm error. Recalibrate after any mechanical change.
- Treating min, max and midpoint as interchangeable. They differ whenever the lines are not parallel; the specification names one and the code computes that one.
Optics and lighting that hold the number
Three hardware choices decide whether Steps 1-4 stay stable in production.
Distortion before megapixels. Lens distortion bends the geometry and no line fit repairs it. The LS-FFL-0567 holds distortion below 0.3 % over its 1.1" field at F2.8-22. An uncorrected 0.3 % bend across this example's 79.2 mm field is about 0.24 mm at the frame edge, 24 times a ±0.01 mm budget, so keep the measured edge inside the calibrated center or calibrate the distortion map; geometric versus TV distortion explains which spec to read.
Vibration. A bench that shakes changes focus, and changed focus changes the scale. For lines near presses or conveyors, the LS-FFL-0475 is a 36.61 mm anti-vibration lens for 2/3" sensors in C-mount.
- Type
- Machine vision fixed
- Sensor format
- 1.1"
- Type
- Anti-vibration
- Sensor format
- 2/3"
Light. The edge lives on a gray-level threshold, so brightness must repeat cycle after cycle, and strobing freezes motion blur on moving parts. The LS-CTL-0070 is a 4-channel, 200 W digital control box with RS-232 and strobe output.
- Output
- Constant voltage 24 V
- Channels
- 4 channels
- Control
- Digital
- Feature
- Strobe
Browse machine vision lenses by focal length, sensor format and distortion with the lens finder, match the light to its driver in the light controller finder, and let the configurator walk camera, lens and light in one pass. The systems topic page collects the sibling measurement guides, and the contact page reaches the applications team when a number refuses to settle.
Questions
Why does the distance tool reject a rectangle and a line?
Shape-pair distance functions compare like with like, and a rectangle has no single distance to a line unless the two are parallel: minimum, maximum and midpoint gaps all differ and drift apart as tilt grows. Reduce the rectangle to reference points instead - the midpoint of the edge that faces the part for a gap check, the corners for a clearance limit - and compute point to line.
How accurate is the mask-to-edge distance in millimeters?
Budget in pixels first: a subpixel edge fit repeats to 1/10-1/5 px with clean contrast. At 0.05 mm/px that is 0.005-0.01 mm; at the worked example's 0.0258 mm/px it is 3-5 µm. Then add scale error from working distance, threshold drift from lighting and distortion toward the frame edges - a controlled production rig lands near ±0.01 mm.
How do I get the mm/px scale without a calibration target?
Compute it from the optics: field of view = sensor width × (WD - f) / f, then scale = field of view / horizontal pixel count. Example: 8.8 mm × (350 - 35) / 35 = 79.2 mm, and 79.2 / 3072 = 0.0258 mm/px. The computed scale inherits working-distance error directly, so check it once with a 1.00 mm gauge block before trusting it.
Should I report the minimum, maximum or midpoint distance?
Let the function of the check decide. Midpoint perpendicular distance averages the whole fit and is the stable number for trend charts and SPC. Minimum distance is correct only when the limit is physical clearance, and it is noisier because it rides on the single worst corner or edge point. Never average the two - they measure different geometry.
How many edge points does the line fit need?
Twenty to fifty caliper points is the working range. Fit noise falls with the square root of the count, so doubling from 20 to 40 points cuts fit noise by roughly 30 percent, while 200 points buy little and slow the cycle. Spend the budget instead on even spacing along the edge and on outlier rejection before the fit.



