Segment a package whose tape matches the conveyor color

A carton closed with tape crosses a dark belt and your detector returns two packages instead of one: the threshold sees belt, tape, belt, because the tape and the belt share one gray level. Boxes of 300 x 200 mm and 200 x 200 mm on a near-black belt are the classic case. The fix has a strict order: bright, diffused light and a short strobed exposure first, an Otsu threshold second, a morphological merge third. Below are the gray-level targets, the exposure arithmetic, the OpenCV calls, the kernel-sizing rule and the acceptance test — enough to take a split detection to one measurable box in an afternoon.

Why the box splits in two

Underexposure breaks these systems before the tape does. In a dark frame the cardboard sits near gray 60 and the belt near 35; sensor noise spans both levels, so the Otsu threshold lands inside the cardboard texture and the box shreds into fragments no kernel can repair. The tape is a different failure: it is dark for chemical reasons, and no exposure setting makes it bright. Nothing needs it bright — you bridge it after the threshold, not during it.

That order is the whole method: light, exposure, threshold, merge. Tuning kernel sizes on an underexposed frame tunes them to noise; the same settings on a properly lit frame hold for weeks. If you have worked through dark contaminant detection on dark surfaces, you already know the rule it teaches: earn the contrast in the lighting before you touch a threshold.

What you need

A global-shutter camera, a 16 mm-class lens, two diffused bar lights and a strobe-capable controller. Rolling shutter rules itself out on a moving belt: at 0.5 m/s, a 20 ms frame readout displaces the last row about 10 mm against the first — 3 % of a 300 mm package, enough to bend every edge. Among area scan cameras:

The MV-CU020-90GC is a color unit; convert each frame to grayscale. Its Bayer filter typically costs about half the sensitivity, and the strobe pays that back. At 2.5 MP across a 400 mm field of view you get 0.21 mm per pixel, the right class for parcel measurement — the worked example shows why more would hurt.

Match the lens to the field of view at the working distance you can actually mount. The example below lands on a 16 mm focal length:

Two bar lights face each other across the belt at 35-45° from vertical, aimed at the center of the field. Every extra 100 mm of belt width wants about 100 mm more lit length; the light finder filters by size. All three of these run on 24 V:

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

The LS-BAR-1276, a 372 x 28 mm diffused white bar, is the first choice: diffusion stops tape glare punching white holes through the dark side of the histogram. The LS-BAR-0857, a 540 x 12 mm bar, carries the lit length to cover a wide belt from one side. The LS-BAR-0400, a 312 x 30 mm high-luminance bar, buys pulse energy in a shorter housing.

Drive the bars from a controller with a trigger input and overdrive, wired to a photoelectric sensor upstream. Open the configurator at the controller step to match driver channels to the bars.

Step 1: Fix the light and the exposure first

1. Turn auto exposure and auto gain off. Automatics drift with every package, and the threshold pays for the drift.
2. Power the bars until the belt reads at or below gray 60 and the cardboard at or above 160. That 100-level gap is the budget your noise, your dirt and your threshold all spend from.
3. Cap exposure where motion blur stays under one pixel. Blur is belt speed times exposure: 500 mm/s times 1 ms is 0.5 mm, already 2.4 px at 0.21 mm per pixel. At 0.5 m/s the cap is 400 µs — or strobe at 100 µs and stop worrying about it.

Step 2: Threshold with Otsu and read the histogram

Otsu picks the threshold that minimizes within-class variance; with two clean modes it lands in the valley with nothing to tune. One median pass first removes dust and flecks:

gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
gray = cv2.medianBlur(gray, 3)
t, binary = cv2.threshold(gray, 0, 255,
                          cv2.THRESH_BINARY + cv2.THRESH_OTSU)

Read two numbers before trusting the binary frame. The threshold t should land between 90 and 130 with the Step-1 targets, and the valley between the two histogram peaks should reach at least 30 gray levels. A t above 140 means the belt is reflecting light back; a shallow valley means the scene is still too dark. How a binary edge behaves at a threshold is covered in threshold basics without a vision library.

Step 3: Merge the halves: dilate k, erode 2k, dilate k

You now have two bright cardboard halves with a dark, tape-shaped gap between them. Bridge it with three passes of a 5 x 5 elliptical kernel:

k = 50
el = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5, 5))
m = cv2.dilate(binary, el, iterations=k)
m = cv2.erode(m, el, iterations=2 * k)
m = cv2.dilate(m, el, iterations=k)
cnts, _ = cv2.findContours(m, cv2.RETR_EXTERNAL,
                           cv2.CHAIN_APPROX_SIMPLE)
box = cv2.minAreaRect(max(cnts, key=cv2.contourArea))

Size k from the tape. One pass of the 5 x 5 ellipse grows a region 2 px per side, so k iterations bridge a gap of 4k px: k = 50 bridges 200 px.

Each stage earns its place. The first dilation swells both halves until they join across the tape. The double erosion cuts the swollen outline back past the true edge — and erases every debris speck the dilation absorbed, because anything under 4k px across cannot survive it. The final dilation restores the boundary: on a straight edge the net shift is 2k - 4k + 2k = 0 px, so the merge leaves your 300 mm measurement intact.

If the halves still refuse — tape wider than your k, or dark print dragging half the box under the threshold — merge by geometry: take the two largest contours, stack their points, and fit one minAreaRect to the union. Gate the merge so it can never join two real packages: only pair blobs that each cover at least 15 % of the expected footprint (9,000 mm² of a 60,000 mm² box) and sit under 60 mm apart. Induct spacing on parcel lines runs 100 mm and up, so the gate holds. The smaller-defect side of these same operators is walked through in dark speck detection with thresholding and morphology.

Worked example: 300 x 200 mm carton on a 0.5 m/s belt

Inputs, each with its source:

• Package 300 x 200 mm; tape strip 40 mm wide — a standard closing tape, so measure yours and substitute.
• Belt speed 0.5 m/s. Every number below scales linearly with it.
• 1920 px across the frame, the width of the image you already capture.
• Field of view 400 mm: the 300 mm package plus 50 mm of placement margin per side.

Lighting: two diffused bars put the belt at gray 50 and the cardboard at gray 180 — a 130-level gap against the 100-level minimum, and the 100 µs strobe pulse freezes the package to a quarter of a pixel.

Lens: a sensor in the 1/1.7-inch class has an active width near 7.6 mm. Magnification m = 7.6 / 400 = 0.019, and f = WD x m / (1 + m) = 800 x 0.019 / 1.019 = 14.9 mm, so the 16 mm catalog step fits, which puts the working distance at 16 x 1.019 / 0.019 = 858 mm — mount at 860 mm. Sensor widths in this class run 7.2-7.6 mm, worth about ±1 mm of focal length, so recompute with your own numbers:

Depth of field at f/8 with a 2-pixel circle of confusion (0.42 mm on the belt): 2 x 8 x 0.42 x 1.019 / 0.019 = 360 mm, an estimate from the lens relation. A ±50 mm box-height variation sits far inside it.

Timing: the photoelectric sensor mounts 300 mm upstream, so the trigger delay is 300 mm / 500 mm/s = 0.6 s, then the strobe fires for 100 µs inside a 400 µs exposure.

Decision: with the belt at or under gray 60, the cardboard at or above 160, blur at or under a quarter pixel and k = 50, the merge returns exactly one contour, and its minAreaRect reads 300 ± 1 mm by 200 ± 1 mm. Sign-off tolerance: ±2 mm per side across 100 cartons. A rect that runs long means k is too big or focus is soft; a box that splits again means the tape is wider than 42 mm — raise k by 1 for every 4 px you measure.

Verify before you commit

1. Run 100 consecutive cartons. Every single-carton frame must give exactly one contour with sides within ±2 mm of 300 and 200 mm. Log the sides: a mean offset points at focus or k, a widening spread at dirt or drifting strobe current.
2. Track one long edge across a shift. Its position should hold a standard deviation under 1 px (0.21 mm); softer edges point at blur, the failure mode in diagnosing machine vision motion blur.
3. Re-check the histogram valley weekly. Under 30 levels deep, clean the lens or raise strobe energy; the two-frame test in strobe lighting ambient rejection separates daylight drift from LED ageing.

When the usual answer is wrong

• More pixels. A 12 MP sensor takes the pixel size from 0.21 mm to 0.10 mm, doubles the tape gap to 400 px and quadruples the morphology time — for a box you already measure to ±0.5 mm. Size the sensor from the smallest feature you must resolve, not the largest object; sorting conveyor camera selection works that arithmetic end to end.
• Rect or cross kernels, tuned harder. They bridge along their axes only, so a carton rotated 30° keeps a diagonal seam. The ellipse kernel is isotropic; when rect and cross kernels failed you, the frame was underexposed, not the shape wrong.
• Adaptive thresholding. A local-mean pass fixes uneven light, not two materials sharing one gray level. With the tape at 192 px — wider than the usual tile — tiles inside the tape have no bright neighbor to average against; local mean adaptive thresholding covers the cases where it genuinely wins.
• A glossy belt. Bright-field bars mirror off a polished belt into the lens, and the background grows bright patches that break Otsu. Steepen the bar angle to 60-70°, or cross-polarize per cross polarization for surface glare — a crossed pair passes well under a quarter of the light, so budget strobe energy for it.
• A rolling-shutter camera. The 10 mm of shear at 0.5 m/s lands in every dimension you report; no software recovers it at line rate.

Checklist before you order

• Global-shutter camera, 2-3 MP class, C-mount; convert color frames to grayscale.
• 16 mm lens for an 860 mm working distance — or your recomputed focal length from the calculator above.
• Machine vision lenses matched to the sensor format you actually buy.
• Two diffused white bar lights, mounted facing each other at 35-45°, each long enough to cover its half of the field.
• Strobe controller with trigger input and overdrive; photoelectric sensor 300 mm upstream, delay 0.6 s at 0.5 m/s.
• Gray targets posted at the station: belt at or below 60, cardboard at or above 160, valley 30 deep.

When the parts are on the bench, configure the full station or send one captured frame through the contact page and we will read the histogram with you.