Segment white objects in OpenCV Python: threshold S and V, not gray

Highlighting white objects is the first script most people write with a webcam and OpenCV in Python, and it is the one that most often returns an empty mask or a frame that is white everywhere. The failure is not the camera and not the conversion: it is treating white as a color with a hue. White is low saturation with high value, so the mask needs two bounds and a fully open hue channel. This how-to gives the starting bounds, a percentile method that tunes them on your own frames, the cv2.connectedComponentsWithStats call that turns the mask into labeled boxes, the area math that separates a real part from speckle, and the camera choice the webcam hides.

Why does the white mask come out empty in OpenCV?

The bounds decide everything, and the pair that circulates in white-detection snippets — lower_white = (0, 0, 0) with upper_white = (0, 0, 255) — demands a saturation of exactly zero, which a noisy real camera almost never produces, so the mask comes back black or speckled.

OpenCV stores 8-bit HSV as H from 0 to 179, S from 0 to 255 and V from 0 to 255. When a pixel has zero saturation its hue angle is undefined, and cv2.cvtColor writes H = 0 for it, so the hue channel carries no whiteness information at all. That kills both broken patterns at once. Demanding H = 0 with S = 0, as that pair does, passes almost nothing; widening the bounds to (0, 0, 0) through (255, 255, 255) passes every pixel in the frame, because each channel simply sits inside its range. The selection has to come from S and V: a ceiling on saturation, a floor on value, hue wide open.

The complementary problem — pulling one saturated color out of the scene as the region of interest — is a different bound set, and HSV color segmentation with a saturated single-color background covers it.

Why not convert the color image to gray and threshold it?

A gray conversion throws away the channel that defines white, because after cv2.cvtColor with cv2.COLOR_BGR2GRAY a white part is indistinguishable from any bright color: gray is a weighted sum of the three channels, so bright yellow, pale pink and white all land near the top of the range.

cv2.COLOR_BGR2GRAY computes 0.114 B + 0.587 G + 0.299 R, the Rec. 601 luma weights. One number cannot hold the difference between white and a bright color. Pure white at BGR (255, 255, 255) converts to 255. Saturated yellow at BGR (0, 255, 255) converts to 0.114 × 0 + 0.587 × 255 + 0.299 × 255 = 149.7 + 76.2 ≈ 226. Any gray cut that keeps a white part measured at 215 under dimmer light also keeps that yellow at 226; the two populations overlap, so no single gray threshold separates them. The HSV value channel fails the same way, because V = max(B, G, R) = 255 for that yellow too. Saturation is the discriminator: S = 255 for pure yellow and near 0 for white, which is why the mask thresholds S and V and leaves hue alone.

Gray thresholding is still the right tool for marks, text and machined surfaces; if that is your path, the integer-division trap in gray thresholds is worth two minutes before you set the cut.

Which HSV bounds actually select white?

Start from cv2.inRange(hsv, (0, 0, 200), (255, 60, 255)): hue fully open, saturation capped at 60 of 255, value floored at 200 of 255, in 8-bit OpenCV where H runs 0–179 and S and V run 0–255.

Each bound has one job. The saturation ceiling at 60 of 255 — about 24 percent — rejects anything visibly colored: saturated colors sit above 80, while white plastic and paper under white light measure 10 to 40. The value floor at 200 of 255 — about 78 percent — keeps the bright quartile and drops a mid-gray background, which sits near V = 128. The hue range stays fully open because white has no hue to match.

What moves the numbers: dim the line and the V floor has to come down 20 to 40 counts, which lets more gray background through; add colored ambient light and the S ceiling has to drop toward 40; a glossy part is fine, because its clipped highlights are already white. Tune from measurements, not from the monitor.

How do you tune the white bounds from your own frames?

Measure instead of guessing: grab 50 to 100 frames of the real part under the real light, crop the regions you know are white, and set the bounds at the 5th and 95th percentiles of S and V inside those crops, shifted by a margin of about 15 counts.

# crops: hsv patches of the known-white region, one per frame, shape (N, h, w, 3)
s_p95 = float(np.percentile(crops[:, :, 1], 95))    # example: 42.0
v_p05 = float(np.percentile(crops[:, :, 2], 5))     # example: 214.0
UPPER_S = int(round(s_p95 + 15))                    # 42 + 15 = 57 -> round to 60
LOWER_V = int(round(v_p05 - 15))                    # 214 - 15 = 199 -> round to 200

Example, with illustrative numbers: on a white polymer cap under a white LED ring, the S 95th percentile measures 42 and the V 5th percentile 214. The ceiling becomes 42 + 15 = 57, rounded to 60; the floor becomes 214 − 15 = 199, rounded to 200 — exactly the (0, 0, 200) to (255, 60, 255) pair from the previous section. The margin absorbs exposure wobble; the percentiles absorb the highlight and shadow pixels inside the crop.

Save the frames where the blob count jumps, too: recording frames automatically around a fault gives you a free re-tuning set for the day the lamp ages.

How do you turn the mask into detected objects?

Run cv2.connectedComponentsWithStats on the mask: it labels every white blob and returns a stats matrix whose rows are the blobs and whose columns — cv2.CC_STAT_LEFT, cv2.CC_STAT_TOP, cv2.CC_STAT_WIDTH, cv2.CC_STAT_HEIGHT, cv2.CC_STAT_AREA — give each bounding box and pixel area.

The label count includes the background as label 0, so the loop starts at 1. Clean the mask before labeling: cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel) with a 3×3 uint8 kernel erases single-pixel speckle, because open is an erode followed by a dilate. The AREA column is the one to gate on; the LEFT, TOP, WIDTH and HEIGHT columns draw the box; centroids returns each blob center as float64. Per-blob width, length and area measurement downstream follows the same columns — crack width, length and area measurement after segmentation walks that step, and measure blob area in a binary image by pixel counting shows the pixel-count check on a plain binary image.

What does the complete working script look like?

The detector fits on one screen: grab a BGR frame, convert with cv2.COLOR_BGR2HSV, mask with cv2.inRange, open with a 3×3 kernel, label with cv2.connectedComponentsWithStats, and draw a rectangle for every blob above the area gate.

1. Read a frame as a uint8 BGR array — cv2.VideoCapture for a webcam, the camera SDK's numpy buffer for an industrial camera.
2. Convert: hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV).
3. Mask: mask = cv2.inRange(hsv, LOWER_WHITE, UPPER_WHITE) with the tuned uint8 bounds.
4. Clean: mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel) with a 3×3 uint8 kernel.
5. Label: num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(mask).
6. Gate: skip blobs whose stats[i, cv2.CC_STAT_AREA] is below MIN_AREA.
7. Draw: cv2.rectangle per surviving blob, using the LEFT, TOP, WIDTH and HEIGHT columns.
8. Verify on the synthetic test below before pointing the script at production frames.

import cv2
import numpy as np

LOWER_WHITE = np.array([0, 0, 200], dtype=np.uint8)   # hue open, S ceiling, V floor
UPPER_WHITE = np.array([255, 60, 255], dtype=np.uint8)
MIN_AREA = 4200                                        # px^2, from the area example below

cap = cv2.VideoCapture(0)
kernel = np.ones((3, 3), np.uint8)

while True:
    ok, frame = cap.read()                  # frame: uint8 BGR, shape (H, W, 3)
    if not ok:
        break
    hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, LOWER_WHITE, UPPER_WHITE)
    mask = cv2.morphologyEx(mask, cv2.MORPH_OPEN, kernel)

    num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(mask)
    for i in range(1, num_labels):          # label 0 is the background
        if stats[i, cv2.CC_STAT_AREA] < MIN_AREA:
            continue
        x = int(stats[i, cv2.CC_STAT_LEFT])
        y = int(stats[i, cv2.CC_STAT_TOP])
        w = int(stats[i, cv2.CC_STAT_WIDTH])
        h = int(stats[i, cv2.CC_STAT_HEIGHT])
        cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 0, 255), 2)

    cv2.imshow('detections', frame)
    if cv2.waitKey(5) & 0xFF == 27:         # Esc quits
        break

cap.release()
cv2.destroyAllWindows()

# test: three white dots of known size on a mid-gray background
test = np.full((480, 640, 3), 128, dtype=np.uint8)       # V = 128 -> below the 200 floor
for cx, cy in [(100, 100), (320, 240), (540, 380)]:
    cv2.circle(test, (cx, cy), 40, (255, 255, 255), -1)  # area = pi * 40^2 = 5,027 px^2

hsv_t = cv2.cvtColor(test, cv2.COLOR_BGR2HSV)
mask_t = cv2.inRange(hsv_t, LOWER_WHITE, UPPER_WHITE)
n_t, _, stats_t, _ = cv2.connectedComponentsWithStats(
    cv2.morphologyEx(mask_t, cv2.MORPH_OPEN, np.ones((3, 3), np.uint8)))
assert n_t - 1 == 3 and (stats_t[1:, cv2.CC_STAT_AREA] > 4200).all()

The dots pass — each is about 5,027 px², above the 4,200 gate — and the gray background fails the V floor, so the assert holds: three labels, three boxes, one per dot.

How big must a white blob be before you count it?

Gate on area in pixels: with a 2592 × 1944 frame covering 100 mm of field, 1 mm is 25.9 pixels, a 4 mm dot spans 103.7 pixels and covers about 8,450 px², so any blob under 4,200 px² is speck, noise or a clipped edge.

Worked example: 2592 px ÷ 100 mm = 25.92 px/mm. A 4 mm dot is 25.92 × 4 = 103.7 px across. Its full area is 0.7854 × 103.7² = 0.7854 × 10,754 ≈ 8,450 px², and half of that — 4,220, rounded to 4,200 — is MIN_AREA in the script above. The 0.5 factor tolerates a dot clipped at the frame edge or partly occluded; for a strict presence check use 0.8, which puts the gate at 0.8 × 8,450 ≈ 6,760 px².

Run the same math in reverse — from a 4 mm feature and a 100 mm field to the pixel count the sensor must have — with the camera resolution calculator:

Which camera can see a white part at all?

Saturation is a spread across the three color channels, so a mono camera has no S channel to threshold and its mask degenerates into a plain brightness test that cannot tell a white part from any bright color.

Any color sensor feeds the same script — the frame arrives as a uint8 BGR numpy array whether it comes from cv2.VideoCapture or an industrial SDK, so nothing downstream changes. For a PC build, GigE and USB3 color area scan cameras give you the resolution the area math consumes; where no PC belongs in the cell, a color smart camera runs the inspection on the device. Most stations then hand the blob count or centroid coordinates to the PLC; sending results from a Python OpenCV loop to a PLC over Modbus TCP is a common integration pattern.

Three color sensors from the catalog that fit this job: the MV-CS200-10GC is a 20 MP GigE color camera with a C-mount, the MV-CU020-90GC is a 2.5 MP GigE color camera with a global shutter on a 1/1.7" sensor, and the MV-SC6050C-08M-XBN is a 5 MP color smart camera with an 8 mm M12 lens.

The camera finder filters both families by color, resolution and interface.

When does saturation-and-value segmentation fail?

The two-bound mask fails in three known ways — pastel or pale-gray backgrounds, specular highlights on shiny parts, and white parts on white surfaces — and each failure has a specific fix rather than a lower threshold.

1. Pale-gray or pastel backgrounds. A belt measuring V 210–230 with S 20–50 sits inside the bounds by definition, because it is genuinely whitish. No S or V bound fixes this: restrict the ROI to the strip where parts pass, or raise the V floor if the part is brighter than the belt. On busy scenes full of white features that are not parts, threshold white text on a busy background for OCR in Python shows the shape and position gates that finish the job.
2. Specular highlights on shiny parts. Clipped highlights measure S = 0 with V = 255 — perfectly white to the mask, even on a metal part you would never call white. Diffuse the light: a dome or coaxial source softens highlights, and the machine-vision lights finder groups the diffuse geometries.
3. White part on white conveyor. Identical colors cannot be separated by any S/V pair. Backlight instead: a bright-field backlight behind a translucent belt turns the part into a dark silhouette, and the logic inverts — threshold V below a ceiling instead of above a floor.

The decision, compressed: white is S at or below 60 with V at or above 200 and hue wide open; tune those two numbers from percentiles of your own frames; label with cv2.connectedComponentsWithStats and gate on an area derived from part size and field of view — 4,200 px² for the 4 mm dot in the example. When you are sizing the color sensor for a new station, the camera configurator starts at resolution and interface, and the contact page reaches the applications team for tuning help on sample parts.