Bayesian color image segmentation: train, classify, threshold

A color cutoff like R > 150 survives exactly one change of lamp, lens or white balance. Bayesian color image segmentation replaces hand-picked thresholds with models trained on your own parts: you draw sample regions for each color class, Bayes' rule assigns every pixel to the class it fits best, and a measured reject band holds out the pixels that fit nothing well. This guide gives the classifier arithmetic, a worked pixel-level example you can check by hand, the lookup-table method that brings a 300 × 200 image down to a few milliseconds, and the thresholds that keep glare and shadows out of your mask.

What Bayes' rule decides for one pixel

For every pixel and every trained class, the classifier computes one number: the posterior probability of that class given the pixel's R, G and B values. Bayes' rule says the posterior is the likelihood of the color under the class, times how common the class is in the scene (its prior), normalized across all classes.

The product form treats the three channels as independent within each class — the naive assumption. Real channels are correlated, but for saturated, visually distinct colors the class separation dwarfs that correlation. You never need the denominator to pick the winning class; you compare numerators. You need it only for an absolute posterior to threshold against, which is exactly what the reject band uses.

Train each class from labeled regions

Training is drawing, not tuning. On a few production frames, draw regions over every shading a class shows: flat top, curved edge, highlight, shadow side. From those labeled pixels, estimate a mean and a standard deviation per channel per class — or a 32-bin histogram per channel if you want no distribution assumption.

A 32-level-per-channel histogram has 3 × 32 = 96 bins per class, so budget about 10 samples per bin: roughly 1,000 pixels per class. The Gaussian model needs less, and the standard error of σ is about σ/√(2(N−1)): 1,000 pixels put the estimate within 2.2% of the true value, while 100 pixels leave a 7.1% error. A few hundred well-chosen pixels per class is a solid start.

Set each prior to the class's share of training pixels — a cap that covers 5% of the frame gets a prior of 0.05. Equal priors are the honest default until you know the scene.

Lock exposure, gain and white balance in manual before you draw anything. Training on frames whose sRGB rendering shifts values between configurations poisons every mean — see how sRGB rendering shifts image brightness. Stray room light does the same damage between shifts; controlling ambient light in vision cells covers the shielding and timing.

If the cell is still on the bench, capture training frames with the same camera family that will inspect. A 2.5 MP color GigE model with a 1/1.7" global-shutter sensor, such as the MV-CU020-90GC, suits typical color-sorting fields — browse the area scan cameras for the full range.

RGB or HSV as the feature space

In the sector where red dominates, hue = 60° · (G − B)/(R − min(R,G,B)). Sensor noise moves that fraction quickly when the channel spread is small. Take a washed-out pixel (200, 180, 180): saturation is 20/200 = 10%, and two counts of noise (G to 182, B to 178) swings hue from 0° to 60° × 4/22 ≈ 11°. The same noise on a saturated pixel (200, 100, 100), at 50% saturation, moves it 60° × 4/102 ≈ 2.4°. Hue also wraps at the 0°/360° seam — where red lives — and goes undefined near the gray axis. Default to RGB with locked lighting; train on hue only when brightness varies and saturation stays above roughly 30% across the class.

Worked example: a 160, 60, 60 pixel against two trained classes

Inputs, from a training run on a red part carried over a gray belt (example values; your own run produces its own):

• Red cap class: per-channel mean μ = (180, 40, 40), standard deviation σ = 20, from a 1,000-pixel ROI.
• Gray belt class: μ = (80, 80, 80), σ = 20.
• Priors 0.5 and 0.5.
• Pixel to classify: (R, G, B) = (160, 60, 60).

The rule in words: measure each channel's distance from the class mean in units of that class's σ, square and sum, and the likelihood is the Gaussian product — the constant 1/(σ√(2π)) = 1/(20 × 2.507) = 0.01995 per channel, times exp(−half the squared sum). With equal priors, the posterior is the class's share of the two likelihoods.

Cap z-scores: (160−180)/20 = −1, (60−40)/20 = +1, (60−40)/20 = +1. Each channel contributes 0.01995 × e^(−0.5) = 0.01995 × 0.6065 = 0.01210, so L_cap = 0.01210³ = 1.77 × 10⁻⁶.

Belt z-scores: (160−80)/20 = +4, (60−80)/20 = −1, −1. Red contributes 0.01995 × e^(−8) = 0.01995 × 3.355 × 10⁻⁴ = 6.69 × 10⁻⁶; green and blue contribute 0.01210 each, so L_belt = 6.69 × 10⁻⁶ × 0.01210² = 9.8 × 10⁻¹⁰.

Posterior(cap) = 1.77 × 10⁻⁶ / (1.77 × 10⁻⁶ + 9.8 × 10⁻¹⁰) = 0.9994. That clears 0.9, so the pixel is labeled red cap.

The likelihood ratio is about 1,800, so priors barely move it: with the cap covering 5% of the frame, the posterior is 0.05 × 1.77 × 10⁻⁶ / (0.05 × 1.77 × 10⁻⁶ + 0.95 × 9.8 × 10⁻¹⁰) = 8.85 × 10⁻⁸ / 8.94 × 10⁻⁸ = 0.990 — still the cap. Priors decide matters only near the class boundary. A blend pixel of (130, 60, 60) scores squared distances of 6.25 + 1 + 1 from both class centers, the likelihoods tie at a posterior of exactly 0.500, and the threshold rejects it — the right answer for an edge between the two colors.

Set the reject threshold at 0.9

Label a pixel only when the best posterior clears 0.9, and treat 0.5–0.9 as reject: those pixels stay out of the mask. The band is where glare, shadows and color blends land — the boundary pixel above scored exactly 0.500. A mask with explicit unknowns keeps blends out of your area counts instead of splitting them silently between classes.

Make it an operating rule: overlay the reject band on 20 or more known-good and known-bad parts before release. If good parts send more than about 5% of their pixels to reject, do not lower the threshold to 0.7 or 0.6 — that admits the blend zone where false accepts live. Add a class for the offending shading instead: a shadow class trains in minutes, and adaptive thresholding for uneven part shading shows the symptom set. Masking ambiguous areas out of training helps too — see training with a validity mask.

Classify from a lookup table to stay fast

Direct evaluation costs one Gaussian per class per pixel: a 300 × 200 image with three classes is 60,000 × 3 = 180,000 evaluations, about 18 ms per frame at an order-of-magnitude 100 ns each. The standard fix is a decision table. Quantize each 8-bit channel to 32 levels (shift right 3 bits), giving 32 × 32 × 32 = 32,768 cells, and precompute the winning class and its posterior for every cell.

Building the table costs 32,768 × 3 ≈ 98,000 evaluations — about 10 ms, roughly half of one naive frame. Every frame after that is one indexed read per pixel: index = 1024·R_q + 32·G_q + B_q, where R_q is the 5-bit red value. Sixty thousand reads at about 10 ns is 0.6 ms, and a few milliseconds with mask copy and overlay rendering included. That puts a three-class, 300 × 200 classification in the few-millisecond range on a vision-controller-class CPU — the table pays for itself in the first frame.

Quantization costs almost nothing here: with σ = 20 the 8-count cell is 0.4σ wide, so posteriors move a few percent at most, and an argmax with a 1,800-to-1 likelihood ratio does not flip.

Where the classifier runs

The algorithm is plain array math, so the platform decision is image volume, not exotic compute. For a one-camera cell, an x86 smart camera runs train, classify and measure on board — see the smart cameras range. The LS-SMC-0017 pairs an x86 platform with an 8.8 MP, 1", C-mount, global-shutter sensor: a single device for the whole chain.

When several cameras share one controller and its light ports, move up to a vision controller. Two current tiers cover the range:

The entry tier carries 8 GB of memory and four built-in light channels for one or two cameras; the tier above adds PoE camera ports, eight light channels and 32 GB of memory for multi-camera cells. Size the sensor first — the pixel-rule guide to camera resolution for defect detection gives the arithmetic — then filter by mount and shutter in the camera finder. The vision configurator walks the whole chain from part to parts list.

The LabVIEW implementation path

LabVIEW's vision library ships the pair this article describes: a Naive Bayes classifier step that learns class statistics from labeled sample regions on training images, and a classification step that produces the labeled image from a live color frame. Draw regions, train, classify, then convert the labeled output to a mask. For that conversion, this walkthrough of configuring color threshold masks in LabVIEW shows the settings in detail.

To own the arithmetic yourself, it is three array operations: extract the R, G and B planes, shift each right by 3 bits, and index a prebuilt 32,768-entry decision array with 1024·R_q + 32·G_q + B_q. Getting live frames into your own code is covered in calling a camera SDK DLL from LabVIEW, and passing inspection results into the rest of the application in smart camera LabVIEW integration.

When a fixed threshold beats the classifier

Two situations argue against Bayes. First, one bright part color on a stable dark background: Otsu thresholding in Lab color or the local-mean method for bright objects reaches the same mask with no training step at all. Second, classes that overlap in color space: when the best posterior plateaus near 0.6–0.7 no matter how many samples you add, the same RGB values genuinely occur under both labels, and no threshold fixes that. The cure is a new feature — saturation, texture, a second wavelength band — a lighting-geometry change that separates the colors, or a trained detector when appearance is too rich for color alone; rule-based versus AI inspection sets that choice out.

Where the color classes are genuinely distinct, Bayes earns its keep on multi-class problems. Sorting five wire-insulation colors under one light set is the classic case — see five colors, one set-up.

From labeled mask to measurement

The labeled mask is the beginning, not the deliverable. Blob tools turn labels into numbers: area by pixel counting gives class coverage and part presence, automatic per-region line counts verify features inside each color zone, and color blob tracking follows one class across frames for positioning. Erode each class region by one or two pixels before measuring when edge blends would skew areas or centroids.

Checklist before you trust the mask

• Draw training regions on production frames under production lighting — posteriors describe the light they saw and no other.
• Lock exposure, gain and white balance in manual — auto modes move pixel values and shift every trained mean.
• Budget about 1,000 pixels per class across flat, curved, highlight and shaded areas — that holds σ within about 2%.
• Set priors to real scene fractions, or to equal values on purpose — not by accident.
• Overlay the 0.5–0.9 reject band on 20 or more good and bad parts — it shows exactly where the threshold cuts.
• Hold the reject fraction on good parts at or below about 5% before release — above that, add a class or retrain.
• Cross-check the lookup table against direct evaluation on a few hundred random pixels — an indexing slip is otherwise silent.
• Erode class regions by one to two pixels before measurement — blended edge pixels bias areas and centroids.

If you want the hardware chain sized around this classifier, the vision system configurator builds it from your part and field of view, or tell us about the application and we will spec the camera, lens and controller with you.