Robot pick color detection: camera set-up that reads colors reliably

Your robot arm carries a camera, the tray holds parts in a handful of colors, and this cycle's pick must go to one color and not the others. Color detection for a robot pick fails in predictable ways: white balance drifts between frames, white and grey parts masquerade as pale colors, and a specular highlight punches holes in the blob. This guide walks the whole pipeline — HSV hue windows with a fixed tolerance, saturation and value gates, a locked camera, a white diffused ring light, and the blob rules that turn a mask into a pick command. One worked example sizes the lens, field of view, depth of field and exposure for a 30 mm part at a 300 mm standoff.

The decision, stated first

Build the detector this way, in this order:

• Convert the frame to HSV. On the OpenCV 8-bit scale, hue runs 0–179 and saturation and value run 0–255; halve the windows if your library returns hue in 0–360 degrees.
• Accept a pixel when it lies within ±10 hue units of the taught center for the commanded color, with saturation at or above 70 and value at or above 60.
• Teach every center on the real part under the final light, and keep any two centers at least 25 hue units apart.
• Treat saturation below 40 as achromatic: classify those pixels by value as white, grey or black, never by hue.
• Lock white balance and exposure before tuning anything.
• Light the part with a white diffused ring light mounted on the camera.
• Pick the largest blob whose area is 0.5–1.5 times the taught area; report "no target" instead of lowering a gate.

The rest of this guide is why those numbers hold and what moves them.

Why hue, not RGB

In RGB, color and brightness share channels. Dim the scene by half and the red channel of a red part falls with it, so a box tuned at one brightness starts rejecting the same part at another. Hue — the angle on the color wheel — separates what the color is from how bright it is. Shading moves value, bends saturation a little, and leaves hue where it was. That stability is why hue thresholds survive factory lighting that RGB boxes do not.

Hue is circular, and red sits across zero on the 0–179 scale. That is what the wrap term handles, and why red needs two windows in practice.

Hue windows for the common colors

Teach on the part, but start from these centers:

Orange and yellow sit only 15 units apart, so their ±10 windows overlap from 20 to 25. If a line runs both, merge them into one warm class or teach until the measured centers are at least 25 units apart; the spacing rule holds for every neighboring pair. A five-color production line built on this exact method is described in wire color classification in machine vision.

Keep white, grey and black out of the hue test

The hue of an achromatic pixel is noise: a white part under a warm lamp can land anywhere on the wheel, which is how white gets picked as pink. Saturation below 40 means the pixel carries no color to read — classify it by value: 200 and above reads white, 50 and below reads black, and the band between is grey. The 40–69 band is dim color; settle it from the taught histogram of the real part, not by guessing. The value floor of 60 discards shadowed pixels whose hue is equally meaningless.

Lock the camera before you tune

Auto white balance re-estimates the color cast every frame, and every re-estimate moves the hue center. Lock it to fixed gains — lock auto white balance to a fixed value for repeatable color — and if the camera only offers continuous auto, run a one-shot balance and freeze the result. Exposure and gain belong to the same discipline; runtime camera parameter control shows where those settings live in a running system.

The sensor must be color: a Bayer camera reconstructs hue from its filter mosaic, and when to choose a Bayer color camera lists the trade-offs.

Worked example: a 30 mm red cube at 300 mm

Inputs: the part is a 30 mm red cube; the arm stops with the camera 300 mm above it; the camera is a 1600 × 1200 color sensor with 4.5 µm pixels, so the active area is 1600 × 0.0045 mm = 7.2 mm wide and 1200 × 0.0045 mm = 5.4 mm high. Target field of view: about 200 × 150 mm, so the tray around the part stays visible.

Magnification: m = 7.2 / 200 = 0.036. Focal length: f = 300 × 0.036 / (1 + 0.036) = 10.8 / 1.036 = 10.4 mm. The nearest standard C-mount focal length above that is 12 mm — round up, because rounding down widens the field and shrinks the part.

Recompute with the real lens: m = 12 / (300 − 12) = 0.0417, so the field of view is 7.2 / 0.0417 = 173 mm wide and 5.4 / 0.0417 = 130 mm high. Scale: 173 / 1600 = 0.108 mm per pixel. The cube spans 30 / 0.108 = 278 pixels, which is 17% of the width — safely above the 10% field rule that camera orientation precision on small parts sets for accurate centroids. One cube face is about 278 × 278 = 77,000 pixels; at a 0.7 fill for a real object the taught blob lands near 54,000 pixels, and the gate is 27,000–81,000 pixels.

Depth of field: the tray height varies by ±10 mm, a 20 mm spread:

with c the acceptable blur circle, here one pixel = 0.0045 mm. At f/4: 2 × 4 × 0.0045 × 1.0417 / 0.0417² = 21.6 mm, so the 20 mm spread barely fits with no margin. At f/5.6 the same formula gives 30.2 mm. Decision: stop the lens at f/5.6.

Exposure: trigger after the arm settles and any exposure works. To capture on the fly at 50 mm/s, keep blur under one pixel: exposure ≤ 0.108 / 50 = 2.2 ms, so run 2 ms with the ring light continuous, or strobe the light for 200 µs, which at that speed smears 0.01 mm — a tenth of a pixel.

Decision for this cell: 12 mm lens at f/5.6, field of view 173 × 130 mm, trigger at rest or expose 2 ms, red windows 0–10 and 169–179, blob gate 27,000–81,000 pixels.

Check your own numbers in the lens focal length calculator or straight from here:

From mask to pick command

• Open the mask with a 3 × 3 kernel to delete single-pixel speckle, then run connected components — the order used in threshold then morphology.
• Keep the blobs inside the area gate and take the largest as the target.
• Convert its centroid to millimetres with the scene scale (0.108 mm per pixel here) and hand it to the robot frame — how the 2D pick cycle works covers the transform and grip offset.
• A blob above 1.5 times the taught area is two touching parts; reject and re-image rather than aim between them.
• Several valid blobs means one snapshot can serve several picks — pick a full tray per cycle.
• No blob passing the gates: report no target. Do not widen the hue window or drop the saturation floor to force a pick.

Run the first cycles at reduced speed with the cell's safety functions active; the vision gate decides whether to pick, not whether the motion is safe.

Hardware that holds the tune

For a PC-driven cell, a small color area-scan camera is enough. The MV-CU020-90GC is a 2.5 MP color camera on GigE with a 1/1.7" sensor, a C-mount and a global shutter, and the global shutter keeps a moving arm from smearing colors into streaks. Browse the area scan cameras or the camera finder for other resolutions.

If you would rather not put a PC on the arm, a smart camera decides on the device. The LS-SMC-0063 is a 1.6 MP smart camera with a 16 mm lens, its own white polarized light and a GigE interface, running at 60 fps — camera and light move as one unit, so the illumination geometry never changes as the arm travels. More options in smart cameras.

The light is a diffused ring bolted around the lens. Two fit this mounting — a 120 mm unit at 0° and a 100 mm unit with diffuser at 30°, both on 24 V:

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

Take the white LED version of whichever size you choose; colored LEDs pull every part toward their own hue and erase the differences you are measuring. Sizes and diffuser options are in ring lights and the light finder.

When hue windows fail

• Glossy anodized or plated colors throw white highlights whose saturation collapses, leaving holes in the blob. Cross polarizer on lens and light removes the specular bounce; budget a longer exposure, since polarizers cost light.
• Daylight or overhead shop light shifts the cast between shifts. The co-mounted ring and a short exposure overpower most of it; the rest is handled by controlling ambient light variation in pick and place vision.
• Two near-identical colors — forest and kelly green — have centers closer than 25 units and will not separate. Put them on a contrasting saturated background, as in HSV color segmentation with a saturated single-color background, or move to Lab thresholds with Otsu segmentation in Lab color.
• Wrist-mounted cameras add mount-specific failure modes; fixing wrist-camera color detection works through the ones that show up on robot-mounted heads.
• Color that exists only as a print on a white body: segment the print region, not the body, or the achromatic gate will discard the part.

Pre-flight checklist

• White balance locked, exposure locked, gain fixed.
• Every color taught on the real part under the final light; window set to the histogram peak ± 10.
• Centers at least 25 units apart; 100 frames logged per color with at least 99% of pixels inside the window.
• Saturation and value gates at 70 and 60; achromatic parts classified by value, not matched on hue.
• Blob gate 0.5–1.5 times the taught area; the merged-blob reject tested with two touching parts.
• Trigger at rest, or exposure computed from the blur rule for your arm speed.
• Twenty dry-run picks at reduced speed with grip offset inside tolerance before the cell runs at rate.

Once the part list is fixed, the configurator started at the camera step walks camera, lens and light together, more set-ups like this one sit on the applications topic page, and the contact page reaches the applications team for a review of your colors and lighting before you buy.