Grayscale camera for machine learning: halve your pixel budget

A machine-learning pipeline budgets bytes before it budgets accuracy: every frame has to fit in RAM, cross the camera link, sit in a training set and pass through the first convolution. Pixel format sets that budget. An 8-bit grayscale camera moves 1 byte per pixel where RGB565 needs 2 and RGB888 needs 3, so the same scene costs half or a third of the buffer, link time and storage. This guide works the arithmetic end to end, shows where the savings land in an embedded vision system, and gives you the decision rule for when a mono sensor holds model accuracy — and when dropping colour loses the inspection.

The pixel budget in one formula

Bytes per frame is the whole game, and it has one formula:

A 320 × 240 frame is 76,800 px, so the rows come from 76,800 B, 153,600 B and 230,400 B — rounded to 75, 150 and 225 KiB. The 1.6 MP column repeats the same arithmetic at 1,600,000 px.

Two rows share the 1-byte line and they are not the same thing. Mono8 comes from a sensor built without colour filters. Bayer Raw8 comes from a colour sensor that sends one undecoded byte per pixel, with colour reconstructed on the host by demosaicing. RGB565 itself packs colour into 2 bytes — 5 bits red, 6 green, 5 blue — and reading a grayscale stream as RGB565 in a viewer is a classic corruption bug (one such case on composite video output). The pixel-format row in the camera's GenICam feature list decides which format you get.

Depth can silently cancel the saving. A 10- or 12-bit mono stream usually unpacks to 2 bytes per pixel in memory — the same cost as RGB565 — which is why 8 bits is the format that buys the budget back. Where the extra depth comes from, and when the output format wastes it, is covered in why 10-bit sensors ship 8-bit images. When you do keep wider depth, save 16-bit grayscale images losslessly so the extra bits survive, and note that some display paths choke on 16-bit buffers — grayscale U16 display crash fix names the usual mismatch.

Where the halved budget lands

Four budgets shrink together. Here is each one with the arithmetic shown.

RAM and frame buffers

A five-deep ring of 320 × 240 frames holds 5 × 76,800 B = 384,000 B ≈ 375 KiB as Mono8, and 750 KiB as RGB565. On a microcontroller-class target that difference decides whether the pipeline fits at all. Allocate the ring once and write frames into it — per-frame allocation is its own failure mode.

Camera link and frame rate

A 1.6 MP frame is 1.6 MB at Mono8 and 3.2 MB as RGB565. A 100 Mbit Fast Ethernet link carries roughly 12 MB/s of usable payload, so the same sensor runs 12 ÷ 1.6 = 7.5 fps mono against 12 ÷ 3.2 = 3.75 fps in colour — the byte budget is the frame-rate budget. GigE moves about 118 MB/s usable, enough for 118 ÷ 1.6 ≈ 74 fps mono or ≈ 37 fps as RGB565. On USB3 the camera link seldom binds first; the per-camera USB3 budget math shows how to check it.

Training storage and training RAM

Take 20,000 labelled crops at 224 × 224 px. One crop is 224 × 224 = 50,176 px, so the set is 50,176 B × 20,000 = 1,003,520,000 B ≈ 1.0 GB as 8-bit grayscale and 3.01 GB as RGB888. Loaded for training as float32, the same sets occupy 4.01 GB and 12.04 GB before any augmentation copies. Scale the formula to your crop size; never guess it.

First-layer compute

The first convolution is the only layer whose cost scales with input channels. For a 96 × 96 input, 3 × 3 kernel, 16 filters and same-size output: MACs = 96 × 96 × C_in × 3 × 3 × 16. One channel: 96 × 96 × 9 × 16 = 1,327,104 ≈ 1.33 million MACs. Three channels: 3.98 million. A batch of 32 such inputs as float32 occupies 32 × 9,216 × 4 B = 1,179,648 B ≈ 1.2 MB mono against 3.5 MB RGB. Later layers are identical, so the saving lands exactly where a microcontroller inference budget is tightest.

Why mono holds accuracy when shape and texture separate the classes

Training and inference need information, not channels. If the classes differ in shape, texture, edge layout or overall intensity, colour is redundancy and the model relearns the same boundaries from one channel. Published results line up with that: a 2024 face-recognition study matched its colour pipeline's accuracy with grayscale-only training and testing, and object-recognition work has reported single-channel input matching or beating RGB where colour carried no class signal.

The sensor helps too. A monochrome sensor has no colour filter array, so every photosite receives the full band of light; data sheets listing both variants of the same sensor typically show the mono part with tens of percent higher quantum efficiency in the visible band, because the filters absorb part of the light in every channel.

Demosaicing also disappears. On a Bayer sensor, red and blue sites sit at half the pitch in each direction, so fine colour detail is interpolated rather than measured. Mono measures intensity at every site you paid for, which shows up as crisper edges for the model and a cleaner image at the same exposure. When a mono frame turns grainy at high gain, fixing grainy grayscale images is an exposure-and-lighting exercise before it is a filtering exercise.

When colour is the class signal

Some inspections are hue problems, and no byte trick fixes that. Ripeness and browning on food, wire colour codes, LED and indicator status, stain and oxidation colours, printed registration marks, multi-colour packaging — a mono sensor merges these into similar grey levels and the classes collapse. If a class difference is a colour difference, keep colour and spend the budget elsewhere.

Run the conversion test before you redesign hardware. Score your trained model on a grayscale-converted copy of the validation set, using the standard luma weights Y = 0.299 R + 0.587 G + 0.114 B. If validation accuracy drops by more than about one percentage point, colour was doing separation work; if the drop is inside your pass margin, mono is safe and the halved budget is yours.

There is a middle path. A mono sensor under a coloured LED converts a spectral difference into an intensity difference, so one channel can separate classes that would otherwise need hue — the logic behind choosing LED colour for contrast. Pick the band where part and background reflect most differently, then keep the light fixed so the model sees one stable mapping.

Choosing mono hardware for the job

Inside a machine, board-level cameras keep the footprint and the mechanics down, and most series ship mono and colour variants of the same optical format — plan the mono choice at selection time with the embedded camera selection guide. The MV-CB016-10GM-M-W pairs a 1/2.9" global-shutter sensor with a GigE port on an M12 mount, a common shape for a camera buried behind a bezel.

When the model needs finer detail — small text, hairline defects, dense scenes — step up the pixel count and re-run the byte arithmetic, because storage and link budgets scale with pixels, not with model size. How many pixels a task needs is settled by the pixel rule for defect detection, and the calculator below turns a feature size into a pixel count. The MV-CB060-10UM-B is a 6 MP mono bare board on USB3 with an IMX178 sensor; the MV-CS200-10UM is the housed version of the idea at 20 MP, C-mount, USB3.0, with a 1" IMX183 sensor.

All 2 share: Sensor type CMOS, rolling shutter; Pixel size 2.4 μm × 2.4 μm; SNR 41.5 dB; Gain 0 dB to 24 dB; Mono/color Mono; Pixel format Mono 8/10/10Packed/12/12Packed; Decimation Supports 1 × 1, 2 × 2, 4 × 4; Compatibility USB3 Vision, GenICam. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

When the model should run on the camera itself, smart cameras and vision sensors carry the inference on board, and many connect over 100 Mbit Fast Ethernet — exactly where the Mono8 frame-rate arithmetic above pays for itself. Once the resolution is sized, filter for mono models in the camera finder.

Production lines that need wide fields or high throughput pick from area scan cameras, where the mono variants of a series give the same pixel count with the higher quantum efficiency — at GigE speeds the halved stream is often the difference between the frame rate you need and the frame rate you almost get.

When the usual answer is wrong

The halving belongs to the pixel format, not to the sensor. A colour camera set to Bayer Raw8 also sends 1 byte per pixel and halves the stream against RGB565; you pay for demosaicing on the host instead of losing colour. If the constraint is bandwidth on an existing colour camera, switching its pixel format to an 8-bit raw mode recovers the budget without touching hardware — the price is host CPU, not information.

Mono is also not a fix for a task the sensor cannot see. It neither rescues an under-resolved image nor repairs bad lighting, and an operator-facing display or a downstream inspection that still wants colour can keep a colour camera in the system regardless. Where the model genuinely needs colour and the link binds, Bayer Raw8 with host demosaicing — or a second low-resolution mono camera dedicated to the classifier — beats degrading the colour path.

The decision, per case

• Shape, texture or intensity separates the classes, embedded budget tight: 8-bit mono end to end — Mono8 on the camera, single-channel input to the model. Half the buffer and link cost against RGB565, a third against RGB888, a 3× lighter first layer.
• Hue separates the classes: keep the colour camera, train on colour, and budget the link with 8-bit raw Bayer when bandwidth binds; the halving still applies to the stream.
• Hue matters but the platform is mono: move the spectral split into the lighting — coloured LED, mono sensor, one-channel model — and confirm with the grayscale conversion test before committing.

Size the sensor with the pixel rule, pick the mono variant in the finder, or start the camera step of the configurator and let it walk the choice; if the budget still does not close, talk to the applications team. More camera-side guides are collected under the cameras topic.