Microcontroller vs PC for machine vision: split the work

A small sorting cell raises the oldest architecture question in machine vision: where do the pixels get processed? The microcontroller already runs the motors, the grippers and the operator panel, so it is tempting to hang a small camera module on it and process everything there. Sometimes that works. Often it stalls at a few frames per second, runs out of memory, or quietly misclassifies colors. This guide gives you the three calculations that place the boundary — frame memory, pixels per part and link bandwidth — then maps each side of the split to hardware you can actually order: smart cameras for on-device decisions, vision controllers for PC-class software and multi-camera cells. Run the numbers first; the tier follows from them.

Can a microcontroller run machine vision at all?

Yes, for one narrow slice of the field: color and blob location on small frames, where a microcontroller with about 520 KB of on-chip SRAM holds a 320 × 240 RGB565 frame in 153.6 KB and finds colored parts in it at roughly 10-15 fps.

What runs well is the integer-friendly set: thresholding, color classification inside a fixed window, connected-component labeling, centroid and bounding-box output, and part counting. Those passes touch each pixel once or twice, and camera-capable microcontroller modules typically finish them on QVGA frames in tens of milliseconds. The documented memory constraint sets the ceiling: one QVGA frame plus camera driver buffers lands near 230 KB, more than 40% of that 520 KB SRAM before your own algorithm allocates anything.

What does not run is everything correlation-shaped or float-heavy: pattern and template matching, subpixel edge measurement, lens undistortion and rectification, JPEG handling, and any second frame buffer. Those need many megabytes of working memory or per-pixel float throughput measured in seconds per frame, not milliseconds. The chip stays in the cell — as the motion, I/O and timing brain, not as the image processor.

How much memory does one camera frame actually take?

Frame memory is width × height × bytes per pixel, and that one product decides the split: a 320 × 240 RGB565 frame costs 153.6 KB, while a 1600 × 1200 frame of the same format costs 3.84 MB — memory this SRAM class never holds.

The raw frame is only the start of the footprint. Camera drivers hold DMA and line buffers, and most algorithms want a scratch copy, so the working set for a QVGA frame on this class of module runs to roughly 230 KB — about 44% of the 520 KB SRAM. That leaves room for exactly one algorithm and essentially none for a second view or a frame history. A 2 MP frame is 25× the QVGA frame and demands streaming into external RAM over buses this class often lacks; that ratio alone is why megapixel work migrates to the PC side. If the task really needs 12 MP or more, the pixel budget is its own decision — see high resolution machine vision cameras: how many pixels?.

How many pixels does each part need for reliable blob location?

Give every part 8-10 pixels across its smallest dimension, and treat 5 pixels as the floor where centroid and color decisions stay stable; at the 0.5 mm-per-pixel scale of a 160 mm field on a 320-pixel-wide sensor, that floor is a 4 mm part.

Worked example — sorting 25 mm pucks in a 160 mm wide field:

• Scale: 160 mm ÷ 320 px = 0.5 mm per pixel.
• Part: 25 mm ÷ 0.5 mm/px = 50 px across, a blob of roughly π/4 × 50² ≈ 1,960 pixels.
• Smallest reject you must catch: a 4 mm chip → 4 mm ÷ 0.5 mm/px = 8 px across, exactly at the floor.
• Upsample check: the same field on a 1600-px-wide 2 MP sensor gives 160 mm ÷ 1600 px = 0.1 mm/px and 250 px across the puck — 25× the blob area, which buys color sorting nothing and costs 25× the memory and bandwidth.

Blob area is what makes the floor real: centroid precision improves roughly with the square root of the pixel count, so a 1,960-pixel blob localizes to well under a tenth of a pixel, while a 5-pixel blob can wander a full pixel between frames. Keep the smallest classified feature at 8 px or better, and keep the largest expected part fully inside the frame with about 10% margin.

What frame rate can the small camera link sustain?

Budget the link bandwidth before promising a frame rate: a 153.6 KB QVGA frame at 15 fps is 2.3 MB per second, at the ceiling of a 20 MHz camera link — so this class delivers a realistic 10-15 fps, ample when a feeder presents one part per second.

The arithmetic generalizes to any link: bytes per frame × frames per second = MB/s, and the link ceiling divides it back. A 20 MHz serial camera link carries at most 20 Mbit/s ÷ 8 = 2.5 MB/s, so 153.6 KB frames top out near 15 fps once protocol overhead lands. Faster clocks and parallel interfaces raise that ceiling somewhat, but SRAM caps useful fps harder than the link does: a second frame in flight needs another 153.6 KB that is not there.

For moving parts, check travel per frame: a belt at 50 mm/s under a 15 fps camera advances 50 mm ÷ 15 ≈ 3.3 mm, about 6.6 pixels at the QVGA scale — so trigger on a part sensor and strobe the light to freeze it. The pixels-speed-blur trade for conveyors is worked through properly in sorting conveyor camera selection: pixels, speed and blur, and the same gap between acquisition and processing rate shows up at every scale; acquisition faster than processing: fix the frame rate gap is its fix.

When does the workload force a PC or vision controller?

Four workloads cross the line immediately: pattern or template matching, calibrated measurement with lens undistortion, more than one camera on one decision, and any frame too big for on-chip SRAM — in practice, most tasks above roughly 1-2 megapixels with real algorithms attached.

• Pattern matching localizes parts under rotation, scale and lighting drift by correlating over hundreds of search positions, which needs cached windows and DSP-class throughput. Color pattern matching for reliable part localization shows the technique, and edge-based pattern matching for stable results when lighting changes the lighting-tolerant variant.
• Calibrated measurement runs per-pixel float undistortion and mm-per-pixel scaling: seconds per frame on microcontroller silicon, milliseconds on PC silicon.
• Multi-camera cells need arbitration and buffering for several streams plus one combined decision — several frame buffers exceed single-frame SRAM by definition, which is why the vision controllers built for this expose multiple camera ports alongside built-in light channels.
• On-device processing absorbs some of this without a PC, because smart cameras run acquisition, algorithms and decision in one housing. How much a camera itself can absorb is its own question — on-camera image processing: what can you offload? — and the camera finder filters the options by resolution, shutter and interface.

What does the split look like in a working cell?

The microcontroller keeps motion, I/O and strobe timing where microsecond jitter matters; the vision side returns a small result packet — part class, x-y in robot base coordinates, angle, timestamp — and a 30-byte packet crosses a 115200-baud serial link in about 3 ms.

Map every function to the side that owns its constraint:

• Microcontroller side: motor commutation and interpolation, pneumatics and grippers, part-present sensing, strobe triggering, and light on/off switching with flash current, through the controller's own light outputs or dedicated light controllers.
• Vision side: acquisition, the algorithm, and the result packet, sent over TCP or serial — send camera part coordinates to a robot controller over TCP shows the message pattern.
• The handoff rule: results travel, images never. An image in the handoff recreates the memory problem you just moved off the chip.

When the pixel rate itself outruns any CPU — line scan at tens of kilohertz, or preprocessing that must keep pace with every pixel — the same architecture question returns as FPGA versus microcontroller, and this FPGA and microcontroller work-split guide walks through how such projects divide pixel-rate and control-rate duties.

Which hardware covers each side of the split?

Three hardware tiers cover the split: a smart camera or vision sensor for one view with fixed algorithms on the device, a vision controller for PC-class software and several cameras, and an area scan camera feeding a controller when the lens and sensor must be chosen separately.

On-device tier — one view, one decision, no separate computer:

The MV-SC3050XC-16M-ATN-H is a 5 MP color vision sensor with a global shutter on Fast Ethernet. The MV-SC5060XM-08M-WBN carries 6 MP mono resolution with an 8 mm M12-mount lens on GigE, and the LS-SMC-0063 pairs a 1.6 MP sensor with its 16 mm lens at 60 fps over GigE. In this tier the split dissolves: acquisition, processing and decision share one housing, and only the result leaves it.

PC-class tier — full vision software, several cameras, built-in light outputs:

The MV-VC2140-128G40-1T is a compact Intel N-series controller with 8 GB of memory, 3-4 LAN ports, 2 USB 3 ports and 4 built-in light channels — the entry point once cells outgrow on-device processing. Above the compact tier:

The MV-VC3301H-128G66 steps up to an Intel Core i3 with 5+ LAN ports and PCIe expansion, and ships with a VM6200Pro license. The MV-VC5525-256G70-NN moves to an Intel Core i5 with 32 GB of memory, PoE camera ports and 8 built-in light channels. The MV-SVC2506-16G60-128G provides 4 camera GigE ports plus 1 configuration port and 1 communication port, with 16 GB of memory. Where sensor and lens must be chosen independently — C-mount area scan cameras from sub-3-MP sensors to over-25-MP bodies — the controller is simply the compute side of the same split.

Which side of the split does your task land on?

Run one calculation first — smallest part size divided by millimeters per pixel — and the answer falls out: if that quotient is 8 or more and the frame fits in SRAM, a microcontroller can do it; if not, put the pixels on a smart camera or vision controller.

Then check the rest in order, stopping at the first failure:

1. Memory: width × height × bytes per pixel, plus driver buffers, fits on-chip with your algorithm — for 520 KB-class parts that means QVGA, not more.
2. Pixels per part: smallest classified feature ≥ 8 px across; below that, no algorithm recovers the information.
3. Bandwidth and rate: bytes per frame × fps inside the link ceiling, with a strobe for anything that moves.

Per case, the decision:

• Color or shape sorting at QVGA, parts over the 8 px floor, one camera: the microcontroller path works, and it is the right one when the same chip already owns motion and I/O.
• The same task, but you want supported tooling or an algorithm that evolves without reflashing firmware: the smart cameras tier does the identical job on-device.
• Pattern matching, calibrated measurement, text, codes, more than one camera, or frames above about 2 MP: a vision controller with PC-class software is the tier to start from.
• Codes and labels specifically: decoding lives in dedicated vision tooling, and barcode reader verification for label inspection explains what a verified read means on a label.

When the usual answer is wrong

The hobby answer — do the vision on the microcontroller because it is already there — fails in production for a reason unrelated to compute: maintenance. A new part or a lighting change means reflashing firmware on the microcontroller path, while the smart camera and controller paths take the same change as data. The enterprise answer — a PC for everything — fails at the other end: a single-view, fixed-algorithm check pays for boot time, licenses and administration it never uses. Let the memory, pixel and bandwidth formulas pick the side, not habit.

If the numbers land you in the middle, more architecture guides sit on the machine vision systems topic page, and the contact page reaches the applications team when the split stays ambiguous.