Machine vision camera on Raspberry Pi: build the capture stack

A Raspberry Pi runs a genuine machine-vision camera once three things line up: the camera's interface, the bandwidth the board can move, and a GStreamer pipeline that holds the camera's frame rate. Get them wrong and you spend a week chasing dropped frames; get them right and the stack is boring, which is the goal. This guide gives you the interface decision, the bandwidth arithmetic for USB 3, GigE and MIPI CSI-2, a test pipeline you can run in an afternoon, and the shop cameras that fit a Pi-class budget. It is written for engineers prototyping an inspection station or a guidance camera on a Pi-class board. If you only need proof that the capture path works, a simple task such as a Raspberry Pi machine-vision dice-dot counting build already runs happily on a held frame rate.

Which camera interface should you choose for a Raspberry Pi?

USB3 Vision is the default choice for one camera: a USB 3.0 port sustains roughly 350-400 MB/s in practice, enough for most inspection jobs, while GigE Vision wins beyond USB's 3 m cable reach and MIPI CSI-2 makes sense only when the sensor already has a working Linux driver.

The interface decides your software stack as much as your cable. UVC-class USB cameras appear as V4L2 devices and feed straight into GStreamer's v4l2src. Industrial USB3 Vision cameras usually need the maker's camera library and its GStreamer source element instead. GigE Vision packets bypass V4L2 entirely, so you need a GenICam-based library with its own source element. CSI modules need a kernel sensor driver, then libcamera on top.

Browse area scan cameras by interface to see the field; the filter counts make the shape of the market obvious, with several hundred GigE and USB3 models each.

How much bandwidth can a Raspberry Pi actually move?

Budget about 350 MB/s for a USB 3 camera, about 110 MB/s for the GigE port, and roughly 250 MB/s for the two-lane CSI connector; on a Pi 4 the Ethernet controller hangs off the same USB 3 bus, so camera and network traffic share that 350 MB/s.

Each number has a basis. The USB 3 bus signals at 5 Gbit/s, 8b/10b coding leaves 4 Gbit/s of payload (500 MB/s), and host controllers sustain 70-80% of that. GigE is fixed by the wire: 1 Gbit/s ÷ 8 ≈ 125 MB/s of line rate, about 110 MB/s after packet overhead. The two-lane CSI figure comes from the Pi 4's camera connector; a Pi 5 carries two 22-pin connectors with four lanes each, roughly doubling CSI capacity per connector. Where those caps bite, the MIPI lane bandwidth limits on frame rate and the frame-rate budget of GigE Vision break the arithmetic down further, and choosing an ARM board for USB3 camera acquisition covers the host side.

What does your camera's stream actually demand?

Multiply width by height by frame rate by bytes per pixel: a 4000 × 3000 sensor at 30 fps in 8-bit Bayer needs 360 MB/s, which saturates a Pi's USB 3 port, so you either drop to 15 fps, halve the ROI, or pick a smaller sensor.

Three worked examples, arithmetic shown:

• USB 3. 4000 × 3000 = 12,000,000 pixels × 1 B (8-bit Bayer) = 12 MB per frame. 12 MB × 30 fps = 360 MB/s, or 2.88 Gbit/s of payload: the entire port. At 15 fps, or a 4000 × 1500 ROI, demand halves to 180 MB/s (1.44 Gbit/s) and the pipeline breathes.
• GigE. A 2.5 MP frame in 8-bit Bayer is 2.5 MB. The port's 110 MB/s ÷ 2.5 MB ≈ 44 fps — the link, not the camera, sets that ceiling.
• CSI. 1920 × 1080 = 2,073,600 pixels × 30 fps × 1.25 B (RAW10) = 77.76 MB/s, or 622 Mbit/s of payload. Across two lanes that is 311 Mbit/s per lane, comfortably inside the roughly 1 Gbit/s per lane rate class those connectors run at.

Target 60-70% of the port budget and keep the rest for debayering bursts and your algorithm. Shrinking the ROI instead of the frame rate changes the trade differently; the multi-ROI frame rate arithmetic works that case through.

How do you build the GStreamer capture pipeline?

Every path ends in a GStreamer source element: v4l2src for UVC and vendor USB cameras, the GigE Vision library's source for network cameras, and libcamerasrc for CSI sensors on a Pi 5 — pick the source first, then append debayer, convert and sink elements behind it.

In order:

1. Enumerate. A UVC or vendor USB camera appears as /dev/video0 and its neighbours; v4l2-ctl --list-formats-ext -d /dev/video0 lists the pixel formats and frame sizes the device offers.
2. USB3 Vision. Install the maker's GStreamer source for ARM, matched to your OS architecture — a 32-bit ARMHF build will not load into a 64-bit OS image.
3. GigE Vision. The open-source Aravis library covers many cameras with its own GStreamer source element; set the NIC's MTU to 9000 first (below for why), and keep camera and host on one subnet for discovery.
4. CSI. The sensor driver arrives through the OS's device-tree overlay, libcamera binds it, and libcamerasrc feeds GStreamer. If a CSI camera has no ISP in the path, what raw sensor output means tells you what to expect from the pixels. A camera that loads the wrong sensor driver is an overlay problem, not a camera fault — diagnose it from the kernel log.

Then prove the pipe with no display in the way:

# Test pipeline: the element names are GStreamer's; the device path, caps
# values and buffer count are examples — replace them with your camera's
# enumerated values.
gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=300 ! \
  video/x-bayer,width=1920,height=1080,format=rggb ! \
  bayer2rgb ! videoconvert ! fakesink sync=false

If the board only takes CSI and the camera you want is USB, a converter exists — what a CSI-to-USB bridge board must include lists the non-negotiables.

Why does the frame rate drop below the camera's spec?

Three causes account for nearly every shortfall: the kernel's 16 MB usbfs buffer cap starving USB 3 transfers, CPU debayering of a large Bayer stream, and GigE packet overhead at the default 1500-byte MTU — fix them in that order, starting with usbfs_memory_mb.

The usbfs cap is a Linux default, not a Pi quirk: a bulk stream exhausts 16 MB of device buffers within two 12 MB frames in flight, and transfers stall. Raise the cap with the kernel's usbfs_memory_mb parameter (usbcore.usbfs_memory_mb=1000 on the command line) and re-run the 300-frame test. The USB3 low-frame-rate fix covering usbfs memory walks the whole procedure.

Debayering is the silent killer. bayer2rgb outputs about 4 bytes per pixel: a 12 MB Bayer frame becomes 48 MB, and at 30 fps that is 1.44 GB/s of pixel traffic — more than a Pi's ARM cores give in software. Stream raw Bayer, debayer only the region you inspect, or use the maker's NEON-optimised plugin where one exists.

On GigE, count packets: 110 MB/s ÷ 1500 B ≈ 73,000 packets per second, each costing interrupt and copy time. An MTU of 9000 B cuts that to about 12,000 — set jumbo frames on the camera and the NIC both.

Exposure sets its own ceiling: fps ≤ 1 ÷ exposure time, so a 20 ms exposure caps you at 50 fps before bandwidth is even involved. The GigE frame-rate drop at longer exposures explains the mechanism.

One isolation test separates hardware from software: if plain v4l2src shows frames but the maker's source element finds no camera, the device node and USB link are fine and the fault sits in the library's discovery.

Which shop cameras fit a Pi-class build?

Aim for a camera whose full-resolution stream stays under roughly 200 MB/s: a 2.5 MP GigE model, a 5-6 MP USB3 model, or a board-level module when the camera must disappear into your machine — all three leave headroom for debayering and your algorithm.

The camera finder filters by interface, shutter and resolution, and the camera configurator walks sensor, lens mount and format in one pass. Three profiles cover most Pi builds.

For a GigE build, the MV-CU020-90GC is a 2.5 MP color camera with a global shutter, a 1/1.7" sensor and a C-mount.

Its stream is gentle on the link: 2.5 MB per 8-bit frame puts the Ethernet ceiling near 44 fps, the right neighbourhood for conveyor-speed inspection, and the global shutter freezes motion at those speeds.

When you want pixels more than speed, the LS-CMR-0166 pairs USB3 with a 12 MP IMX226 sensor, a 1/1.7" rolling shutter and 4000 × 3000 output at 30 fps.

At full spec that stream costs 360 MB/s, so on a Pi-class port run 15 fps (180 MB/s) or a 4000 × 1500 ROI; on a desktop-class host the full 30 fps is realistic. Rolling shutter suits static parts; for moving parts stay global.

For a camera that hides inside the machine, the LS-BCM-0003 is a 3.1 MP USB3 board-level camera with an IMX252 sensor, a 1/1.8" global shutter, a C-mount and a 120 fps rating.

At 8 bits, 3.1 MB per frame means 60 fps draws 186 MB/s — inside a Pi's USB budget — while the full 120 fps (372 MB/s) wants a desktop host. Mechanical integration is half the work with modules; choosing a compact machine-vision camera for a 4 cm field walks one tight-space selection end to end, and the board-level cameras category holds the rest.

When should you step up from a Pi to a vision controller?

Move up when the maths fails twice: more than one GigE camera over a single Ethernet port, or a processing load that leaves no CPU headroom at your inspection rate — a vision controller gives you several camera ports, PoE options and built-in light channels in one fanless box.

The triggers are countable. Two GigE cameras need two ports, and the Pi 4 has one; the aggregate 110 MB/s would split between them. When acquisition outpaces your algorithm, the gap only grows — the frame-rate gap fix diagnoses which side is the bottleneck. Lighting is easier there too: the category spans 4, 6 and 8 built-in light channels, PoE camera ports that power GigE cameras over the same cable, and five or more LAN ports on 150 models.

For the step-up build, the MV-VC2140-128G40-1T runs an Intel N-series N97 with 8 GB of memory plus a 128 GB SSD and a 1 TB HDD, and carries 3-4 LAN ports, 2 USB 3 ports and 4 built-in light channels.

That is Pi-class compute with cell-level I/O: the vision controllers category narrows it to a shortlist. For the full loop from camera to robot, PC-based vision guidance with a GigE Vision camera shows how the pieces connect.

What do you check before ordering parts?

Four checks kill most returns: match the vendor library build to your OS architecture (32-bit ARMHF will not load in a 64-bit OS), confirm the CSI connector's lane count, measure your cable run against USB's 3 m limit, and re-run the bandwidth formula with your real frame rate.

The full list, with why each matters:

• Architecture match. Camera libraries and GStreamer plugins are built per architecture; a 32-bit ARMHF binary will not load into a 64-bit OS image, and the failure looks like a missing plugin.
• CSI driver path. A CSI module is only as good as its sensor driver; CSI-2 camera compatibility beyond the connector lists what to settle before you buy, and the host-side resolution limits cover the board's side.
• Cable runs. USB 3 tops out near 3 m passive; a longer run means an active cable or a switch to GigE, which reaches 100 m class.
• Power. Count camera plus light; a Pi 4 allots about 1.2 A to all USB ports combined, so the light gets its own supply or controller channels.
• Bandwidth margin. Re-run width × height × fps × bytes per pixel for every camera, and keep about 30% of each port free.
• Interface landscape. If the CSI path looks tight, the embedded interface comparison from CSI-2 to GMSL2 maps the alternatives.

When the shortlist is set, wire the pipeline from the test line above and measure 300 frames into fakesink before writing any algorithm code. If the numbers land between two options, send them through the contact page, and a configurator pass — started at the camera step — settles the rest, as does the board camera evaluation checklist for embedded builds.