System designGuide

Read camera sensor data with an FPGA without an SDK

How to read camera data with an FPGA: match the sensor's parallel, LVDS or MIPI CSI-2 output, size the pixel clock, and when to use a frame grabber instead.

LumaSpec Vision11 min read Share on LinkedIn

On this page 9 sections
  1. 01What an SDK actually does, and what it never touches
  2. 02Match the receiver to the interface the sensor exposes
  3. 03Reading a parallel CMOS sensor
  4. 04Worked example: pixel clock for a 1.3 MP 10-bit sensor at 60 fps
  5. 05MIPI CSI-2 and SLVS-EC: start from vendor receiver IP
  6. 06When a frame grabber beats your own receiver
  7. 07Documentation to collect before you commit
  8. 08Checks before you build, and when not to build
  9. 09Questions

You want pixels in your FPGA logic, not in a vendor SDK's callback. The distinction that matters: an SDK never touches the sensor. It writes registers over a control bus and unpacks frames that a host interface has already delivered, so reading camera data without an SDK means tapping the sensor's electrical output itself. This guide shows you how to match the receiver to the interface the sensor exposes — parallel CMOS, LVDS or MIPI CSI-2 — how to size the pixel clock with real arithmetic, and when a frame grabber does the job better than anything you could build.

What an SDK actually does, and what it never touches

A camera SDK lives above a working interface. It discovers the device, writes the registers that set exposure, gain, ROI and trigger mode, receives frames through a transport layer, and unpacks pixel formats into buffers your code can use. GenICam and GenTL standardize that control and transport for GigE Vision, USB3 Vision, Camera Link and CoaXPress cameras. Not one line of SDK code ever sees the sensor's electrical output.

That settles what the FPGA path has to be: you skip the camera and the host interface and receive the sensor itself. Either the sensor gives you raw timing your logic can capture, or the device gives you a standard bus, and then the right receiver is a card, not your own design. Where that software boundary sits in practice — why one viewer works on a grabber while another fails — is the subject of our frame grabber SDK support article.

The decision in one line: standard camera, buy the receiver; raw sensor or board-level module, build the receiver. The rest of this guide sizes that build.

Match the receiver to the interface the sensor exposes

The sensor's datasheet timing section names its output, and that name decides the whole project. One path matters beside the image data: sensors are configured over I2C or SPI register writes, and without an SDK you write that boot sequence yourself. A sensor that never received its streaming-mode registers stays in standby and sends nothing.

Sensor or camera output What your FPGA must contain Effort Where you meet it
Parallel CMOS: pixel clock, frame valid, line valid, 8-16 data bits Input logic, frame and line counters Days Legacy and instrument sensors, moderate frame rates
Serialized LVDS or sub-LVDS pairs Deserializer per pair, channel alignment Weeks Faster sensors, tight board routing
MIPI CSI-2 lanes D-PHY receiver IP plus packet layer Weeks, with vendor IP Camera modules, mobile-derived sensors
SLVS-EC lanes Transceiver IP plus lane deskew Weeks, IP licensed High-resolution, high-frame-rate sensors
GigE, 10GigE, USB3, CoaXPress or Camera Link Nothing: a frame grabber is the receiver None Standard industrial cameras

The first two rows run on ordinary I/O pins, so any FPGA family qualifies. The two serial standards need multi-gigabit transceivers and receiver IP, which narrows the FPGA choice before you place a single pixel pipeline. The last row needs no design work at all: the area scan cameras in the shop all speak one of those standard buses, and their receivers are cards you order.

Reading a parallel CMOS sensor

The bus is synchronous, and the contract is three signals plus data. Sample the data bits on the pixel clock edge the datasheet designates, count pixels while line valid is asserted to find the end of a line, and count lines while frame valid is asserted to find the end of a frame. Setup and hold times from the datasheet timing section — not the sensor's headline frame rate — set how fast you can clock the bus reliably.

Configuration comes first. Exposure, gain, ROI and the clock range are register writes over I2C or SPI at power-up, in the order the datasheet's boot sequence gives. Skip a step and the sensor streams nothing, which looks exactly like a broken receiver.

Most bring-up faults are electrical rather than logical. A missing pixel clock is a wiring or level problem, and the parallel camera no pixel clock article lists the signals to measure in order. A bus that delivers pixels in the wrong order is a lane swap, and the parallel camera data bus alignment article shows how to verify the pixel mapping. If the sensor serializes its output over LVDS pairs, match the signaling before wiring anything, as our LVDS line scan camera to frame grabber article explains for the grabber case.

Worked example: pixel clock for a 1.3 MP 10-bit sensor at 60 fps

The inputs, and where each comes from:

  • Bus pattern, documented: a published application note for a 10-bit parallel CMOS sensor describes a receiver that gets 12 pixels in parallel through 12 data channels after every five sensor clocks — 2.4 pixels per clock. The pixel width is already inside that number.
  • 1280 × 1024 active pixels, the common SXGA 1.3 MP format — example input.
  • 60 fps target — example input, the rate a typical inspection line asks for.
  • 20% of frame time in blanking — example assumption; 10-30% is the usual band, and the datasheet's frame format gives the real number.

The rule in words: the sensor clock must deliver the active pixels of one frame every frame period, counting only the clocks during which the bus actually carries pixels.

f_clk = pixels per frame × frame rate ÷ (pixels per clock × active fraction)

The arithmetic:

  1. Pixels per frame: 1280 × 1024 = 1,310,720 px.
  2. Active pixel rate: 1,310,720 px × 60 fps = 78,643,200 px/s.
  3. Clock with no blanking: 78,643,200 px/s ÷ 2.4 px/clock = 32,768,000 Hz, about 32.8 MHz.
  4. Clock with 20% blanking (active fraction 0.8): 32,768,000 Hz ÷ 0.8 = 40,960,000 Hz, about 41.0 MHz.

Cross-check at the no-blancking minimum: 2.4 px/clock × 32,768,000 Hz × 0.8 = 62,914,560 px/s, and 62,914,560 ÷ 1,310,720 px = 48.0 fps. The minimum clock delivers 48 fps once blanking is real, not the 60 you were asked for.

The decision: clock the sensor at 41 MHz or above, inside the clock range its datasheet allows, and the 60 fps target holds with margin. If the documented maximum clock sits below 41 MHz, this 12-channel bus cannot reach 60 fps at SXGA — add data channels, because each doubling of pixels per clock halves the clock requirement, or accept the lower frame rate the arithmetic allows.

MIPI CSI-2 and SLVS-EC: start from vendor receiver IP

High-speed sensors and camera modules carry the pixel stream as packets, not as a synchronous bus. CSI-2 sends short packets for sync information and long packets for pixel data over D-PHY lane pairs, and a receiver must merge the lanes, decode the packets and check CRC before your logic sees clean pixels. That stack is months of work to hand-roll and is exactly what FPGA vendors ship as receiver IP — start from the IP.

Budget the bus before choosing the sensor, with the same division as the parallel example: total lane throughput is lanes times lane rate, and the frame rate ceiling is throughput divided by bits per frame. Our MIPI CSI-2 frame rate estimation article walks that arithmetic lane by lane. SLVS-EC pushes lane rates higher still and always rides transceivers plus lane deskew; the lanes, bytes and pixels of a design-in are worked through in our SLVS-EC sensor to FPGA design-in article.

Know what you are buying at the pixel level too. A direct CSI-2 read delivers the sensor's raw output, with none of the black-level, white-balance or demosaic processing an ISP applies; what that means in practice is the subject of our CSI camera with no ISP article. And a module built around a mobile platform usually exposes CSI-2 and nothing else — there is no parallel fallback to tap, which is why the receiver IP decision comes first.

When a frame grabber beats your own receiver

If the camera already speaks GigE, 10GigE, USB3, CoaXPress or Camera Link, the receiver problem is solved: the card supplies the PHY, the protocol engine, DMA into host memory and GenTL drivers your code calls directly. Your engineering time goes into the processing you actually wanted the FPGA for. Before ordering, run the checks in our camera and frame grabber compatibility article, and start from the frame grabbers category.

Camera Link still runs a lot of factory lines, and its receivers exist as frame grabbers, not as casual FPGA projects. The MV-GC1102IOL frame grabber is that receiver for a Camera Link camera.

Frame grabbersHikrobot MV-GC1102IOL Frame GrabberMV-GC1102IOL
Interface
Camera Link
$2,189.00View partBuild a system with it

CoaXPress covers long cable runs with high, guaranteed bandwidth. The MV-GX1008 is an octo CXP-6 frame grabber for CoaXPress cameras.

Frame grabbersHikrobot MV-GX1008 Octo CXP-6 Frame GrabberMV-GX1008
Interface
CoaXPress
$1,949.00View partBuild a system with it

GigE cameras need no grabber in the strict sense: a PoE network card is the capture part, and it feeds the same GenTL drivers. The MV-GE2002P is a dual-port PoE NIC for GigE cameras.

Frame grabbersHikrobot MV-GE2002P Dual-port PoE NIC, full height V2.0MV-GE2002P
Interface
GigE
Ports
2 ports
$249.00View partBuild a system with it

At 10GigE the same logic holds, and USB3 cameras connect through a host adaptor rather than a grabber. The MV-GT1102P is a dual 10GigE frame grabber, and USB3 cameras plug into a quad USB3.0 host adaptor such as the MV-GU2204.

Frame grabbersHikrobot MV-GT1102P Dual 10GigE Frame GrabberMV-GT1102P
Interface
10GigE
Ports
2 ports
$2,029.00View partBuild a system with it

One side door is worth knowing: some frame grabbers also accept custom LVDS data when you match the signaling and emulate a camera's timing — our custom LVDS acquisition article shows how that works.

Documentation to collect before you commit

A raw-sensor design lives or dies by paperwork, so collect these before the schematic:

  • The datasheet timing section: clock range, setup and hold times, output levels.
  • The register map and the boot sequence for the streaming mode you need.
  • A timing application note with waveforms — most sensor makers publish one for the parallel interfaces, and it answers what the datasheet leaves open.
  • The evaluation board's reference design, which is a working receiver you can copy logic from.
  • For a camera module: the connector pinout and the CSI-2 lane mapping; our MIPI CSI camera connector pinout article explains what to read before matching a module.

Inside the FPGA, treat the pixel path as a pipeline from the first day. A line buffer in block RAM costs a clock of read latency, so a memory read sitting directly on the intake path stalls the whole stream unless the access is pipelined; the FPGA RAM read latency guide shows why the read must be pipelined and how to restructure it.

Checks before you build, and when not to build

Run this list before ordering parts:

  • Name the exact output the sensor or camera exposes. If the answer is GigE, 10GigE, USB3, CoaXPress or Camera Link, order a card from the frame grabber category above instead of designing a receiver.
  • Run the pixel-clock arithmetic with your real resolution, frame rate and blanking, and compare the result with the sensor's documented clock range.
  • Match voltages and termination: FPGA bank voltage to sensor output level, and proper termination on every LVDS pair.
  • Get the documentation list above confirmed, in writing, from the sensor maker.
  • Prototype the receiver on the evaluation board before you lay out your own PCB.

The usual answer is wrong in three cases. If images only need to reach a PC, a standard camera with its SDK is the maintained path and your FPGA adds risk for nothing. If the processing is the product but the acquisition is standard, put the FPGA where the processing is — behind a frame grabber, not instead of one. And if the receiver is actually a microcontroller rather than an FPGA, a parallel DVP interface usually wins on simplicity, as our camera interface for a microcontroller article argues.

If the decision lands on buying, the interface finder filters the grabbers by interface and port count, and the capture-side configurator walks the choice against your camera. If it lands on building, choose the sensor side and contact us with the datasheet in hand — more acquisition and interface guides sit on the systems topic page.

Questions

Can an FPGA read a GigE or USB3 camera directly?

Only by building the whole host side of the standard yourself: link layer, packet decoding and, for GigE, the full network stack, without the compliance machinery the standard assumes. A PoE NIC or frame grabber gives you the receiver, the drivers and the support path in one card. Build your own only when the acquisition and your processing must share a clock domain a standard card cannot enter.

How do I control exposure and gain without an SDK?

Through the sensor's register interface: I2C or SPI writes to the exposure, gain and ROI registers, driven by your FPGA at power-up and whenever a trigger changes the settings. The register map and boot sequence come from the sensor datasheet, and a misordered boot sequence is the most common reason a raw sensor never starts streaming.

How much data can CSI-2 lanes carry into an FPGA?

Multiply lanes by lane rate and divide by bits per frame. Four D-PHY lanes at 2 Gbps move 8 Gbps, or 1 GB/s. A 25 MP frame at 8-bit is about 25 MB before packet overhead, so the bus ceiling is roughly 40 fps, and blanking pushes the real number lower. Run the numbers for your own frame before fixing the lane count.

Does a direct sensor read give the same image as the SDK path?

No. You get the sensor's raw output: Bayer data straight off the pixels, with none of the black-level, white-balance or demosaic processing a camera's ISP or driver applies. For measurement that is often exactly what you want; for display or color work you must build or add that processing yourself.

When should I capture with a frame grabber instead of designing an FPGA receiver?

Whenever the camera speaks GigE, 10GigE, USB3, CoaXPress or Camera Link. The card supplies the PHY, the protocol engine, DMA into host memory and maintained drivers, and your schedule skips packet-layer debugging entirely. Design your own receiver only when you are reading a raw sensor output, or your processing must sit in the same clock domain as the pixels.