MIPI CSI capture latency measurement: exposure to buffer done

When someone asks for the capture latency of your MIPI CSI-2 camera, they want one number: the time from exposure start at the sensor to the moment the capture driver delivers a filled buffer to your code. This guide gives you a way to measure that on an embedded Linux host, using an LED, a spare GPIO and the monotonic clock the driver already stamps with. You will pin down which timestamp your driver writes, build a stimulus you can time, run a hundred flips, and check the reading against a stage-by-stage budget. The worked example at 30 fps, 1080p and 12-bit raw gives you numbers to hold your own rig against.

What you are measuring

Capture latency, as defined here, is one interval: exposure start to the driver's vb2_buffer_done call, the point where a completed buffer becomes yours to dequeue. Everything after that — copies, image processing, encoding, display — is a different measurement, and the glass-to-glass latency method covers the viewer-side half. Timestamping stages rather than screens is the discipline behind a depth-camera latency write-up: measure the pipeline, not the viewer.

Between exposure start and buffer done sit four stages: sensor readout, MIPI CSI-2 transfer, ISP, and the driver hand-off. On a rolling-shutter sensor the readout dominates, because rows convert one after another. The CSI transfer overlaps the readout whenever the link payload rate exceeds the sensor data rate — the normal case at 1080p. For the meaning of each field in the stamp itself, the v4l2 buffer timestamp walkthrough reads it field by field, and interpreting buffer timestamp latency covers what the interval tells you.

For a rolling sensor, estimate readout as 0.85 to 0.95 of the frame period; the remainder is blanking. If the driver stamps end of frame (EOF) instead of start of exposure (SOE), the exposure rides inside the number. Which stamp you have is the first thing to establish.

What you need

• An embedded host with the camera on MIPI CSI-2 and a V4L2 capture path you can edit.
• An LED with a series resistor on a spare GPIO, or any light your code can switch on the same board.
• Manual exposure and frame-rate control, with auto exposure off.
• A monotonic-clock read in the same process that handles buffers.
• Optional: an oscilloscope with a photodiode, or a second GPIO your application pulses when a buffer lands.
• Optional: a bench board-level camera or area scan camera to validate the harness on an interface you fully control.

Step 1: identify the timestamp source

V4L2 buffers carry a timestamp in the kernel's clock domain, and devices generally stamp with the system monotonic clock. A per-buffer flag names the source: start of exposure (SOE) or end of frame (EOF). The difference matters because EOF includes the exposure. At 30 fps with a 10 ms exposure, an EOF-based number carries 10 ms that is not pipeline delay, and the error scales with whatever exposure you run.

Confirm the flag empirically: lock the frame rate, step the exposure between 2 ms and 10 ms, and watch the stamps. EOF timestamps move by the full 8 ms; SOE timestamps stay put. Two minutes of work that prevents a wrong number.

The delivery point is easy to find in driver source: the capture engine hands each finished buffer to user space in its vb2_buffer_done call, with the stamp in the buffer's timestamp field. If the driver exposes no way to switch source, lock the exposure and carry the offset as a constant.

Step 2: build a stimulus you can timestamp

Put an LED in front of the lens and drive it from a GPIO on the board that captures. Write the GPIO from the same code that logs timestamps, reading the monotonic clock immediately around the write. A GPIO write costs microseconds; if the last 0.1 ms matters, put a scope on it once and subtract what you see.

Keep the LED on for at least one full frame period, so no exposure can fall into the gap. Run a short exposure — 1 to 2 ms — so the lit frames bracket the flip tightly.

A rolling shutter sees the flip row by row: at 30 fps the lit edge sweeps the frame for roughly the 30 ms readout. That skew is why you judge the flip from stamp arithmetic rather than from where it appears in the image. If the whole frame must see one instant, that is a strobe-alignment problem — synchronizing an LED strobe with frame capture on a rolling shutter camera shows the technique. For certainty about the light itself, measure the optical edge with the LED strobe rise-time method.

Step 3: run the measurement

1. Lock exposure and frame rate; auto exposure off.
2. Start the stream and discard the first 10 buffers while the pipeline fills.
3. For at least 100 cycles: flip the LED on and log the monotonic flip time; flip it off in the next cycle.
4. Log every buffer's timestamp and sequence number.
5. For each flip, take the first buffer whose timestamp is later than the flip time: latency = buffer timestamp − flip time.
6. Report minimum, median and maximum, never a single frame.

With SOE stamps, that first-later buffer is exactly the exposure that saw the LED, because exposure start follows the flip. With EOF stamps the logic still holds, since end of frame comes later still. If the GPIO write took a measurable time, subtract it once — it is the same for every sample.

A worked example at 30 fps, 1080p, 12-bit raw

Assume a rolling-shutter sensor streaming 1920 × 1080 at 30 fps, 12 bits per pixel, over four CSI-2 data lanes at 1.5 Gb/s each, with a 10 ms exposure.

• Frame period: 1000 ms ÷ 30 fps = 33.3 ms.
• Frame size: 1920 × 1080 × 12 bit = 24,883,200 bit = 3.11 MB.
• Link payload: 4 × 1.5 Gb/s = 6 Gb/s of line rate; allowing for packet overhead, plan on about 600 MB/s of payload.
• Link-limited transfer: 3.11 MB ÷ 600 MB/s = 5.2 ms, which hides inside the readout.
• Readout: 0.9 × 33.3 ms ≈ 30 ms.
• ISP: 3 ms in this example; expect 2–5 ms from a hardware ISP at this resolution, and measure yours against a raw path.
• Hand-off: under 1 ms for a consumer thread that is not starved.

Totals:

• Exposure start → buffer done: 30 + 3 + 1 = 34 ms, about 1.02 frame periods.
• With EOF stamps: 34 + 10 = 44 ms.
• Flip → buffer in free-run: mean 16.7 + 34 ≈ 51 ms; worst case 33.3 + 34 ≈ 67 ms.

When the measured median lands within about 10% of the budget, the instrumentation is sound. When it does not, the gap is a stage nobody instrumented — most often a copy or a compressor.

Cross-check on a camera you fully control

Before trusting the CSI number, run the identical LED test on a bench camera over an interface whose stack you control end to end. The floor you measure there — readout plus hand-off, no CSI receive stage — shows what the embedded receive path adds.

The LS-BCM-0003 is a 3.1 MP USB3 board camera with a global shutter and a C-mount.

At 120 fps the frame period is 8.3 ms, so the alignment term shrinks to at most 8.3 ms and 100 flips take seconds.

A second reference rules out a quirk of one sensor. The MV-CB016-10GM-M-W is a 1.6 MP mono board camera with a global shutter and an M12 mount.

Close the loop with the scope: channel 1 on the LED drive, channel 2 on a GPIO your application pulses when it dequeues a buffer. The interval on screen is flip to buffer, with no clock-domain argument left. It should agree with the timestamp arithmetic within about 1 ms; a wider gap means the userspace wake is slower than assumed.

Where the milliseconds go, and how to cut them

• A hardware trigger deletes the alignment term: exposure starts within microseconds of the edge, and then exposure plus readout time set the maximum frame rate.
• Manual exposure keeps the EOF term small and constant; if your pipeline writes exposure every frame, the one-frame delay that setting adds belongs in the budget too.
• A raw path removes the ISP stage; what a CSI camera with no ISP outputs covers the trade-off.
• Readout scales with the frame period: a higher frame rate or a shorter ROI — fewer rows — shortens it proportionally.
• Map the stages before you tune: the MIPI CSI-2 capture pipeline from sensor to host walks them, and sizing the data path across a CSI-to-USB bridge applies the same data-rate arithmetic to link sizing.

Common mistakes

• Comparing the stamp to a wall-clock log. Subtract in the monotonic domain only; NTP steps poison the difference.
• Assuming SOE when the driver stamps EOF. The error equals one exposure — 10 ms here — and it drifts with the lighting.
• Leaving auto exposure on: the spread you measure is then the AE loop, not the pipeline.
• Quoting a free-run median as the worst case. Alignment alone pushes the mean past 1.5 and the worst past 2 frame periods in the example; publish mean and worst separately.
• Judging the LED from one row of a rolling frame — the flip sweeps the readout, so that row misleads.
• Calling encode or display time "capture". The clock you are reading stops at vb2_buffer_done.

Before you quote the number

• State the definition with the figure: "SOE to buffer done, 30 fps, 10 ms exposure, 1080p 12-bit". A bare millisecond value is unreadable in six months.
• Confirm the timestamp source by flag and by the exposure-step test.
• Take at least 100 flips and report minimum, median and maximum.
• Keep one clock domain and cross-check against the scope within about 1 ms.
• Make budget and measurement agree within about 10%; if they do not, the missing stage is in the pipeline, not the sensor.

If the budget says the sensor, the trigger input or the link is what stands between you and the number, filter by shutter, interface and resolution in the camera finder, or configure a camera step by step. More timing and synchronization methods sit on the systems topic page, and a second opinion on a latency budget is one contact page away.