Recording video from a machine vision camera in Python

You can pull stills from an area scan camera all day, and the first continuous recording still goes wrong three ways: the frame rate lands far below the number on the data sheet, the file plays faster or slower than real time, and color footage comes out black and white. None of these are writer problems. Two camera settings decide them — AcquisitionFrameRate and PixelFormat — along with one habit: stamping the file with the fps you measured, not the fps you asked for. This guide walks the whole chain: pixel format, link bandwidth, the settings, a working script with an ffmpeg pipe, and the checks that prove the take.

The three settings that decide it

Every working pipeline comes down to three decisions, and none of them live in the video writer:

1. The frame rate comes from the camera. Set AcquisitionFrameRate before capturing anything, then read back ResultingFrameRate. That read-back is the number the rest of the chain must honor.
2. Color crosses the link as YCbCr 4:2:2. The YCbCr422_8 pixel format has the camera do the color conversion in hardware at 2 bytes per pixel, and H.264 encoders consume YUV natively, so no per-frame conversion runs on the host.
3. The file's fps is measured, not requested. Count the frames you actually wrote, divide by the seconds you actually recorded, and stamp the container with that value.

Match the pixel format to the encoder

H.264 encoders work in planar YUV: libx264 accepts yuv420p, yuv422p and yuv444p, plus grayscale, and nothing packed. A machine vision camera outputs packed or raw formats, so you need one of three bridges:

• YCbCr422_8 from the camera. GenICam's YCbCr422_8 packs as Y-Cb-Y-Cr, which is exactly ffmpeg's yuyv422 input format, so frame bytes reach the encoder untouched.
• BayerRG8 plus a host-side debayer. Half the wire bytes of YUV, at the cost of one cv2.cvtColor call per frame. Use it when the link or the fps target is tight.
• Mono8. No color, no conversion: declare gray at the input and the encoder takes the bytes as luma.

The bridge that fails is raw Bayer fed straight to the encoder. libx264 refuses a Bayer format and negotiates yuv444p in its place; the Bayer-to-color reconstruction never runs and the file plays back in black and white. The same negotiation greys out packed YUV choices in some capture tools, which is what the YUV 4:4:4 packed format being greyed out untangles. If your SDK will not switch formats at all, start from why a camera pixel format will not change and how to set it; a blank or colorless preview during set-up has the same root, covered in diagnosing an unsupported Bayer pixel format.

The decision: color takes go out as YCbCr422_8; fps-critical or link-limited takes go out as Bayer 8-bit; monochrome stays Mono8; RGB8 is a display format — triple the wire cost, zero encoder benefit.

The bandwidth arithmetic that sets your ceiling

Two link budgets cover most benches. A GigE Vision camera delivers 100-115 MB/s of image payload; a USB3 Vision camera delivers 350-380 MB/s, the 5 Gb/s line rate after 8b/10b encoding and protocol overhead. Plan on 100 MB/s for GigE and 350 MB/s for USB3, and treat the remainder as margin.

Work a 1920 × 1200 sensor at 25 fps, the classic 2.3 MP take:

• 1920 × 1200 = 2,304,000 pixels. Bayer or mono 8-bit: 2.30 MB per frame, 57.6 MB/s.
• YCbCr 4:2:2: 4.61 MB per frame, 115.2 MB/s.
• RGB8: 6.91 MB per frame, 172.8 MB/s.

On GigE, Bayer 8-bit records 25 fps with room to spare, YCbCr 4:2:2 at 25 fps sits exactly on the ceiling so budget 20 fps (92 MB/s) for margin, and RGB8 does not fit at any useful rate. On USB3, all three fit at this sensor size and the camera's internal processing becomes the limit.

Move to 12 MP and the format decides the take. A 12 MP USB3 camera such as the LS-CMR-0166 runs 30 fps at its full 4000 × 3000 resolution. Bayer 8-bit needs 12.0 MB per frame, so 30 fps needs 360 MB/s — inside the link with little margin. YCbCr 4:2:2 doubles the frame to 24.0 MB: 30 fps would need 720 MB/s, which no USB3 link carries, and the ceiling lands near 15 fps. RGB8 triples the frame to 36.0 MB and the ceiling falls near 10 fps. That is the mechanism behind every "rated fps unreachable in RGB8" report — the headline rate on a data sheet is measured in the leanest format the camera offers, usually Bayer or mono 8-bit.

For a GigE color build or a monochrome USB3 build, these two cover the common starting points:

Set the frame rate in the camera, not in the writer

AcquisitionFrameRate is a standard GenICam feature: enable it, set the value, then read ResultingFrameRate, where the camera recomputes the achievable rate against your pixel format and exposure. If the read-back says 14.9 fps when you asked for 25, the camera is naming the binding constraint — link bytes or exposure time.

Exposure is the quieter of the two. The frame period at 25 fps is 40 ms, and fps_max = 1 ÷ exposure_time: keep exposure at 30 ms or less for 25 fps, while a scene forcing 100 ms caps any camera at 10 fps regardless of settings. Auto exposure running during a take is the hidden version of the same trap, the drift behind RGB camera frame rate limits from pixel format and exposure. Confirm the real rate on your own machine with measuring camera frame rate in Python before you commit a number to the writer.

Record on a wall clock, then stamp the true fps

The tempting loop grabs fps × seconds frames and stops. It breaks the moment delivery runs below the set rate: a 24 fps request capped at 15 fps by the link collects about 150 frames in 10 s, and a container stamped 24 fps plays them in 6.2 s — fast motion, and a file shorter than the wall clock says.

Run the take on a timer instead: start with time.perf_counter(), grab until the deadline, count the frames, and compute measured fps = frames written ÷ seconds elapsed. With the ffmpeg pipe you declare the rate up front, so do a 10 s trial take, put the measured value into -r, and record the real take. A file already stamped wrong is fixed without re-encoding:

ffmpeg -r 15 -i take.mp4 -c copy take_true.mp4

For frame-level traceability on long takes, camera buffer and timestamp latency explains what the camera's own timestamps can and cannot tell you.

The recording script, step by step

You need a USB3 or GigE camera, Python with your SDK's bindings or a GenICam wrapper such as Harvester, an ffmpeg binary on PATH, and OpenCV for the Bayer recipe.

1. Open the camera, enumerate PixelFormat, and select YCbCr422_8 when it is listed — that is the color recipe below.
2. Set ExposureTime to at most three quarters of the frame period: 30 ms at 25 fps.
3. Enable and set AcquisitionFrameRate to 25, then confirm ResultingFrameRate matches.
4. Run a 10 s trial take and put the measured fps into the writer.
5. Start ffmpeg as a subprocess reading raw frames from standard input.
6. Grab and write raw bytes in a wall-clock loop — no color conversion anywhere on the host.
7. Close standard input, wait for the process, and print frames ÷ seconds.
8. Check the file: duration equals frames ÷ stamped fps, and a known color patch holds its hue.

import subprocess as sp
import time

W, H, FPS, SECONDS = 1920, 1200, 25, 30   # FPS = measured value after the trial

cmd = [
    'ffmpeg', '-y',
    '-f', 'rawvideo',
    '-pix_fmt', 'yuyv422',             # GenICam YCbCr422_8 packs as Y-Cb-Y-Cr
    '-s', f'{W}x{H}',
    '-r', str(FPS),
    '-i', '-',
    '-c:v', 'libx264', '-preset', 'fast', '-crf', '18',
    '-pix_fmt', 'yuv422p',             # yuv420p when every player must open it
    'take.mp4',
]
proc = sp.Popen(cmd, stdin=sp.PIPE)

# camera: PixelFormat = YCbCr422_8
#         ExposureTime <= 30 ms
#         AcquisitionFrameRate = 25, then check ResultingFrameRate

frames = 0
t_stop = time.perf_counter() + SECONDS
while time.perf_counter() < t_stop:
    buf = grab(timeout_ms=1000)         # your SDK's blocking frame call
    proc.stdin.write(buf.tobytes())
    frames += 1
proc.stdin.close()
proc.wait()
print(f'measured fps = {frames / SECONDS:.1f}')

If the payload is larger than W × H × 2, the buffer carries line padding: slice each row to the true width and join, or declare the padded width at the input.

When the link is tight, record Bayer 8-bit and debayer on the host — one OpenCV call per frame:

import cv2
bgr = cv2.cvtColor(bayer, cv2.COLOR_BayerRG2BGR)  # pattern must match the format name
proc.stdin.write(bgr.tobytes())                   # declare -pix_fmt bgr24 at the input

Swap BayerRG8 in at the camera and bgr24 in at the input; everything else holds. For monochrome, use Mono8 with -pix_fmt gray and write bytes as they arrive. PyAV does the same work in-process when you would rather not manage a subprocess: build the frame from the packed array as yuyv422, reformat to yuv422p, send it to the encoder. OpenCV's VideoWriter also works once frames are BGR and suits short clips, but the fourcc values a given build supports vary by machine, and a writer that never opened is the usual cause of an empty camera recording file.

Keep the light steady

Video inherits every lighting wobble a still grab would hide. A PWM-dimmed LED against a rolling shutter produces banding that crawls through the footage, and a strobed channel produces black frames whenever pulse and exposure miss each other. For continuous recording, drive the light from a channel held at constant output: a 60 W 4-channel digital constant-voltage controller such as the LS-CTL-0290 keeps one light steady while the other channels serve triggered stations. Banding that appears anyway is diagnosed in camera flicker under artificial lighting.

Checks before you call the take good

• Frame count. Frames land within ±1 of measured fps × seconds: 750 frames for 25 fps over 30 s.
• Duration and speed. The file reports frames ÷ stamped fps, and a clock in frame matches a stopwatch.
• Color. A known patch holds its hue, and a gray card shows no crawling bands during the take.
• Headroom. Keep CPU below about 70% on the grabbing core. H.264 at CRF 18 typically lands 10-30× below the raw stream for scene content, so a disk that streams 20 MB/s holds a 115 MB/s raw take with room — still, watch the first take's write rate.
• Cabling. USB 3.0 passive cable holds spec to 3 m; beyond that use an active or optical cable, or a GigE camera on up to 100 m of Cat6. On a moving head, a drag-chain-rated 3 m USB3.0 camera cable such as the LS-CBL-0219 is the one that survives.

When compressed video is the wrong deliverable

H.264 is 8-bit, chroma-subsampled and lossy. The moment pixel values feed a measurement — diameters, gray-level checks, presence tests — record FFV1 in an MKV container instead: lossless, the same raw pipe, every bit the camera sent preserved. Analysis stills deserve full depth, the way saving 12-bit camera images without losing intensity data lays out, and frames you only need in memory should never touch a writer — acquiring camera frames without saving files is that pattern.

Scale changes the tools, not the rules. Several cameras writing one disk is a queueing problem handled in multi-camera recording to disk without dropped frames; rates past a few hundred fps bring the capture tactics in high-speed camera recording without dropped frames. When the recorder lives in an operator panel and must start and stop with the machine, recording camera capture alongside PLC communication covers that side of the integration.

To size the whole chain in one pass, start from the camera finder with your link budget in hand, or run the system configurator from the camera step. Bring your format and frame rate to the contact page and we will check the arithmetic with you.