GStreamer pipeline strings for on-board camera capture explained

You copied a capture pipeline string that works on one board, moved it to another, and got a missing-plugin error or a not-negotiated error. The string was never magic: it is five GStreamer elements in a row, each with a typed contract covering what it accepts and what it hands on. This piece breaks the string down element by element, source, caps filter, hardware converter, software conversion to BGR, appsink, so you can rebuild it for any on-board camera instead of hunting for a board-specific copy-paste. It covers what stays the same between a Jetson Nano, a TX2 and a plain Linux SoC, what the (int), (string) and (fraction) fields mean, why OpenCV needs BGR and not BGRx, and it ends with a worked example that sizes the caps from the inspection target rather than the other way round.

The whole string, read as five contracts

Here is the string most Jetson guides hand out, written in the shape that generalizes to any board camera:

nvarguscamerasrc sensor_id=0 sensor_mode=0 !
video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,
format=(string)NV12, framerate=(fraction)30/1 !
nvvidconv flip-method=0 !
video/x-raw, width=(int)1280, height=(int)720, format=(string)BGRx !
videoconvert !
video/x-raw, format=(string)BGR !
appsink sync=false max-buffers=1 drop=true

The exclamation marks are the plumbing: each connects one element's source pad to the next element's sink pad. Read the row as five contracts. The source owns the camera. The first caps filter fixes what leaves it. The converter pair changes resolution and colour space. The final caps fix what reaches your code, and appsink hands buffers to your application. Every capture string for an on-board camera is this shape; only the names change between boards.

The source element owns the camera

On the Jetson family, Nano, TX2, Xavier, MIPI CSI modules are driven through the Argus camera stack, and nvarguscamerasrc is its GStreamer front door. That answers the porting question directly: a string written for a Nano runs on a TX2 with the same structure, because both boards use the same stack. What moves between boards and between modules are two properties. sensor_id selects which connector the module sits on, and sensor_mode picks one entry from the module's own resolution and frame-rate table, so the mode number that gave 1280 x 720 on one camera can mean something else on the next.

On any other Linux SoC, and for UVC webcams everywhere, the front door is v4l2src with the node the driver assigned: v4l2src device=/dev/video0. A MIPI module can expose several nodes with only one of them capturing, so if the pipeline prerolls to a black frame, walk the node numbers upward until the right one answers.

Industrial buses are a different path. A GigE Vision or USB3 Vision camera is not a video device node: frames arrive through the vendor SDK or an aravis-based source element. CoaXPress and Camera Link cameras come through their frame grabbers and the grabber's SDK. The caps logic below still applies, width, height, format and frame rate still get fixed, but a capture string is not the delivery route.

The caps filter is a contract, not a converter

A caps filter converts nothing; it narrows what is allowed to flow past it. After a camera source it has one job: pin the single mode you want from everything the source can produce. The typed fields say how each value is parsed:

video/x-raw(memory:NVMM), width=(int)2048, height=(int)1536,
format=(string)NV12, framerate=(fraction)30/1

(int) fields are integers, (string) fields are names, and (fraction) is a ratio: 30/1 is thirty frames per one second, written as a ratio because rates like 30000/1001 are real camera modes. The (memory:NVMM) feature says the buffers live in the Jetson's multimedia memory, which only that family's hardware elements can read, and that is why nvvidconv, not videoconvert, follows the source. On a board without the stack, the same caps drop the feature and the format becomes whatever the driver actually offers, such as YUY2 from a webcam.

Ask for anything the source cannot produce and the pipeline stops in caps negotiation, the not-negotiated error, before a frame moves. The fix is never inside the filter; it is matching the filter to the mode list the source offers. The camera pipeline format negotiation walkthrough traces these failures through start-up, and the USB camera format negotiation guide covers the same failure for USB cameras.

One conversion to BGR, done once

The middle of the string is where resolution and colour space change, and the rule is to pay for each change exactly once. nvvidconv is the Jetson's hardware converter: it scales, crops, rotates and flips, with flip-method=0 passing the image through untouched and the other values rotating or mirroring, all while the buffers stay in multimedia memory. videoconvert is the CPU stage that lands on the format your code wants.

OpenCV's GStreamer path wants BGR, the 8-bit three-channel layout, and it rejects BGRx, the 4-byte-per-pixel variant, at appsink. That is why the tail of the string is videoconvert followed by video/x-raw, format=(string)BGR: the 32-bit hardware-friendly layout becomes the 24-bit layout the application side expects. If you resize, resize in one stage only, on the nvvidconv output caps or on the final caps, never both.

appsink is the door into your code

appsink ends every capture string because it hands buffers to application memory instead of to a screen or a file. Three properties earn their place. sync=false stops the sink pacing against a clock your loop does not follow. max-buffers=1 with drop=true means that when processing falls behind the camera, the sink keeps the freshest frame and discards the backlog instead of queueing stale ones. The burst capture versus link bandwidth guide treats the same rate mismatch one level up, when bursts outrun the link itself.

Building the string in Python keeps the numbers in one place. The parameter names below are ours; the element and property names are GStreamer's:

import cv2

def capture_pipeline(capture_width: int = 1280,
                     capture_height: int = 720,
                     framerate: int = 30,
                     flip_method: int = 0,
                     sensor_id: int = 0) -> str:
    return (
        f'nvarguscamerasrc sensor_id={sensor_id} sensor_mode=0 ! '
        f'video/x-raw(memory:NVMM), width=(int){capture_width}, '
        f'height=(int){capture_height}, format=(string)NV12, '
        f'framerate=(fraction){framerate}/1 ! '
        f'nvvidconv flip-method={flip_method} ! '
        f'video/x-raw, width=(int){capture_width}, '
        f'height=(int){capture_height}, format=(string)BGRx ! '
        f'videoconvert ! '
        f'video/x-raw, format=(string)BGR ! '
        f'appsink sync=false max-buffers=1 drop=true'
    )

pipe = capture_pipeline()            # the string with stock values
assert pipe.count('!') == 6          # seven elements, six links
assert pipe.endswith('drop=true')    # appsink is the last element
cap = cv2.VideoCapture(pipe, cv2.CAP_GSTREAMER)

Read frames from cap in a loop; the hardware trigger grab loop how-to is that loop with a trigger line driving it.

Worked example: size the caps from the target

Caps deserve better than 1280 x 720 out of habit. Work backwards from the smallest thing you must resolve. Take a face-recognition station: a 2.0 m standoff, a face about 150 mm wide, and a working budget of 200 pixels across the face. Detection holds up nearer 80 pixels; recognition models want two to three times that, so budget 200.

Run it with a 1/1.8-inch-class 3.1 MP sensor, meaning 2048 x 1536 photosites at 3.45 um pitch and 7.07 mm of sensor width:

• sampling = 200 px / 0.15 m = 1333 px per metre
• FOV = 2048 / 1333 = 1.54 m, the widest view that still puts 200 px on the face
• m = 7.07 / 1540 = 0.0046
• f = 2000 x 0.0046 / 1.0046 = 9.2 mm, so the nearest standard C-mount focal length is 12 mm
• at 12 mm: m is about 12 / 2000 = 0.0060, FOV = 7.07 / 0.0060 = 1.17 m wide and 0.88 m tall, and the face covers 150 x (2048 / 1172) = 260 px

Motion sets the exposure, not the frame rate. One pixel spans 1172 / 2048 = 0.57 mm of scene, and a person walking at 1.0 m/s crosses about 1.7 px per millisecond, so hold exposure at or under 2 ms and blur stays under 3.5 px. The 30 fps frame window is 33 ms, so light is the constraint, not timing; at f/2.8 the depth of field runs about 1.6 m to 2.7 m around the 2.0 m focus plane, more than a walk-up station needs. The lighting finder is where that 2 ms exposure budget gets spent.

The caps that follow are the sensor's own mode:

video/x-raw(memory:NVMM), width=(int)2048, height=(int)1536,
format=(string)NV12, framerate=(fraction)30/1

A shop camera in exactly this sensor class, with a C-mount for standard board lenses:

A 12 mm C-mount lens is the pick from the arithmetic above. Change the standoff or the pixel budget and the focal length moves with it:

One transport note: this example sizes optics and caps, and the same four settings travel with the camera whatever the bus. On a USB3 Vision or GigE Vision module they are set in the camera's feature tree through its SDK instead of a caps filter.

Adapting the shape to your board

For a complete design built this way on low-power hardware, the embedded DVR GStreamer pipeline reference design strings capture, conversion and file writing together on a BeagleBoard.

If the module itself is still open, start from the board-level camera selection guide for embedded machines and the board camera evaluation checklist; board-level cameras and area scan cameras cover the shop range. Two modules at the extremes:

When the copied string fails

Failures arrive in a predictable order, and each has a one-line test:

1. Missing plugin. nvarguscamerasrc and nvvidconv used off the Jetson family do not exist on the host, and the pipeline refuses to build. The missing-plugin walkthrough covers the case where the viewer shows nothing at all.
2. Not negotiated. The caps ask for a mode the source does not offer. Run the string under gst-launch-1.0 -v and the terminal prints the caps every pad settled on, so the mismatch is visible before your code is involved.
3. Wrong node or wrong sensor_id. The pipeline prerolls and hands you black frames or the wrong camera; step sensor_id or the node number until the right module answers.
4. BGRx at the door. OpenCV rejects it; keep the final videoconvert and the format=(string)BGR caps.
5. Two scalings. Sizing on both the nvvidconv caps and the final caps wastes the hardware stage and softens the image; scale once.

Checks before you commit the string

• gst-launch-1.0 -v runs the string end to end and prints every negotiated caps pair before any Python exists.
• Width, height and frame rate in the first caps match a mode the module actually outputs.
• The final caps read video/x-raw, format=(string)BGR when OpenCV reads the sink.
• appsink carries sync=false max-buffers=1 drop=true whenever processing runs slower than capture.
• Exactly one stage changes resolution.
• MIPI ribbons are seated with the board powered down.

From here, the camera finder filters board-level and area scan modules by sensor and mount, the system configurator walks camera, lens and light in one pass, and the contact page takes questions about a specific module.