System designHow-to

From filesrc to nvarguscamerasrc: pipe a CSI camera in GStreamer

Swap filesrc for nvarguscamerasrc and set sensor-id: pipe a CSI camera into an existing GStreamer app, with NVMM caps, a converter and gst-launch tests.

LumaSpec Vision8 min read Share on LinkedIn

On this page 9 sections
  1. 01Why only the source element changes
  2. 02What you need
  3. 03Swap the source in C, step by step
  4. 04Worked example: 1920 × 1080 NV12 at 30 fps
  5. 05Prove frames flow before you touch the app
  6. 06Four mistakes that cost an afternoon
  7. 07When v4l2src is the better source
  8. 08If CSI keeps costing more than it returns
  9. 09Questions

Your capture code already works: it builds a filesrc, sets location to a video file and lets the rest of the chain decode. On a Jetson Nano with a MIPI CSI camera the file disappears, but almost nothing else changes. The camera's source element is nvarguscamerasrc, the path property becomes sensor-id, and the caps after the source must carry the NVMM memory feature. This guide walks the element swap in C, the caps and converter that follow it, a bandwidth worked example for 1920 × 1080 at 30 fps, and the tests that prove frames are flowing before you touch your application logic.

Why only the source element changes

A filesrc reads bytes from a path and pushes them downstream; a CSI sensor is not a file. On Jetson, sensor frames arrive over the CSI-2 lanes into the Argus stack: the nvargus-daemon owns the sensor, runs auto-exposure, auto-white-balance and color correction through the ISP, and hands out finished frames in NVMM memory. That is why the source is a dedicated element instead of an open on /dev/video0, and why its output caps carry the (memory:NVMM) feature.

Reference sample apps and their source-bin code wrap this same element, so the swap below fixes your own pipeline, not just a sample config. The same sensor is usually also reachable as a raw V4L2 device; when V4L2 lists the sensor but nvarguscamerasrc finds no camera explains how the two enumeration paths differ, and the MIPI CSI-2 capture pipeline from sensor to host covers the link layer underneath.

What you need

  • A Jetson board (a Nano in this guide) whose sensor is already detected at the driver level; if it is not, start with the IMX477 CSI bring-up checks on Jetson Orin Nano or the broader acquisition troubleshooting path.
  • GStreamer 1.x with the Jetson plug-in set, and nvargus-daemon running.
  • The two properties that change: location on filesrc (a string path) and sensor-id on nvarguscamerasrc (an integer, default 0).
  • Any working sink, even a test one.

Swap the source in C, step by step

  1. Make the new source. Replace the filesrc factory call and drop location:
/* before — file input */
GstElement *source = gst_element_factory_make ("filesrc", "file-source");
g_object_set (G_OBJECT (source), "location", input_file, NULL);

/* after — CSI camera input */
GstElement *source = gst_element_factory_make ("nvarguscamerasrc", "csi-source");
gint sensor_id = 0;                  /* second camera = 1 */
g_object_set (G_OBJECT (source), "sensor-id", sensor_id, NULL);
  1. Cap the source with NVMM caps. The element will not link into plain video/x-raw; request the NVMM feature and the mode you want (30/1 is this example's framerate — keep the mode your driver defines):
GstElement *capsflt = gst_element_factory_make ("capsfilter", "csi-caps");
GstCaps *nvmm = gst_caps_from_string (
    "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1");
g_object_set (G_OBJECT (capsflt), "caps", nvmm, NULL);
gst_caps_unref (nvmm);
  1. Add the converter. A CPU sink needs system memory, so link a converter that accepts NVMM caps and outputs plain video/x-raw. Every Jetson GStreamer image ships one: run gst-inspect-1.0 on the board and use the NVMM-capable video converter it lists.

  2. Pin the output format. A second capsfilter after the converter keeps the sink simple: format=NV12 for encoding paths, BGRx when OpenCV is next.

  3. Prove the graph in one line first. Before touching C, run it with gst-launch-1.0 and a fakesink; the on-board GStreamer pipeline strings guide covers launch-line syntax:

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1' ! CONVERTER ! 'video/x-raw,format=NV12' ! fakesink silent=false
  1. Keep the rest. Bus watch, error handling and state changes are identical; the demuxer and decoder your file path used simply drop out.

Worked example: 1920 × 1080 NV12 at 30 fps

Inputs: 1,920 and 1,080 pixels come from the caps above; 1.5 bytes per pixel is the NV12 format (12 bits per pixel: Y plus subsampled UV); 30 fps is the framerate those caps request. The rule in words: bytes per frame = width × height × bytes per pixel, and throughput = bytes per frame × frame rate.

bytes/frame = 1920 × 1080 × 1.5 throughput (B/s) = bytes/frame × 30

Arithmetic: 1920 × 1080 = 2,073,600 pixels. 2,073,600 × 1.5 = 3,110,400 bytes, so every buffer is about 3.11 MB. 3,110,400 × 30 = 93,312,000 bytes/s ≈ 93 MB/s, or 746 Mbit/s of sustained payload.

Decision: a link that carries this stream needs about 93 MB/s. A USB 2.0 port sustains roughly 35-40 MB/s, so this mode is out of reach for any CSI-to-USB fallback; USB 3, with hundreds of MB/s, carries it easily. Even 1280 × 720 at 30 fps works out to 41 MB/s (1280 × 720 = 921,600 pixels; × 1.5 = 1,382,400 bytes; × 30 = 41,472,000 bytes/s), still above USB 2.0's comfortable range. Inside the Jetson pipeline the fix is free: keep frames in NVMM up to the single converter, convert once, and size an appsink queue on 3.11 MB per buffer — 10 buffers hold about 31 MB.

Prove frames flow before you touch the app

Run the launch line. fakesink with silent=false prints one line per buffer it receives, so continuous handoff lines with no error on the bus mean sensor, daemon, caps and converter all agree. The failures you will actually see:

Then swap fakesink for an appsink. The appsink-to-OpenCV pull without blocking shows the non-blocking callback, and if OpenCV crashes on the first frame it is nearly always caps again — the caps fix for OpenCV segmentation faults on a CSI camera. Two deeper failures sit below GStreamer: a stream that starts and later dies points at the CSI link or power, as in a CSI camera that times out after working for a while, and a driver that probes while no data reaches the camera input hardware is covered in why an Orin Nano CSI camera fails to deliver data to the VI.

Four mistakes that cost an afternoon

  1. Trusting the connector label for sensor-id. The daemon numbers sensors in its own enumeration order; a sensor wired for one marked connector does not always land on the id you guess. Probe each id with a one-line launch and record the order for your board.
  2. Plain caps after the source. video/x-raw without the (memory:NVMM) feature never links; the capsfilter sits immediately after nvarguscamerasrc.
  3. Asking for an undefined mode. Sensors expose fixed modes. 1920 × 1080 at 30 fps links only when the driver defines it — check with v4l2-ctl before blaming the daemon.
  4. Fixing the sample instead of your pipeline. Sample configs bake in extra settings; steps 1 and 2 above are the whole change your own code needs.

When v4l2src is the better source

Default to nvarguscamerasrc: processed 8-bit output, exposure and white balance handled by the daemon, frames already in NVMM for zero-copy into encoders. Move to v4l2src when you need raw Bayer, sensor-driven exposure timing set through the sensor registers (the register route when no API exists), or bit-exact detail — and mind how packed depths arrive (12-bit values inside 16-bit words). The price is owning the ISP work the daemon was doing for you.

If CSI keeps costing more than it returns

When the goal is a production inspection cell rather than one embedded board, skip CSI entirely: machine-vision cameras speak GigE or USB3 and come with their own capture APIs, so the whole element dance disappears. Browse the area scan cameras with the camera finder, or start a build in the camera configurator. For compact builds where you would have glued a CSI module in anyway, board-level cameras connect over the same standard buses:

Board-level cameras3.1 MP USB3 board-level camera, IMX252, 1/1.8" global shutter, 120 fps (LS-BCM-0003)LS-BCM-0003
Sensor format
1/1.8"
Lens mount
C-mount
$690.00View part
Board-level camerasHikrobot MV-CB016-10GM-M-W 1.6 MP GigE board-level camera, IMX296, Mono, M12 lens mount, WTB GigE portMV-CB016-10GM-M-W
Sensor format
1/2.9"
Lens mount
M12
$559.00View part
Board-level cameras6.3 MP board-level camera, IMX178, 1/1.8" rolling shutter, 18 fps (LS-BCM-0020)LS-BCM-0020
Sensor format
1/1.8"
Lens mount
CS-mount
$290.00View part

The LS-BCM-0003 lists 120 fps at 3.1 MP over USB3; MV-CB016-10GM-M-W and LS-BCM-0020 are the GigE options, the latter a CS-mount 6.3 MP part at 18 fps. On a board with a free USB 3 port, a plain UVC camera streams with no config file at all — the zero-config UVC path. More acquisition fixes live on the systems topic page; when you would rather have a quoted, supported capture subsystem than one more afternoon of bring-up, tell us the part and resolution on the /contact/ page and we will spec the camera, lens and light around it.

Questions

What does the sensor-id property select in nvarguscamerasrc?

An integer (readable and writable) that picks which CSI sensor the Argus daemon binds to, counted from 0; omit it and the source uses sensor 0. The numbering follows the daemon's enumeration order, not the connector label on the carrier board, and it is not the V4L2 /dev/video index. With two cameras, probe sensor-id=0 and sensor-id=1 with a one-line gst-launch before wiring both into one pipeline.

Can nvarguscamerasrc deliver raw Bayer data?

No. The element sits behind the ISP, so frames arrive processed, in NVMM memory, in 8-bit formats such as NV12 or BGRx, with auto-exposure and white balance already applied. For raw sensor output, open the V4L2 device with v4l2src instead and set exposure through the sensor registers; packed depths are the norm, so 12-bit values commonly arrive inside 16-bit words.

Does nvarguscamerasrc set exposure and white balance automatically?

Yes. The nvargus-daemon runs auto-exposure, auto-white-balance and color correction on every frame, which is why default output looks balanced with no settings. For manual timing, exposure-time properties take nanosecond ranges; for full control down to the sensor, use v4l2src and write the registers directly.

How much buffer memory does an appsink need at 1080p30?

Each 1920 × 1080 NV12 frame is about 3.11 MB (2,073,600 pixels × 1.5 bytes), and at 30 fps the stream runs at 93 MB/s. A queue of 10 buffers therefore holds about 31 MB; depth 4-10 suits most display and copy paths. Each queued buffer also adds about 33 ms of latency at 30 fps, so keep the queue short when your callback is fast.