Stream a MIPI camera over RTSP with GStreamer: fix preroll failures

Your MIPI sensor previews and records perfectly with gst-launch-1.0, four cameras at once. Wrap the same elements in test-launch from gst-rtsp-server and nothing reaches the client: the log ends in "failed to preroll pipeline" and VLC reports no media. This guide is for engineers streaming an embedded MIPI camera head to network viewers. You will read the log in the right order, see why a live source cannot preroll where a test pattern can, and rebuild the factory line so VLC gets its stream.

The symptom: read the log in this order

The server itself is fine. test-launch prints "stream ready at rtsp://127.0.0.1:8554/test", so the process, the /test mount and port 8554 are all up. The failure starts when the first viewer connects: VLC's DESCRIBE request makes the factory build and prepare the pipeline, and that prepare is what dies.

The failing sequence keeps one order. Argus applies your exposure and ISP gain settings, opens the output stream, then an Error BadParameter propagates out of FrameConsumerImpl.cpp, gstnvarguscamerasrc logs "Failed to create FrameConsumer", and the source thread dies. About 20 seconds later — the log's own timestamps move from 0:00:28 to 0:00:48 — rtsp-media gives up: WARN rtspmedia rtsp-media.c:3272:wait_preroll: failed to preroll pipeline, followed by gst_rtsp_media_prepare: failed to preroll pipeline.

Everything after that is consequence, not cause. rtsp-client logs "can't prepare media", the DESCRIBE handler answers "no media", and the next SETUP says media '/test' not found, because the mount is empty once prepare fails. Read those lines as one failure, not four.

Noise you can ignore

Three lines look alarming and mean nothing here: the repeated "Failed to determine interlace mode", the "Failed to probe pixel aspect ratio with VIDIOC_CROPCAP" line, and the FIXME about a random stream-id. "NvMMLiteOpen : Block : BlockType = 4" is the hardware encoder block coming up, which is good news, and "Opening in BLOCKING MODE" is normal capture startup. Line numbers such as 3272 move between gst-rtsp-server releases; this log is 1.16.3. Search on the function names instead: wait_preroll and gst_rtsp_media_prepare.

Why a live MIPI source cannot preroll in test-launch

test-launch serves one shared media on the /test mount: every viewer attaches to the same pipeline, and the encoder runs once no matter how many clients connect. A session starts with DESCRIBE. On the first DESCRIBE the factory builds the whole chain and brings it to PAUSED, then waits for preroll: one buffer must travel from the source through the converter, the encoder and the payloader before any viewer has issued PLAY.

videotestsrc is not a live source. It pushes test frames as fast as the pipeline accepts them, so the first buffer arrives immediately and prepare succeeds. That is why the videotestsrc pattern always works in this exact test, whatever else is broken.

nvarguscamerasrc is a live source. Its frames arrive on the clock once the chain actually runs against a consumer, and your log shows that consumer never gets created. No frame crosses the payloader, wait_preroll runs out at the prepare timeout (the 20 seconds you saw above), and teardown destroys the half-built Argus stream — which is the BadParameter you meet again on every retry. gst-launch-1.0 -e works because fpsdisplaysink and filesink consume from the very first frame; it never has a prepare-before-client phase.

Likely causes, in order of probability

Work the table top down; each test takes minutes.

Causes one to four share one gate: get videotestsrc through the same factory first. If the pattern plays, the server, payloader, network and viewer are clean, and the camera half is all that remains.

Step 1: prove the server path with videotestsrc

1. Serve a test pattern through the same encoder and payloader names the factory expects:

GST_DEBUG=3 test-launch "videotestsrc ! x264enc ! h264parse ! rtph264pay name=pay0 pt=96"

x264enc is the standard software encoder and stands in for nvv4l2h264enc. The element named pay0 with payload type 96 is the stream the client mounts.

2. Open rtsp://10.0.0.15:8554/test in VLC on another machine — substitute your board's IP; localhost only works on the board itself.

3. Success is the moving test pattern. Server, payloader, network and viewer are now proven, so everything left is the camera half.

If even the test pattern fails, stop here: plugins or transport, not your camera. Start at the missing-plugins checklist and the RTSP UDP media in containers guide.

Step 2: the camera pipeline that prerolls

Hand the whole chain to test-launch as one quoted argument; the breaks below are only for reading:

nvarguscamerasrc sensor-id=0 sensor_mode=0 !
video/x-raw(memory:NVMM), width=1936, height=1220, framerate=(fraction)50/1, format=NV12 !
nvvidconv !
video/x-raw(memory:NVMM), width=1936, height=1220, format=NV12 !
queue !
nvv4l2h264enc !
h264parse !
rtph264pay name=pay0 pt=96

• The source caps pin sensor_mode=0 at 1936 × 1220 and 50 fps in NV12 — exactly the caps your working gst-launch line validated, so the sensor starts in the mode you know.
• nvvidconv keeps every buffer in NVMM, so the raw stream never crosses into system memory.
• The second caps filter is the contract: one fixed NV12 format at one size, so the encoder negotiates exactly once during prepare. Loose caps between converter and encoder are a classic prepare-time renegotiation trap.
• The queue before nvv4l2h264enc decouples the encoder's buffer pool from the capture side; the copy-threshold warning in your log is the sign that coupling is tight.
• h264parse hands rtph264pay a clean byte stream, and pay0 is the mount point.

Because the factory media is shared, every later viewer attaches to this one pipeline: encode once, serve many.

Preview and recording branches: size each one separately

Your proven local script runs a tee after the source caps: one branch through nvvidconv flip-method=0 into fpsdisplaysink for the display, one through nvv4l2h264enc into qtmux and filesink for the MP4, each with its own queue. Keep that shape inside the factory when you want a preview too: tee after the converter, then a queue and its own small caps — 480 × 300, as in your script — on the display branch, while the encode branch stays at the shipping size.

Know the behavior change: the factory pipeline lives from the first DESCRIBE onward, and with no viewer connected no media runs, so a local preview inside the factory goes dark between clients. Keep the gst-launch script for pure local work and put the factory line on network duty.

To record the MP4 and serve RTSP at the same time, encode once and tee after h264parse: one branch through a queue into qtmux ! filesink, the other through a queue into rtph264pay name=pay0. One encoder serves both consumers, and the file grows only while the media runs.

One detail from the failing test: it encoded the RTSP branch at 480 × 300. That is fine as a plumbing check, but pin the size you actually ship at the converter output instead of leaving the preview size on the encode path.

Sizing the pipeline for a 1936 × 1220 sensor at 50 fps

The inputs come straight from your working caps: 1936 pixels by 1220 pixels per frame at 50 frames per second in NV12. NV12 stores 12 bits per pixel — 8-bit luma plus quarter-size chroma — which is 1.5 bytes per pixel. Four sensors run at once, as in the four-camera script. The share bitrate is a chosen value for this example: 8 Mbit/s per camera, a mid-quality setting you can raise for detail work.

One camera: 1936 × 1220 = 2,361,920 pixels per frame. At 50 fps that is 118,096,000 pixels per second, and at 1.5 bytes per pixel it is 177,144,000 bytes per second — 177 MB/s, or 1.42 Gbit/s. Four cameras push 4 × 177 MB/s = 709 MB/s (5.67 Gbit/s) of raw NV12 through the memory system. The preview branch is far cheaper: 480 × 300 × 50 × 1.5 = 10.8 MB/s, about 86 Mbit/s. The encoded share leaves the board at 4 × 8 Mbit/s = 32 Mbit/s, roughly a third of a 100 Mbit/s link and easy work for 1 Gbit/s. Pacing check: 50 fps means one frame every 20 ms, so a queue ten buffers deep buys about 200 ms of slack.

The decision follows from those numbers. Keep every raw branch in NVMM so the 709 MB/s of raw data never leaves the SoC, and only the 32 Mbit/s of encoded H.264 crosses the network interface. The threshold: one raw NV12 stream at this size runs 1.42 Gbit/s, which already exceeds a 1 Gbit/s Ethernet link, so a raw branch never goes to the network — encode first, always.

Checks before you blame the camera

Run this list before you conclude the sensor is the problem:

• videotestsrc through the same factory — it splits camera trouble from server trouble in one minute.
• Recording sessions stopped, one process per sensor.
• A queue after every tee branch, and one directly before the encoder.
• One pinned NV12 caps filter after nvvidconv, at the size you ship.
• pay0 present with pt=96, since the factory serves the element you named pay0.
• The URL from another machine — rtsp://board-ip:8554/test, not localhost.
• Only three log lines matter: the Argus BadParameter, wait_preroll, and gst_rtsp_media_prepare. The rest is noise.

When it is not preroll

• The sensor never opens, even in gst-launch-1.0 -e with fakesink: capture bring-up, not streaming. Power, clocks and device tree come first in the MIPI streaming bring-up guide, and a sensor that probes but misconfigures its lanes has its own 2-lane versus 4-lane guide.
• Even videotestsrc fails to reach VLC: plugins or transport, covered by the two guides in Step 1.
• The stream plays in VLC but not in your SCADA or HMI viewer: check what the viewer itself supports — this Ignition RTSP viewer limitation note is a typical case.
• On Orin-class boards one more startup failure shows up as a libgomp TLS error in the server process; the Jetson Orin Nano fix note walks through it.
• The pipeline refuses to start with caps errors before any client connects: that is negotiation, and the format negotiation guide covers it.

If the capture head is USB3, not MIPI

The factory pattern does not care where frames come from. A USB3 capture head on the same host streams through v4l2src or the vendor's source element with the identical discipline: converter, one pinned caps filter, a queue before the encoder, h264parse, then rtph264pay name=pay0. The videotestsrc proof and the sizing arithmetic apply unchanged; only the source element name changes.

When the SerDes and driver work is the expensive part of the project, a USB3 head removes it entirely. The LS-BCM-0003 runs 120 fps at 3.1 MP with a global shutter, the MV-CB060-10UM-B covers 6 MP mono on a bare board, and the LS-BCM-0047 handles 1.3 MP color duty, all over USB3.

The board-level cameras category lists the rest, the camera finder filters by resolution, shutter and interface, and configuring a camera walks the pick step by step. More bench-tested fixes from this area sit on the systems troubleshooting topic page. Run the videotestsrc proof today, rebuild the factory line with the pinned NV12 caps and the queues, and the VLC URL that failed this morning carries your camera by the afternoon.