System designTroubleshooting

YUYV images show two channels in OpenCV: convert before cvtColor

Your appsink hands OpenCV a YUYV image with two channels and cvtColor refuses it. The COLOR_YUV2BGR_YUYV fix, the BGR caps fix, and how to keep 30 fps.

LumaSpec Vision8 min read Share on LinkedIn

On this page 8 sections
  1. 01Why YUYV arrives as a two-channel image
  2. 02Fix 1: convert the frame with COLOR_YUV2BGR_YUYV
  3. 03Fix 2: ask for BGR in the pipeline caps
  4. 04Which fix belongs where
  5. 05Tests that pin the cause in one pass
  6. 06The arithmetic: where the conversion cost lands
  7. 07Prevent it: make the channel count a contract
  8. 08Questions

Your GStreamer pipeline streams at the frame rate you asked for, and then the first colour conversion fails with Invalid number of channels in input image, where scn is 2. Nothing is broken and no frames are lost: the camera really is delivering YUYV, and packed YUV reaches OpenCV as a two-channel array rather than a three-channel colour frame. This guide shows why the array has two channels, gives the one-line cvtColor fix for appsink pipelines, shows the caps change that delivers BGR straight out of camera.read(), and works out where the conversion cost lands so you keep the 30 fps your gst-launch-1.0 test reached.

Why YUYV arrives as a two-channel image

YUYV is a packed 4:2:2 format: two pixels travel in four bytes, ordered Y0, U, Y1, V, and the single U and V are shared by both pixels. That averages 2 bytes per pixel, a third less than BGR's 3, which is why CSI and USB camera drivers offer it as their default output. When appsink hands the buffer to OpenCV, those bytes become an 8-bit, two-channel array of shape (720, 1280, 2): channel 0 is the luma sample of that pixel, and channel 1 alternates U and V between neighbouring pixels along the row. The width stays 1280 because every array pixel carries one luma byte; the chroma is not a missing plane, it is interleaved into the second channel.

Every cvtColor code that expects an RGB, BGR or gray input demands 3 or 4 channels and refuses the call before doing any maths. The VScn::contains(scn) check in the error is that gate, and scn is 2. The frame itself is intact; only the conversion code was wrong for the layout.

bytes_per_frame(YUYV) = width × height × 2 bytes_per_frame(BGR) = width × height × 3

Fix 1: convert the frame with COLOR_YUV2BGR_YUYV

cv2.COLOR_YUV2BGR_YUYV is the documented conversion for exactly this layout. It unpacks each pixel pair, colours both pixels from the shared chroma, applies the YUV-to-RGB matrix and returns a 3-channel BGR image:

import cv2

pipeline = (
    "imxv4l2src device=/dev/video1 ! "
    "video/x-raw,format=YUYV,width=1280,height=720,framerate=30/1 ! "
    "appsink"
)
camera = cv2.VideoCapture(pipeline, cv2.CAP_GSTREAMER)  # True when the sink opens
ok, frame = camera.read()   # frame: uint8 ndarray, shape (720, 1280, 2)
bgr = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_YUYV)  # uint8 ndarray, shape (720, 1280, 3)

print(frame.shape, frame.dtype)  # (720, 1280, 2) uint8 - the proof of cause
print(bgr.shape, bgr.dtype)      # (720, 1280, 3) uint8 - the proof of fix

cv2.imshow, cv2.imwrite and your detection code all work on bgr from here.

Match the caps to what your element actually advertises: run gst-inspect-1.0 on your source element and copy the media type and format names from its pad template. Older 4.14-era board support packages list the caps under a different media type — the string that produced the two-channel error carried video/x-raw-yuv — and a mismatched caps name silently delivers no frames at all. Every field of the string is explained in GStreamer pipeline strings for on-board camera capture explained, and if VideoCapture fails at open rather than at cvtColor, OpenCV was built without GStreamer support — see OpenCV VideoCapture fails when GStreamer is not built in.

Fix 2: ask for BGR in the pipeline caps

Put the conversion inside the pipeline and camera.read() returns a 3-channel BGR frame directly:

pipeline = (
    "imxv4l2src device=/dev/video1 ! "
    "video/x-raw,format=YUYV,width=1280,height=720,framerate=30/1 ! "
    "videoconvert ! video/x-raw,format=BGR ! appsink"
)

The videoconvert element does the same unpack-and-matrix work cvtColor would, once per frame inside the pipeline, so every consumer — imshow, imwrite, a recorder, an inference thread — gets BGR with no per-call-site conversion to remember. OpenCV's convention is BGR; if a downstream library expects RGB, ask for format=RGB instead, and read convert recorded camera frames to BGR or RGB for OpenCV for how the two orders differ. If your driver itself can produce GREY or BGR, the formats gst-inspect-1.0 lists for the source, the caps alone negotiate it and no converter element runs at all.

One caution: with format=BGR every captured frame is converted, including the ones you drop. If you process one frame in ten, converting on demand does a tenth of the work; if you consume nearly every frame, the pipeline conversion is the cleaner place for it.

Which fix belongs where

Situation Use Conversion work
Colour needed for some frames only cv2.cvtColor(frame, cv2.COLOR_YUV2BGR_YUYV) on demand only frames you keep
Several consumers, recording, preview windows videoconvert ! video/x-raw,format=BGR in the caps every frame, in the pipeline
Inspection needs luma only frame[:, :, 0], no conversion none
Driver offers GREY or BGR natively format=GREY or format=BGR in the caps alone none

For the single-script case that produced the two-channel error, Fix 1 is the answer: it changes one line and leaves the pipeline string untouched. Fix 2 pays for itself once more than one piece of code consumes camera.read().

Tests that pin the cause in one pass

Run these in order; each either confirms the cause or moves you to the next.

  1. Print shape and dtype after the first read. (720, 1280, 2) uint8 confirms packed YUV and ends the diagnosis. A three-channel frame with wrong colours is a different failure — a swapped conversion code, covered in the FAQ below.
  2. Check the element's caps with gst-inspect-1.0 and compare them field by field with your string. A silent no-frames result, or a crash at open, is a caps mismatch — OpenCV segmentation fault on a CSI camera: fix the GStreamer caps walks that case.
  3. Run the same pipeline under gst-launch-1.0 ending in fakesink. If that holds the advertised 30 fps, the source and driver are fine and the bottleneck is downstream of the caps.
  4. Time 100 reads in your Python loop around the same pipeline. If gst-launch-1.0 holds 30 fps but the loop delivers a fraction of it, the callback or its copy is the limit — GStreamer v4l2src lost frames with appsink: your callback is too slow covers that path, and GStreamer appsink to OpenCV: pull frames without blocking the pipeline shows the non-blocking pull pattern.
  5. Black frames, or an image from the wrong sensor, point to a device-node problem rather than a channel problem — Dark frames from cv2.VideoCapture: the wrong video node is that case.

The arithmetic: where the conversion cost lands

Use the numbers for the 1280 × 720, 30 fps stream in the pipeline above.

  • YUYV out of the driver: 1280 × 720 × 2 = 1,843,200 bytes ≈ 1.84 MB per frame. At 30 fps: 1,843,200 × 30 = 55,296,000 bytes/s ≈ 55.3 MB/s.
  • BGR after conversion: 1280 × 720 × 3 = 2,764,800 bytes ≈ 2.76 MB per frame, or 82.9 MB/s at 30 fps.
  • Each converted frame moves 1,843,200 bytes read plus 2,764,800 bytes written ≈ 4.6 MB of memory traffic; at 30 fps that is 4,608,000 × 30 ≈ 138 MB/s spent on the colour conversion alone.

That traffic is why placement matters on a small board. The unpack, chroma interpolation and matrix cost the same whichever side performs them, so converting only the frames you keep, or none at all for luma-only inspection, is what saves the budget. If your gst-launch-1.0 test reached 30 fps while plain device-path capture delivered 7.5 fps, keep the pipeline — its caps and buffer negotiation are what the source sustains — and put the channel fix on top of it.

Prevent it: make the channel count a contract

Add a start-up check that reads one frame and dispatches on shape[2] before any cvtColor call exists in the code:

ok, frame = camera.read()
assert ok and frame.ndim == 3 and frame.dtype == "uint8", (ok, frame.shape if ok else None)
channels = frame.shape[2]  # 2 = packed YUV 4:2:2, 3 = already BGR/RGB, 1 = gray or raw Bayer

One helper mapping 2 to COLOR_YUV2BGR_YUYV, 3 to passthrough and 1 to your gray or Bayer path turns this failure into a printed diagnosis instead of a crash three calls deep. Packed sensor formats behave the same way across the family: a raw Bayer stream also arrives unconverted and single-channel, and needs its own code — Bayer raw stream from an RGB camera: how to know you have it shows how to tell which family you are holding. More capture-path diagnoses like this one are collected on the systems topic page.

If the module side of this pipeline is the part you are still choosing, the board-level cameras page lists CSI- and USB-ready modules, the area scan cameras finder covers GigE and USB3 cameras, and the camera finder filters by interface, resolution and shutter. Send your pipeline string through the contact page and we will check the caps with you.

Questions

My two-channel array prints pairs of values per pixel. Where is the colour hiding?

Channel 0 is the luma sample for that pixel. Channel 1 carries the shared chroma: at even columns it holds U, at odd columns V, because 4:2:2 stores one U and one V per two pixels. cv2.COLOR_YUV2BGR_YUYV unpacks each pair and colours both pixels from the shared chroma in a single optimised call.

Can I reshape the array to (720, 640, 4) and split the bytes myself?

You can, and you will get Y, U and V planes — but they are still planes, not a colour image. You would then have to upsample the chroma and apply the YUV-to-RGB matrix yourself. cvtColor does unpack, interpolation and matrix in one call, so reshape-and-split only pays off when you need the raw planes for custom processing.

Does the same two-channel problem happen with GigE or USB3 cameras?

Yes, whenever the camera streams a packed YUV 4:2:2 pixel format and you take frames raw. GigE Vision cameras commonly offer a YUV 4:2:2 packed format alongside mono and RGB, and USB modules advertise YUYV. The shape check and COLOR_YUV2BGR_YUYV behave the same; many SDKs hand you BGR already, so read frame.shape before assuming.

Why did plain device-path capture give 7.5 fps while gst-launch-1.0 reached 30 fps?

The plain device path can negotiate different caps or convert pixels inside the capture call, and that work costs the frame rate. Your gst-launch-1.0 test negotiates caps and buffers the source sustains. Keep that pipeline string in cv2.VideoCapture, fix the channel handling on top of it, and do not drop to the device path.

The conversion runs but the colours are still wrong. What next?

Check the exact packing your source negotiated. YUYV, UYVY and YVYU order the bytes differently, and each has its own code: COLOR_YUV2BGR_UYVY and COLOR_YUV2BGR_YVYU exist alongside COLOR_YUV2BGR_YUYV. Using the YUYV code on UYVY data swaps the chroma and shifts colours toward red or blue. gst-inspect-1.0 prints the format string to match; the shape stays (720, 1280, 2) in every case.