16-bit depth and 8-bit infrared streams on one depth camera

Depth, infrared and color from one camera, simultaneously, at the highest bit depth each stream can carry. The pixel-format list in the SDK makes the rules look arbitrary, but they follow from where each stream comes from. Each stream carries the bit depth of the format set on it. The depth stream has exactly one format. The interface budget decides how many pixels the combination can move, and it binds before bit depth does. Below you will find the combinations that stream together, the reason 16-bit infrared usually displaces depth, the 16-bit color alternative, and a one-line bandwidth calculation that tells you whether the set-up fits a USB3 port.

The short answer: which bit depths stream together

Every stream is independent: set the format, get those bits. On the compact active stereo type - one module, two mono imagers, an infrared projector - the depth stream exposes Z16, 16-bit depth, and nothing else. There is no 10-bit or 12-bit depth format to fall back on. The infrared stream offers Y8 (8-bit mono) and usually Y16 (16-bit mono). Color, which this camera type derives from the depth imagers because it has no separate RGB sensor, arrives as 16-bit YUV: UYVY for the depth-plus-texture pair, YUYV when three streams run at once.

The table describes hardware, not policy. Y16 and depth cannot coexist because both are the same imager's output; the camera dedicates that imager to one job at a time. A stream also refuses any format outside its fixed list - the same behaviour as a camera pixel format that will not change on an ordinary camera. Modules that add a separate infrared or RGB sensor relax these conflicts, which is why format lists differ between camera families.

Why the depth stream is locked at 16 bits

An active stereo camera measures disparity: the shift of the projected pattern between the two imagers, converted to distance per pixel. That conversion, the mapping behind stereo disparity to depth conversion, writes a 16-bit range image - enough levels that millimetre values do not collide anywhere near the working distance, and precise enough to feed a point cloud. The interface exposes that array as Z16 and offers no thinner depth format. Reducing depth to 12 bits to fund a 12-bit infrared stream has nothing to switch to.

The levers that exist move pixels, not bits. A decimation post-process downsamples the depth image after capture, quartering what the PC stores and processes. Converting Z16 down to 8 bits in your own code - the trick behind 8-bit depth color maps - saves memory and processing time, but the full 16-bit stream has already crossed the wire, so it never reduces interface load.

The infrared stream: Y8, Y16 and the UYVY substitute

Y8 costs 1 byte per pixel and is what most pipelines should stream: it shows the projected speckle pattern, the target and the focus, while the measurement lives in Z16. Depth accuracy is unaffected by the infrared format. Y16 doubles that to 2 bytes per pixel and earns its place when the infrared image itself is the measurement - reflectance, material contrast, print inspection. On compact stereo modules the price is the depth stream: selecting Y16 turns depth off.

The substitute is UYVY, a 16-bit YUV color format. Because color on this camera type comes from the mono imagers, the UYVY image reads as infrared-like monochrome: 16-bit texture that runs beside Z16 depth. When depth, infrared and color must all stream, color maps into YUYV instead - the two formats differ in channel byte order, and the camera exposes each under the name the chosen combination needs. Exposure on that derived color stream is its own subject; setting color exposure on a depth camera with no RGB sensor covers what is adjustable.

One metadata difference matters for reprojection: 16-bit infrared streams on depth cameras often lack the intrinsics the 8-bit stream carries. The 16-bit infrared intrinsics gap explains the workaround.

Worked example: USB3 load for Z16 + Y8 + YUYV at 1280 x 720 and 30 fps

Inputs, and where each comes from:

• 1280 x 720 pixels per frame - the resolution the three streams share in the module's combined mode.
• 2 bytes per pixel for Z16, 1 byte per pixel for Y8, 2 bytes per pixel for YUYV - the formats' own definitions, sent uncompressed.
• 30 fps - an illustrative streaming rate for this camera class; the result scales linearly with the figure your mode list shows.

Arithmetic:

• Pixels per frame: 1280 x 720 = 921,600 px
• Z16: 921,600 px x 2 B = 1,843,200 B = 1.84 MB per frame
• Y8: 921,600 px x 1 B = 921,600 B = 0.92 MB per frame
• YUYV: 921,600 px x 2 B = 1,843,200 B = 1.84 MB per frame
• Frame total: 1,843,200 + 921,600 + 1,843,200 = 4,608,000 B = 4.61 MB
• Per second at 30 fps: 4,608,000 B x 30 = 138,240,000 B/s = 138 MB/s, or x 8 = 1,106 Mbit/s = 1.11 Gbit/s

USB3 delivers about 400 MB/s of practical payload once the 5 Gbit/s line rate loses encoding and protocol overhead. The three-stream set-up therefore uses about 35% of the port and leaves roughly 260 MB/s free.

The decision: Z16 + Y8 + YUYV at 1280 x 720 and 30 fps fits one USB3 port with margin. The failure point is sharing, not capacity: doubling to 60 fps gives 4,608,000 B x 60 = 276 MB/s, still inside the port but past the 75% working ceiling (about 300 MB/s) once a hub or another device joins. Two sensitivities: the load scales linearly with frame rate, and swapping Y8 for Y16 would add 0.92 MB per frame, for 166 MB/s in total - the wire could carry it, which shows the 16-bit infrared limit is set by the format list, not by bandwidth. For sizing across more streams and ports, the USB3 bandwidth sizing for a multi-stream pipeline guide goes wider, and the stream-timeout bandwidth check covers the symptom when a set-up exceeds its port.

Why resolution gives way before bit depth

The trade the hardware offers is pixels. On the common compact module, 848 x 480 is the depth-optimal mode: depth accuracy is tuned there, and it is the right choice when measurement quality leads. 1280 x 720 trades a little depth quality for streams that all share one size - and it is a ceiling, not a waypoint. Color comes from the depth imagers, so it cannot exceed their native 1280 x 720; 1920 x 1080 color is simply not offered on this type. Pixel density is also why decimation costs real information: precision, trueness and spatial resolution shows how the three scale with the image.

So the working rules: measurement first - 848 x 480 depth with Y8 infrared, and accept the smaller texture image. Three streams or a texture-led task - 1280 x 720 across the board, and fund PC savings with decimation rather than asking for thinner formats.

When the application needs true 16-bit texture next to depth

If the auxiliary image is itself the measurement - defect contrast under projected light, reflectance maps, surface grading - do not negotiate with the depth camera's format list. Two designs work. Capture Y16 alone, depth off, for the frames that need full infrared detail. Or add a separate area scan camera fired from the same trigger, align the two images with your own checkerboard calibration, then fuse them along the standard point cloud with RGB and IR intensity path.

For the depth side of such a rig, a structured-light depth camera returns dense depth alongside the projected texture:

Both project a pattern onto the scene and compute depth from its deformation - the same measurement principle as the compact stereo module, in a form built for fixed inspection cells.

When the usual answer is wrong: other depth technologies

Time-of-flight cameras skip stereo matching and return depth per pixel straight from the light's return time, again as a 16-bit range image in distance units. Their auxiliary streams differ from stereo modules, so treat the Y8/Y16 rules above as stereo-specific and enumerate the camera's own format list. A ToF depth camera suits volume and logistics cells where a direct depth map is the whole job:

Laser profilers change the stream shape entirely: a profile sensor streams height across a scanned line, producing the 2.5D height map that point clouds are built from, not an area image - so the depth-plus-infrared question does not arise there.

Checks before you commit the stream set-up

• Enumerate the format list per stream in the SDK, and set the scan mode first where the stream locks formats until you do - the 3D camera pixel format rules article shows the lock in action.
• Run the exact combination at the target frame rate for a minute and count delivered frames; a silent half-rate stream is the signature of an over-subscribed port, and the USB hub failure mode is the first place to look.
• Recompute the load with the formula above at your real frame rate, and keep a shared port under roughly 75% of USB3's 400 MB/s practical payload.
• Read the depth scale the SDK reports before interpreting Z16 values as millimetres.
• Decide color once: derived 1280 x 720 color from the depth imagers, or a triggered second camera for true full-HD.

The whole decision compresses to three numbers: 16 (depth bits, fixed), 8 or 16 (infrared bits, with 16 displacing depth on compact stereo), and the byte total per frame against the port. More depth-stream mechanics sit on the 3D topic page. If you are selecting hardware rather than configuring it, start from the camera finder filtered to depth cameras, or bring the camera, resolution and frame rate to the contact page and we will size the stream set-up with you.