Split a 16-bit packed stereo frame into two 8-bit images

A stereo camera that packs both sensors into one video frame hands you a single 16-bit buffer where your code needs two 8-bit images — one per eye — for calibration, matching and display. The fix is a byte-level split, not a display setting: take alternate bytes or split each word into low and high bytes, reshape each half to the per-eye resolution, then label the two arrays with a lens cap. This guide gives the whole operation: a one-number test that picks the packing from the buffer length, the extraction for each layout, a worked 640 × 480 example with the arithmetic shown, and the checks that prove the split before you build on it.

What a packed 16-bit stereo frame contains

Some stereo heads send both sensors through one connection as a single frame; IEEE 1394 cameras that expose the packed mode as Format_7 Mode 3 are the classic case. The driver presents that frame as one monochrome 16-bit image, but only 8 bits of each 16-bit element carry one eye's pixel. The other 8 bits hold either the second eye or padding. Three packings cover the cases you will meet:

The element count picks the group, and the image pattern picks the row within the first group. Count elements before writing any splitting code. If a buffer ever looks half the size you expected, that same arithmetic sits behind why your 16-bit image buffer has half the expected pixels.

What you need

• Raw-frame access: the SDK call that hands you the frame as a flat array of unsigned 16-bit words or raw bytes, not the rendered display image.
• Per-eye resolution w × h in pixels; both eyes share it on every packed stereo head.
• An opaque card or lens cap for the labeling test.
• An image container that reshapes a flat array to h rows × w columns.
• The frame's bytes-per-row stride, for the case where your SDK pads rows.

Split the frame in six steps

1. Grab the raw frame as a flat unsigned 16-bit array or byte buffer, and copy it before splitting if the SDK reuses grab memory — a reused buffer tears in a way that masquerades as a split bug, the trap behind stereo frames saved wrong: copy Mat data from the SDK buffer.
2. Count the elements. 2 × w × h words points to interleaved words or an 8-bit side-by-side container; w × h words points to the byte-pair layout. Write both numbers next to the extraction so a changed camera mode cannot slip through.
3. Extract the two eye streams. Word-interleaved: decimate — take every other element — and mask each value with 255 to drop the padding byte. Byte-pair: one eye is value & 255, the other is value >> 8; that mask-and-shift pair is the same operation used when splitting a 16-bit PLC register into two 8-bit values. Row-interleaved: reshape to 2h rows × w columns and take every other row.
4. Reshape each eye stream to h rows × w columns in row-major order, unless the SDK's documentation states a different row order.
5. Label the eyes. Cap one lens, grab, and find which array collapses toward black; that array belongs to the capped lens. Set the left and right names in code from that observation, never from assumption.
6. Lock the splitter with a self-test. Build a synthetic buffer with known bytes at each position, run the same extraction lines, and assert the outputs byte for byte; the worked example below carries one.

Worked example: a 640 × 480-per-eye packed frame, little-endian

Inputs, and where each comes from:

• Per-eye resolution w = 640, h = 480 pixels — the integration case this method comes from.
• The raw grab is one flat unsigned 16-bit array of 614,400 elements, which is 1,228,800 bytes, as the SDK's raw-frame accessor reports it.
• Delivery is little-endian with pixel data in the low byte of each word — the common case; the parity check below catches the exception.

The rule in words: match the element count against the two predictions, 2 × w × h and w × h; the match names the packing and therefore the extraction.

The arithmetic:

• Pixels per eye: 640 × 480 = 307,200.
• Word-interleaved prediction: 2 × 307,200 = 614,400 elements, which equals the grab length — the eyes alternate word by word.
• Byte-pair prediction: 640 × 480 = 307,200 elements, half the grab — ruled out.
• Decimate: 614,400 ÷ 2 = 307,200 words per eye; masking with 255 leaves 307,200 bytes per eye.
• Reshape check: 480 × 640 = 307,200 for each eye.
• Reconciliation: 307,200 + 307,200 = 614,400, so every element of the grab is used exactly once.

Decision: decimate and mask. A byte-pair splitter on this grab would merge both eyes into one striped image and hand back padding for the second — exactly the failure the pattern check catches.

# Illustrative names: w, h, raw_bytes, eye_one, eye_two are ours.
# grab_raw_frame() stands in for your SDK's raw-frame or buffer-access call.
w, h = 640, 480                      # per-eye resolution, pixels
raw_bytes = grab_raw_frame()         # bytes; one packed stereo frame

assert len(raw_bytes) == 4 * w * h   # 1228800 for 640 x 480: word-interleaved

eye_one = raw_bytes[0::4]            # 307200 bytes: low byte of every even word
eye_two = raw_bytes[2::4]            # 307200 bytes: low byte of every odd word
img_one = to_image(eye_one, h, w)    # 480 rows x 640 columns, 8-bit
img_two = to_image(eye_two, h, w)

# Self-test with no camera: a synthetic frame with known bytes must come
# back split exactly.
test = bytes([10, 0, 200, 0] * (w * h))       # 10s for eye one, 200s for eye two
assert test[0::4] == bytes([10] * (w * h))    # eye one: 307200 bytes of 10
assert test[2::4] == bytes([200] * (w * h))   # eye two: 307200 bytes of 200

to_image stands in for the reshape to 480 rows × 640 columns. On big-endian delivery the data sits in the high byte, and the offsets move to 1::4 and 3::4. One grab moves 1,228,800 bytes: the strided slices run at memory-copy speed, while a per-pixel loop costs orders of magnitude more per frame.

Prove the split before you build on it

Three checks, in the order to run them:

• Lens-cap test. Grab with the scene unchanged, cap one lens, grab again. The capped eye's mean pixel value falls to near zero and the other barely moves. That result proves the extraction and labels the arrays at the same time; a shift in both images means the grab buffer was reused mid-read.
• Parity check. An all-zero image means you read the padding bytes — big-endian delivery — so shift the slice offsets by one byte.
• Pattern check. One-pixel vertical stripes mean the slice visits one eye twice or straddles words; blocky, sheared output means the camera packs row by row, and the reshape to 2h rows with alternate rows is the correct extraction.

Keep the timestamps and frame IDs of the two streams aligned after the split, which raw stereo image calibration: verify streams and metadata walks through. The payoff of two clean labeled images is per-eye control: set auto exposure per stereo camera in a depth rig covers the exposure step that packed output blocks.

When decimating is the wrong answer

Side-by-side 8-bit frames need a region split, not byte decimation. Reshape to h rows × 2w columns and take columns 0 to w-1 for one eye and w to 2w-1 for the other; a stacked frame splits into the first and second halves of the rows. Zoom to 200% on the raw frame: a hard vertical seam at column w confirms this packing.

A genuine 16-bit image holding 10- or 12-bit data is a conversion problem, not a stereo split: values above 255 are real depth, and an 8-bit cast destroys them. ADC bit depth vs pixel format: why 10-bit sensors ship 8-bit images covers that background, and packed 12-bit data has its own unpacking, shown in packed Bayer 12-bit conversion to a 16-bit image. If the SDK types the buffer as signed 16-bit, reinterpret the bits as unsigned before masking; I16 to U16 conversion: fix signed 16-bit camera data fast covers that fix.

Buy hardware that spares you the split

For a new build, make output format a selection question rather than an integration surprise. The one-minute check is the output-format list in each SDK: per-eye buffers delete this whole article from your codebase, a side-by-side frame needs only a column split, and a packed container needs the decimation above. The shop's 3D sensors range holds the stereo class, the area scan cameras page covers the single-sensor side of the same choice, and the camera finder filters both by interface and resolution. Stereo cameras currently in that range include:

The MV-DPS200P-02 is a projected structured-light 3D camera, and the MV-DLS300P-04 is a laser galvanometer stereo camera. Read the output-format section of each SDK before ordering, because it decides whether your code carries a splitter or a column slice.

Pre-flight checklist

• Element count checked against 2 × w × h and w × h before the extraction runs — a camera-mode change shows up immediately.
• Each eye array holds w × h bytes after masking, and the two totals reconcile to the grab length.
• No all-zero image and no one-pixel striping: parity and stride confirmed.
• Left and right labels come from the lens-cap test and are written into the code, not remembered.
• The synthetic-buffer self-test passes after every SDK or firmware update.
• The raw buffer is copied before splitting whenever the SDK reuses grab memory.

That is the whole operation: one count, two slices, one reshape per eye, and a lens cap to label the result. If the camera side of your build is still open, start at the camera configurator, and the 3D vision learn topic collects the calibration and matching guides that consume these two images.