A structured-light depth camera hands you a finished depth map, but the two infrared images behind that map are often what you actually need: a matched left-right pair for your own stereo matching, for checkerboard calibration, or for texture-based inspection the depth stream cannot give you. Both streams are there — the camera just does not enable them together by default in many viewers. This guide walks the capture from wiring to saved files: which streams form the pair, how to lock them to one shared profile such as 640 × 480 at 30 fps, when to run the IR projector and when to shut it off, the exposure and bandwidth numbers, and the checks that tell you the pair is usable before you build on it.
Which streams make up the stereo pair on a structured-light depth camera?
A projected structured-light depth camera is a stereo pair in one housing: its left and right infrared imagers each publish their own raw IR stream, and the depth map you normally consume is computed from those two views of the projected dot pattern.
The two streams are monochrome infrared captures, usually offered as 8-bit Y8 or 16-bit raw. Request the 8-bit Y8 format for texture work; the global shutter infrared stream in IR and Y8 guide shows where those switches live. Both arrive raw and unrectified — lens distortion is still in the pixels — so calibration must use the raw geometry, as rectified vs raw stereo IR images: which intrinsics apply explains. The baseline between the imagers is machined into the housing, so it never changes between runs: one good calibration covers the camera for its life unless it takes a knock. A projected structured-light 3D camera such as the MV-DPS200P-02 is built on exactly this layout — a stereo pair of IR imagers behind a dot projector:
- Type
- Stereo / structured light
A scanned-line stereo unit follows the same pair logic, with a swept laser line instead of a static dot pattern:
- Type
- Stereo / structured light
How do you enable both IR streams at the same resolution?
In the viewer or the capture config, enable the left and right infrared streams as two separate video streams and lock both to one shared profile — in this guide's example, 640 × 480 at 30 fps — because rectification and block matching assume the same pixel grid on both sides.
In the viewer: select the module, enable the left and the right infrared entry, then choose one profile both entries list. The profiles come from the camera itself, so an unlisted combination returns a no-matched-profile error rather than a stream — set a listed video stream profile is the fix. In a capture script, request both IR streams in one configuration and leave RGB and depth off; color and depth streams at the same resolution shows why matching at the sensor beats scaling afterwards. If you keep 16-bit raw frames, note they carry no intrinsics — why 16-bit infrared streams lack intrinsics — so record the profile in your own metadata.
Use the largest shared resolution you can. Depth error grows with the square of distance and shrinks with the calibrated focal length in pixels:
Worked example with illustrative inputs: Z = 500 mm working distance, B = 50 mm baseline, f = 350 px calibrated focal length (a 640-wide imager at 350 px spans about 85° horizontally, typical for these optics), δd = 0.2 px matching error. Then δZ = 500² × 0.2 / (50 × 350) = 50 000 / 17 500 ≈ 2.9 mm. Double the resolution, f roughly doubles to 700 px, and the same scene resolves to about 1.4 mm.
Should the IR projector be on or off while you capture?
Leave the projector on — most depth SDKs expose this as an emitter switch, commonly emitter_enabled, set to 1 — when you want the dot pattern for depth work, and set it to 0 for a clean texture pair, adding plain illumination the imagers can see.
The dot pattern is synthetic texture: on bland surfaces it is what makes matching possible, which is why depth mode keeps it running. Turn it off when the image itself is the product — photometric inspection, print quality, or a target with printed features — and light the scene with something the IR imagers see; a room that looks bright to you can still be dark through their IR-pass filter. The projector-off path is the same one used for a clean 2D feed — disable the projector for a 2D live feed walks that setting. Whatever you choose, keep both halves in the same state, and after any toggle grab one throwaway frame: the change lands on a later frame than the request.
What exposure and frame rate does a clean pair need?
Set both imagers to the same manual exposure of about 10 ms at 30 fps: the frame period is 1/30 s ≈ 33.3 ms, so a 10 ms exposure fits inside one period with room for readout and emitter switching.
Turn auto-exposure off first. With it on, each imager converges on its own value and the halves drift apart photometrically — exactly what block matching punishes. Then apply three numbers:
At 30 fps a matched pair lands every 33.3 ms, and a 10 ms exposure uses about 30% of that period, keeping motion blur short. Bandwidth for the 640 × 480 example: 640 × 480 × 1 byte × 30 fps × 2 streams = 18.4 MB/s ≈ 147 Mbit/s — about 16% of the roughly 118 MB/s a GigE link carries, and comfortably inside USB 3.
One cap to expect: unrectified raw IR streams often run slower than the headline depth rate — unrectified raw infrared stream fps limits on stereo depth modules collects the reasons and the ceilings.
How do you trigger and save the pair without tearing it?
Grab the module as one unit — the two imagers inside a stereo depth camera are hardware frame-synced, so every left frame has a matching right frame — and write both halves of each matched set losslessly under one frame index.
What you need: the camera, its vendor SDK or viewer from the manufacturer's site, a rigid mount, a lit scene or target, and a host whose link carries the bandwidth you budgeted above. Then:
- Mount the camera on a stiff bracket and let it settle; nothing moves during a capture — mounting stillness during capture explains why a half-second of wobble ruins a pair.
- Connect straight to the host controller; through a hub, raw streams frequently refuse to start — why raw streams refuse to start on a USB hub documents that case.
- In the viewer, enable both infrared streams and disable RGB, depth and any auxiliary stream.
- Lock both IR entries to one shared profile — 640 × 480, 30 fps, 8-bit in the running example.
- Set auto-exposure off and both imagers to the same manual value, 10 ms.
- Choose the emitter state (1 for pattern, 0 for texture), start the stream, and drop one throwaway frame.
- Grab 20-30 pairs for calibration, moving and tilting the target between grabs and holding still for each; for an inspection run, capture the frames the cycle needs.
- Write both halves losslessly (PNG or raw) under one index — pair_0042_left.png and pair_0042_right.png — and let the capture script record a text sidecar with resolution, fps, exposure and emitter state.
How do you verify the pair is usable?
Run the saved pairs through a stereo calibration on a checkerboard target: a reprojection error under about 0.5 px RMS says the pair is rectifiable, while more than about 1 px points to motion, blur or a profile mismatch.
Before calibrating, open the files: both halves at the same resolution, frame indexes aligned, histograms not clipped at 0 or 255, and the same content visible on both sides — dot pattern if the emitter ran, scene texture if it did not. Then the real test: stereo camera calibration from paired checkerboard images covers target, poses and solver, and raw stereo image calibration: verify streams and metadata lists what a healthy capture carries. One more cross-check costs nothing: enable the depth stream once and compare its depth at a few known points against the disparity your own pipeline computes from the same scene.
What breaks the capture most often?
The three failures we see most often, in order: the two streams end up on different profiles, auto-exposure is still on so the halves drift apart photometrically, and the camera or scene moves during the 33.3 ms frame — each has a one-setting fix.
Profile mismatch: enable the two IR entries one at a time and read back what each reports, instead of trusting one dropdown for both. Auto-exposure left on: switch it off before the first grab; it cannot be repaired after the fact. Motion inside the frame: stiffen the mount and pause the scene — you cannot deblur a pair into agreement. Bandwidth is the fourth, quieter failure: leave RGB and depth streaming and a saturated link drops or delays frames, which shows up as mismatched indexes. And if a stream refuses to start at all, check the connection path before the settings — the hub problem in the capture steps accounts for a good share of no-stream reports.
What do you check in the ten seconds before capture?
Spend the last ten seconds before a capture run on six checks — both streams on one profile, manual equal exposure, deliberate emitter state, free bandwidth, rigid mount, lossless saving — then grab one throwaway pair and inspect it before the real run.
- Both IR streams enabled and reporting the same resolution, fps and pixel format — rectification assumes one pixel grid.
- Auto-exposure off, both imagers at the same manual value — matching compares pixel values.
- Emitter state chosen on purpose, 1 or 0 — and one throwaway frame after the last change.
- RGB, depth and auxiliary streams disabled — the pair needs the link's headroom.
- Camera rigid, scene still for the full exposure — motion inside one frame is unrepairable.
- Saving lossless with one index across both halves — lossy compression invents pixels a matcher will trust.
That is the whole loop: one shared profile, a deliberate emitter state, a manual exposure inside the period, and lossless indexed saving. To pick the sensor itself, the 3D camera finder filters by type, interface and resolution, and more capture and calibration fixes sit on the 3D vision topic page.
Laser profilers and stereo 3D cameras
170 models · from $699.00The configurator matches a camera, lens, light, light controller and cables to each other, step by step, and lists only parts that fit what you already picked.
Questions
Can I reconstruct the stereo pair from the depth map instead of the IR streams?
No, not losslessly. The depth map is already the product of matching: disparities are quantized, half-occluded pixels are filled or dropped, and the original grayscale values are gone. If you need custom matching, sub-pixel refinement or your own texture handling, capture the two raw IR streams — the depth stream tells you nothing about the pixels that made it.
Why do my saved IR frames look almost black?
The imagers see near-infrared through an IR-pass filter, so a scene that looks bright to you can still be dark to them. Set the emitter to 1 or add an IR-capable floodlight, then raise exposure toward the 11 ms ceiling that the 30 fps period allows. Judge by the histogram, not by preview brightness.
Should the left and right exposures ever differ?
Not for matching or calibration: rectification and block matching compare pixel values, so any radiometric offset between the halves shows up as disparity noise. Keep both at one manual value. A deliberate left-right offset is only worth trying for radiometric experiments, and even then record both exposure values with the pair so the difference can be corrected.
What resolution should I capture the IR pair at?
The largest resolution both imagers list in common. Depth error falls as the calibrated focal length in pixels rises, and doubling the resolution roughly halves depth error at the same distance — at the 500 mm worked example that moves the 2.9 mm estimate toward about 1.4 mm. If high profiles refuse to start, cut bandwidth elsewhere first, not resolution.
Can I get dot-pattern and clean texture pairs from one camera in one run?
Yes, by alternating the emitter state between frames. At 30 fps that yields about 15 patterned and 15 clean pairs per second, enough for most runs. Toggle the setting between grabs and discard one frame after each change, because it lands on a later frame than the request. Keep the exposure fixed across both states so the pairs stay comparable.

