An integrated stereo depth module hands you clean, rectified infrared by default — and the moment you ask for the raw pair, the frame rate collapses to a short, fixed list of modes. This guide explains why both behaviours are designed in, walks the bandwidth arithmetic that sets the caps, and ends with a decision for each case: keep the rectified stream, take the raw stream at its listed modes, or build a discrete stereo pair from mono area scan cameras and rectify on the host. The worked numbers use the common full-HD module class (1920×1080 output, USB 3.0); the same arithmetic scales to any resolution.
What is the difference between the rectified and raw infrared stream?
The rectified stream is the output of the module's onboard depth engine — 8-bit images already warped so matching features sit on the same row in both eyes — while the raw stream is the sensor's own unrectified output in a 16-bit container, exposed only in a few fixed modes.
Stereo matching is only efficient when both views are rectified: calibration warps each image so a feature on one scanline can only match on the same scanline in the other eye. The depth engine needs that aligned pair to compute disparity anyway, so the module generates it internally and the driver exposes it first. On this module class the rectified formats are 8-bit and labelled Y8, and both infrared images come rectified by default; the 16-bit format does not.
The raw stream skips the warp. You receive the sensor output before the engine's geometry work and 8-bit conversion — exactly what you want for your own rectification, custom gain handling or texture-sensitive matching. On this class the raw format is labelled Y16, and it bypasses all of that processing.
| Property | Rectified stream (Y8) | Raw unrectified stream (Y16) |
|---|---|---|
| Who warps the image | the module's depth engine, onboard | nobody — you warp on the host |
| Pixel depth | 8-bit after onboard processing | 16-bit container, sensor-native range |
| Modes | the module's full stereo mode set | 1920×1080 at 25 fps with depth off; 960×540 |
| Epipolar rows | aligned for matching | as mounted, unaligned |
| Best for | depth, rate, plug-in matching | your own stereo, maximum detail |
Both behaviours live in firmware. There is no driver switch that rectifies Y16 or returns Y8 to sensor-native geometry.
Why does the rectified infrared image lose detail?
Rectification is a per-pixel coordinate remap computed from calibration parameters, and every remap resamples: each output pixel is a weighted blend of several input pixels, so fine texture, edges and speckle are low-pass filtered before the image ever reaches your code.
The warp grid is almost never pixel-aligned: where it magnifies locally the engine interpolates and softens, where it minifies it discards samples. Bilinear interpolation, the usual default, blends a 2×2 pixel neighbourhood — a small averaging filter applied at every output pixel.
If the engine rectifies at an intermediate resolution and rescales on the way out, the smoothing compounds. That matches the symptom users describe on these modules: rectified output that looks downsampled and upsampled again, with gaussian-like softening of edges, gratings and speckle.
You cannot undo it afterwards. A sharpening filter amplifies whatever survived the blend; it does not re-create a 2×2-averaged edge. If your signal lives in the fine texture — speckle contrast for your own matcher, surface grain for defect detection — the rectified feed has already discarded part of it before your code runs.
Why is the raw infrared stream capped at 25 fps at full resolution?
Because the raw path bypasses the depth engine, the firmware only lists a handful of combinations: on this module class, full-resolution raw is 1920×1080 at 25 fps with depth disabled, plus 960×540 — everything else falls back to the rectified formats.
The module is a small embedded computer, not a passive sensor. Its internal path from image sensor to depth engine to USB has a fixed transport budget, and the advertised combination list is that budget made visible. The sensor is not the limiter: the same imager streams faster in the rectified stereo modes.
Depth off is a condition of the mode, not a trick setting. With depth enabled the engine owns the pipeline and the 1080p raw combination is not offered at all; freeing the engine is what makes raw transport at that resolution possible. The 960×540 raw resolution works in the same depth-off mode.
Read the combination list out of the SDK before you design a mount or motion profile around raw — it is the module's contract, and it does not grow with driver updates. The same pixels-versus-rate trade governs every camera: see camera resolution versus frame rate: pick the stream.
What bandwidth does raw infrared at 60 fps actually need?
One raw 1080p stream at 60 fps moves about 1.99 Gbit/s, and both eyes together need 3.98 Gbit/s — past USB 3.0's practical 3.6 Gbit/s — so the two-eye version cannot exist on this interface at any firmware revision.
Running it for 1920×1080 raw, 2 bytes per pixel in the 16-bit container:
- Frame size: 1920 × 1080 px × 2 B = 4,147,200 B ≈ 4.15 MB.
- One eye at the listed 25 fps: 4,147,200 B × 25 = 103.7 MB/s ≈ 0.83 Gbit/s. That fits the cable easily, which tells you the 25 fps cap comes from the module's internal budget, not from USB.
- One eye at 60 fps: 4,147,200 B × 60 = 248.8 MB/s ≈ 1.99 Gbit/s.
- Both eyes at 60 fps: 2 × 1.99 Gbit/s = 3.98 Gbit/s. USB 3.0 signals at 5 Gbit/s and its 8b/10b line coding leaves 4 Gbit/s; real bulk throughput lands near 3.6 Gbit/s, about 450 MB/s. The two-eye request is over the practical ceiling before any control traffic or depth stream is counted.
So the two-eye raw mode cannot exist on USB 3.0 at any firmware revision, while the single-eye 60 fps request fits the cable and is excluded by the module's internal budget instead. 960×540 is the second listed mode for the same reason: exactly a quarter of the pixels, 518,400 px × 2 B = 1.04 MB per frame, about 0.50 Gbit/s at 60 fps.
USB budgets bite in ordinary use too: a hub that streams fine in a viewer can start dropping frames the moment recording starts — see why a depth camera fails on a USB hub and why a USB camera offers only 1920×1080.
Can a firmware update lift the raw infrared frame rate limits?
No. The combination list is fixed in the module's closed firmware, the vendor has given no indication of extra raw modes or an open release, and no upgrade setting changes this — plan around the listed modes or change hardware.
Onboard rectification plus alignment compensation is a deliberate design: the depth engine's accuracy depends on controlling the whole image path from sensor readout to disparity output. Extra raw modes — 10-bit passthrough, raw at 60 fps, 1280×800 at 90 fps — have been requested many times, and the published combination list has stayed exactly as it is. The firmware is closed, so no configuration or update adds modes.
If a project truly needs both, the answer is different hardware, not patience.
Should you stream rectified or raw infrared for your task?
Stream rectified whenever you also want depth or your matcher expects epipolar-aligned rows — it runs at the module's stereo rates. Take raw only when bit depth and unresampled detail matter more than rate, and accept 25 fps at 1080p or the 960×540 mode with depth off.
- Depth plus infrared from one device at the module's full stereo rate: rectified. The engine consumes exactly those images, so nothing extra is spent.
- Maximum infrared quality at modest speed: raw at 1920×1080, 25 fps, depth disabled.
- Raw with a smaller field or a lower data rate: 960×540, depth disabled.
- Raw detail and raw speed together: neither stream — build the discrete pair in the next section.
Whichever stream you subscribe to, the intrinsics that describe it differ, and pairing the wrong matrix with the wrong stream shifts every triangulated point. The rectified vs raw stereo IR images: which intrinsics apply guide sorts out which calibration goes with which feed.
When is a discrete stereo pair of area scan cameras the better answer?
When the task needs raw-quality detail at 60 fps or more, two hardware-synchronized mono global-shutter cameras with host-side rectification deliver it — you choose the sensor, exposure, bit depth and frame rate, and nothing onboard resamples the images.
Pick two identical mono sensors with good near-infrared sensitivity at your illumination wavelength, global shutter if anything moves, matched lenses, and synchronization by hardware trigger so both exposures start together. Take GigE or USB3 per camera according to cable length and data rate.
Rectification then moves into your code. Generate remap tables once from a calibration, resample each frame on arrival, and feed the aligned pair to your matcher: stereo calibration with fixed intrinsics covers the calibration split, raw stereo image calibration: verify streams and metadata the verification workflow, and live stereo rectification for calibrated camera video the streaming version. Stereo depth from host images picks up from the aligned pair onward.
Bandwidth and compute scale the way the arithmetic above showed: a 1080p pair at 60 fps is 248.8 MB/s per camera and 497.7 MB/s into the host, before your matching. Any discrete GPU resamples that in real time with large margin; where the cell runs all shift, mounting an industrial mini PC next to the cameras keeps cable runs short.
The trade is integration effort — two cameras, sync wiring and a calibration you own — in exchange for full control of resolution, bit depth and frame rate. Narrow the sensor choices in the camera finder.
Which depth alternatives skip the raw infrared problem entirely?
If the deliverable is geometry rather than infrared imagery, other 3D families measure depth without exposing a stereo infrared pair at all: projected structured light for dense static shape, laser galvanometer scanning for larger parts, ToF for volumes and logistics.
For dense shape on stationary parts, a projected structured-light camera casts a known pattern and triangulates from its deformation; the MV-DPS200P-02 is that family in our range.
For larger surfaces scanned sequentially, a laser galvanometer stereo camera such as the MV-DLS300P-04 steers a laser line over the part for high-accuracy profiles.
For volumes, logistics and pick-and-place, a ToF camera like the MV-DT01WMU returns depth per pixel with no stereo pair to rectify. ToF trades exposure against frame rate — the ToF exposure time and frame rate trade-off quantifies it — and has its own detail limits on small or distant parts, as covered in why ToF point clouds lose detail. The wider choice lives in the 3D sensors range.
- Type
- Stereo / structured light
- Type
- Stereo / structured light
- Type
- Depth / ToF camera
What do you check before committing to a raw or rectified set-up?
Forty minutes of checks beats a rework: read the advertised raw combinations, score both streams on real parts, and only then commit to the module's two streams or a discrete pair.
- Read the advertised raw combinations from the SDK at your target resolution before designing the mount or motion profile — the list is fixed firmware and will not grow.
- Score both streams of a sample unit on your actual parts: sharpness, speckle contrast and edge definition on the rectified feed against the raw feed at 25 fps. Softness only shows on your content.
- On the raw path, disable depth, confirm the mode enumerates, then log the delivered rate over a shift-length test — enumeration is not delivery.
- On the discrete pair, verify identical sensors, matched lenses, global shutter for motion, hardware trigger sync, and NIR sensitivity at your illumination wavelength before ordering lenses; any mismatch between the eyes becomes disparity noise.
- Budget the host: 497.7 MB/s of incoming raw for a 1080p pair at 60 fps, plus matching compute, per interface.
- Assign calibration ownership up front: factory calibration with rectified streams, or your own workflow with raw. Mixed ownership is the most common source of shifted 3D data.
If the decision lands on raw infrared at speed, the hardware route starts at the camera configurator; if it lands inside the module's two streams, the SDK combination list is your specification. The 3D vision guides page collects the rest of the stereo material, and contact us with your resolution, rate and bit-depth targets if you want the shortest route confirmed.
Questions
Why does the raw infrared format hold 16-bit values from a 10-bit sensor?
The 16-bit container is transport, not extra sensor detail. Sensors in this class are around 10 bits, and the driver scales those codes up so they survive onboard gain and conversion. When you process raw frames, divide by the scale factor — 16 in the common case — to recover the true codes. Storing frames as-is doubles memory for bits that carry no information.
Can I rectify the raw infrared pair myself on the host?
Yes. Export the module's factory calibration or calibrate the pair yourself, generate remap tables, and resample both images on arrival with bilinear interpolation. A GPU handles a 1080p frame in well under a millisecond, so two streams at 60 fps are not a compute problem. You keep the raw detail and bit depth and can re-rectify after any mechanical change; the cost is calibration management the module would otherwise do for you.
Do rectified infrared images use different intrinsics than raw ones?
Yes, and mismatching them is the classic integration bug. A rectified image is rendered through a virtual camera whose focal length, principal point and distortion differ from the physical sensor, so downstream code must use the intrinsics that match the stream it actually receives. Applying raw-camera intrinsics to rectified pixels shifts every triangulated point. Decide once per pipeline which stream feeds which matrix, and log the pairing with your calibration data.
Does the rectified stream's softening affect the module's own depth output?
Not in itself: the depth engine matches on the same rectified pair it hands you, so its disparity map is designed around that resampling. What suffers is your own post-processing — texture analysis, defect detection or custom matching run on the 8-bit images. If your algorithm needs sharper input than the engine needed, that is precisely the case for the raw stream or a discrete camera pair.
How long can the exposure be at 25 fps, and what does that do to moving parts?
Frame rate caps exposure: at 25 fps the longest exposure is 1/25 s, or 40 ms. Smear equals speed times exposure, so a part moving at 100 mm/s blurs 4 mm across the frame, and 1.7 mm at the 16.7 ms a 60 fps stream allows. Shorten the exposure and raise IR drive within the printed rating, or strobe the illuminator: a short pulse freezes motion at any frame rate.


