Global shutter infrared stream from a depth camera: enable IR and Y8

You need smear-free video of fast motion and the depth camera's color module keeps smearing it: its sensor is rolling shutter. The global shutter sensors sit behind the infrared windows — the two monochrome cameras the depth engine reads. Every stereo depth SDK exposes them as ordinary video streams, so the fix is a stream request, not new hardware. The steps below enable the infrared stream with the Y8 pixel format, pick the left IR camera, prove the smear is gone with a moving edge, and size the exposure that keeps blur inside a pixel budget.

Why the infrared camera is the global shutter one

A stereo depth camera is three cameras in one housing: two monochrome infrared cameras and one color module. The built-in projector textures the scene with an infrared pattern, the mono pair photographs it, and the depth engine finds depth by matching the two frames. Matching only works when both frames capture the same instant, so both infrared sensors are global shutter. The color module supplies color only, nothing in the depth computation waits on it, and it is a rolling-shutter sensor — the source of the skew you see.

The global shutter sensors are not hidden. The SDK lists their streams next to depth and color, and requesting one returns the mono sensor's frames directly, at full frame rate, with no line-by-line skew. For the sensor-level trade-off in general, see global vs rolling shutter sensors for moving targets. For how the rolling shutter term corrupts fast depth scans before you even compare streams, see rolling shutter blur limits depth accuracy on fast 3D scans.

Expect two properties from the mono frames. They are gray: one channel, no color filters on the sensor. And they are unfiltered in the near-infrared, because photography modules add an IR-cut filter and depth cameras leave it out so the projector light reaches the sensor. Materials can shift brightness between visible and infrared light, so judge contrast on the IR frames themselves. ToF cameras are built differently and get their own note below.

What you need

• A stereo depth camera with projector and mono pair — the stereo and structured-light types in the 3D sensors category
• Its SDK installed on the capture PC, with a compiler or script runtime for the pipeline calls
• Something moving at production speed: the conveyor itself, or a print with a straight vertical edge on a stick
• No extra hardware; the projector already inside the camera is your infrared light

Enable the infrared stream

Every stereo-depth SDK has the same four knobs: which stream, which pixel format, which of the two IR cameras, and which listed profile. The steps use one SDK's documented C++ names, with the generic knob alongside.

1. Open a fresh stream request. Most SDKs default to depth plus color; drop color if you want video only, or the link carries bandwidth you never read.
2. Enable the infrared stream and set the pixel format to 8-bit mono. The documented enum in this API family is RS2_FORMAT_Y8; other SDKs call the same setting mono8. One channel, one byte per pixel.
3. Pick the camera index for the left IR camera. Two infrared streams exist, left and right, and either gives the same global shutter video; streaming both doubles bandwidth for no benefit. If you later want the pair for your own stereo code, streaming left and right stereo image pairs from a depth camera covers it.
4. Take a listed resolution and frame rate. Enumerate the infrared stream's profiles and request one exactly as printed; an unlisted pair is the same trap that breaks depth set-up, and no matched video stream profile: set a listed depth mode walks through it.
5. Leave the infrared emitter enabled indoors. With it off, the sensor sees only thin ambient near-infrared and frames come back nearly black — that is passive IR, and no depth stream? your depth camera is in passive IR mode is the fix.
6. Start the pipeline and pull frames:

config cfg;                                            // stream request, SDK type
cfg.enable_stream(RS2_STREAM_INFRARED, RS2_FORMAT_Y8); // infrared stream, 8-bit mono
pipeline p;                                            // capture pipeline, SDK type
p.start(cfg);
frameset fs = p.wait_for_frames();                     // blocks until one frameset
frame ir = fs.get_infrared_frame();                    // the global shutter frame

cfg, p, fs and ir are illustrative variable names; the types and the two RS2_* enums are the SDK's documented names.

Prove the smear is gone

1. Shear test: slide a print with a straight vertical edge through the frame at production speed. On the RGB stream the edge tilts, its top lagging or leading its bottom. On the infrared stream the edge stays vertical in every frame. That contrast is the whole point, visible without measurement.
2. Channel check: an infrared Y8 frame is single-channel, 8-bit, values 0-255. If the stream you expected to be color arrives gray, you enabled IR instead — the reverse mix-up is common enough that RGB shows black and white: you are on the IR stream is its own guide.
3. Rate check: the delivered frame rate should sit at the requested profile. If it sags when depth or color also runs, the link is saturated rather than the sensor — frame rate low on software trigger: mode and data streams sorts out which.

Worked example: 300 mm/s belt and a 0.5 pixel blur budget

The set-up below is illustrative; substitute your belt speed, stand-off and profile, and the same two lines give your limit.

• Belt speed v = 300 mm/s — an illustrative mid-range conveyor speed
• Infrared profile 1280 × 720 px — an illustrative listed mono profile for this class
• Scene width FOV = 900 mm at the working distance — an illustrative stand-off; measure yours by laying a ruler in the frame
• Blur budget 0.5 px — a chosen target for gauge-grade edges; tighten or loosen to your tolerance

Global shutter exposes every row at once, so blur has one term: the distance the part travels during the exposure. In pixels, blur = v × t ÷ s, where s is the scene-to-pixel scale. Rolling shutter carries the same term plus skew, because each row is exposed slightly later than the last, and no exposure setting removes skew.

s = 900 mm ÷ 1280 px = 0.70 mm/px. t_max = 0.5 px × 0.70 mm/px ÷ 300 mm/s = 0.35 mm ÷ 300 mm/s = 1.17 ms. Dial setting: 1.0 ms. Blur at 1.0 ms: 300 mm/s × 0.0010 s = 0.30 mm, and 0.30 mm ÷ 0.70 mm/px = 0.43 px — inside the budget.

Set the infrared stream's exposure at or below 1.17 ms and the global shutter stream holds a 0.5 px budget at 300 mm/s; above it, the budget fails. What moves the limit: doubling belt speed to 600 mm/s halves it to 0.58 ms; halving the budget to 0.25 px also gives 0.58 ms; widening the scene to 1,200 mm raises it to 1.57 ms (0.5 × 0.94 ÷ 300). The color module cannot buy the same margin with exposure alone, because its skew term survives every setting — at production speed, the infrared stream is the only one on the camera that holds a pixel-level budget without a strobe.

When the color stream is still the answer

Mono infrared video solves the shutter problem and gives up color. When the inspection decision needs color — insulation grade, a stain, a printed mark — keep RGB out of the fast path rather than fixing it. Fire a short strobe during global reset if the camera offers that mode; it freezes a rolling-shutter sensor mid-read, and rolling shutter global reset with flash for moving objects shows the technique. Or add a dedicated global shutter camera for the moving view — browse the area scan cameras or the camera finder — and let the depth camera keep doing depth. Two cameras on one scene must be paired by device timestamp, not arrival order; sync RGB and depth frames by device timestamp explains why.

A ToF camera has no stereo pair to expose and does not need one: on a ToF sensing camera such as the MV-DT01WMU, the depth sensor is itself a fast full-frame imager, so the motion-safe stream is depth or its amplitude image, not a separate IR video channel.

Parts for a rig like this

Specifying rather than re-configuring? The architecture in the steps above is what a structured-light stereo depth camera is: the projector textures the scene, the mono pair reads it, and depth is computed from that pair — which is why these SDKs expose the pair as streams. In the shop, that is the Stereo / structured light type:

The MV-DPS200P-02 is a projected structured-light 3D camera. The MV-DLS300P-04 is a laser galvanometer stereo camera. Either way, the four knobs from the steps above — stream, format, camera index, profile — are the ones to set in the camera's own SDK.

Common mistakes

• Enabling everything at once. Color plus depth plus both IR streams saturates the link and the rate sags; cut color first when trimming.
• Treating gray frames as a fault. The infrared stream is monochrome by design; the color filters live on the RGB module only.
• Fixing a dark infrared frame with exposure. Blur scales with exposure whatever the shutter type; raise infrared illumination instead and stay under the limit from the worked example.
• Checking on a static scene. Rolling shutter and global shutter look identical when nothing moves; run the shear test at production speed.

The decision, and the next step

Restated for each case. Fast mono video: infrared stream, Y8, one camera, a listed profile, emitter on, exposure at or under the t_max your speed and field of view give. Fast color: strobe with global reset, or a separate global shutter camera beside the depth camera. ToF: stream its depth sensor directly. Slow scenes: RGB as usual. To build the rest of the rig around the camera — mount, trigger, controller — start the configurator at the camera step:

More set-up guides for this camera class are on the 3D topic page, and the team answers specification questions through the contact page.