Infrared image goes dark randomly: fix auto exposure with metadata

A stereo depth camera whose infrared image dips dark for a few frames at a time looks like a failing sensor, and usually is not one. The infrared stream is an exposure-controlled signal: with auto exposure enabled, a burst of extra light (direct sun, a specular flash, the neighboring camera's projector) drags the applied exposure down and the image with it. This guide gives you the metadata test that convicts or clears auto exposure in a one-minute capture, the fixes ranked by cause, and the decision rule: exposure collapses on the dark frames, it is the AE; exposure stays flat, it is your illumination.

What the dark frames actually are

When the infrared frames darken, stereo matching loses texture and depth drops out over the same frames — the cost is not the picture, it is the point cloud.

The camera is usually not losing light; it is choosing less of it. With auto exposure enabled, the exposure value you set is a setpoint only: the AE engine rewrites the applied exposure every frame, and the applied value arrives with each frame as metadata. Reading that metadata, not the setpoint, is the whole diagnosis — the method is in read exposure time from frame metadata under auto exposure.

Two behaviors give the AE a motive. Under direct sun it has been observed pulling exposure to its minimum, then recovering once the flash leaves the view. And because the loop reacts to the previous frame's brightness, a one-frame flash produces an over-correction that keeps several following frames dark. That is the 'random few dark frames' signature. Confirm the values you log are the applied ones — the checks are in per-frame exposure and gain: verifying camera metadata.

Likely causes, ranked

Rank them by how often each explains the symptom:

1. Direct sun or glint through the AE ROI. Outdoors the infrared sensor rides sunlight, and a glint off glass, water or a passing reflective part is enough for the AE to slam exposure down. It recovers when the glint leaves the ROI.
2. The other camera's projector. Depth cameras pulse their IR projector during their own exposure by default. Two free-running cameras drift through every phase, so the neighbor's pulse lands inside your exposure sometimes and misses it other times — random by construction.
3. Any other pulsed light in the cell. Strobe beacons, weld flash and other 3D sensors use the same mechanism with a different source.
4. An AE ROI aimed at the wrong region. A small ROI on the floor reacts to every passing bright patch. An AE ROI can be equal to or smaller than the image, so keep it on the part and leave the flashing surfaces out.
5. AE state at startup. If the setting applies late or is overwritten, early frames run at the wrong exposure and the symptom clusters at stream start — depth auto exposure not applied at startup: fix the setting order.
6. Not exposure at all. An emitter dropout, a power dip, a cable or an occlusion darkens frames with the exposure unchanged. It sits last because the metadata test clears it in seconds.

The test that settles it: log applied exposure, not the setpoint

Run this once and the cause is on paper:

1. Log per-frame metadata for 60 seconds of normal running: frame counter, host timestamp, applied exposure, applied gain. One CSV line per frame.
2. Flag dark frames objectively: mark every frame whose mean pixel value falls below half the running median.
3. Run a second 60-second capture with manual exposure fixed at the nominal value from step 1.
4. Compare the flags with the exposure column.

The rule: dark frames that coincide with an exposure collapse convict the AE; dark frames at flat exposure clear it and send you to the illumination path; dips that vanish under manual exposure confirm the AE even when the trigger is not yet identified.

Sensor signal scales with irradiance times exposure time, so a darker frame at constant scene brightness means a shorter exposure, and the ratio says how far the AE went:

Worked example: two cameras at 30° overlap, emitters off, 30 fps

Inputs:

• Rig: two stereo depth cameras mounted side by side with a 30° overlap, outdoors, IR emitters off because the scene is sunlit, AE on with the ROI set over the bottom two-thirds of the frame — the layout of the field case this article answers.
• Frame rate: 30 fps, so one frame period is 1 ÷ 30 s = 33.3 ms.
• Capture: 60 seconds → 30 × 60 = 1,800 frames per camera.
• The log: applied exposure holds at 4,000 µs on 1,786 frames and sits at 250 µs on 14 frames, and exactly those 14 frames are the dark ones. The exposure values are an illustrative log; your absolute numbers will differ, the ratio is what decides.

Rule in words: with scene brightness unchanged between adjacent frames, image signal scales only with exposure time, so a far darker frame must carry a proportionally shorter exposure.

Arithmetic:

• Exposure drop: 4,000 µs ÷ 250 µs = 16. The dark frames collected one sixteenth of the charge — four doublings, about 4 stops darker.
• Dark-frame share: 14 ÷ 1,800 = 0.78% of frames.
• Lost depth time: 14 × 33.3 ms = 466 ms ≈ 0.47 s per minute, which compounds to 0.47 × 60 ≈ 28 s per hour of running.
• Event size: at roughly three frames per event, each dropout costs 3 × 33.3 ms ≈ 100 ms of missing depth.

Result: the applied exposure fell to one sixteenth of nominal on precisely the dark frames. A stable outdoor scene does not shed 94% of its irradiance between adjacent frames, so the light did not change — the exposure did. The AE is convicted, and with the emitters off the remaining suspect is sun glint crossing the ROI.

Decision and threshold: treat any dark frame whose applied exposure fell below half of nominal as an AE event and go fix the stimulus — ROI, shading or synchronization. Had the log shown 4,000 µs on all 14 dark frames, the AE would be cleared and the hunt would move to the emitter, the supply and the optics.

Fix 1 — keep auto exposure inside a range you chose

Shrink the AE ROI to the surface the depth actually measures. Every bright non-part pixel inside the ROI is a vote for shorter exposure; sky, glazing and specular machine faces vote hardest. Re-check the ROI after any reposition of the camera or the part.

Cap the AE where your SDK exposes exposure and gain bounds for the infrared stream. Newer SDK and firmware releases add these limits to the depth AE; keep the SDK and firmware pair matched when you update — 3D camera header version mismatch: fix firmware and SDK alignment. Scope the cap honestly: a maximum bound protects frame rate, it does not raise the shortest exposure, so the ROI and the stimulus remain the cure for the dip.

If the log shows the scene brightness is stable enough, fix the exposure manually: set it so the brightest surface the ROI sees sits just below saturation, then re-check when the season or the sun angle shifts. In direct sun that fixed exposure may still saturate; size a neutral-density filter in that case — ND filter sizing for stereo depth cameras in direct sunlight.

In a multi-camera rig, give each camera its own AE configuration instead of one shared setting — set auto exposure per stereo camera in a depth rig.

Fix 2 — stop the cameras illuminating each other

When the metadata convicts the AE and the timestamps line up with the neighbor's frames, the mechanism is projector cross-talk. A depth camera pulses its projector during its own exposure by default, and two free-running cameras drift through every phase relation. Your exposure duty cycle predicts the hit rate: duty = t_exp ÷ T_frame. At a 4 ms exposure in a 33.3 ms frame period the duty is 12%, so the neighbor's pulse lands inside your exposure on the order of one frame in eight — a burst of dips every second, appearing and vanishing as the phases drift. Read your own exposure and frame period from the log; the arithmetic is the same.

Try the software test first: where the option exists, force both projectors to stay on continuously. A constant neighbor projection is a fixed attenuation you can calibrate or ignore, while a random pulse is neither; if the dips stop, cross-talk is confirmed without touching a wire.

The durable fix is hardware synchronization: one camera masters the frame timing and the slave exposes in the master's dark phase, genlock-style over the sync line. Afterwards, confirm from the metadata that the exposure phases no longer coincide. If a synced pair calibrates worse than it did free-running, suspect the trigger wiring before the cameras — hardware sync gives worse calibration than software: fix the trigger.

Outdoors the calculus differs: sunlit surfaces swamp the projector, so emitters off is the normal state there and the neighbor cannot inject anything — which is why the field case's random dips pointed back at sunlight. Indoors, where projectors carry the signal, synchronize before production trials.

When exposure stays flat: the illumination path

If the dark frames carry a flat exposure value, the AE is innocent and the light path is guilty. Work through it in this order:

• Emitter state. Some cameras report the projector's enable state per frame; log it beside the exposure. A projector that fires intermittently injects exactly the random burst that triggers an AE collapse, and a projector that dies leaves frames dark at any exposure.
• Power and cabling. A supply shared with the projector's pulse load, a tired connector or a marginal cable can drop the emitter for a frame or two; swap components and rerun the 60-second capture.
• Optics and occlusion. Contamination on a window, a bracket knocked out of place, or something passing in front of the lens part of the time.

Persistent black frames rather than intermittent dips belong to their own workflow — stereo infrared camera black image: isolate the cause. The same AE floor that dips frames shows up as noise in bright scenes — IR depth image noise in bright light: autoexposure floor. And when you toggle the emitter on and off, the AE behaves differently on each side — emitter on/off capture: why auto exposure is fixed and what to do.

Prevention checklist

Six habits keep this fault from coming back:

• Log applied exposure, gain and frame counter in every depth cell from commissioning day; a one-minute CSV answers this class of question forever.
• Keep the AE ROI on the measured surface, never on the whole frame or the horizon.
• Cap AE exposure and gain where the SDK offers the setting.
• Hardware-sync any two cameras whose fields overlap before production trials, not after the first dark-frame complaint.
• Shade the lens outdoors and audit a full day, because the sun path moves the glint across the ROI.
• After any cell change, rerun the 60-second capture and compare the dark-frame share with your baseline — under 1% of frames in the worked example; set your own limit from what the process tolerates.

When the fix is a camera that owns its illumination

In cells where ambient light cannot be controlled — outdoors, welding nearby, several 3D cameras in one bay — the durable answer is a depth camera whose illuminator fires only during its own exposure by design. Structured-light projectors, scanned-laser emitters and time-of-flight illuminators are synchronized to the sensor as part of the measurement itself, so infrared brightness stops riding on the sun and on the neighbors. Two pulsed units side by side still need synchronization or physical standoff.

A projected structured-light 3D camera such as the MV-DPS200P-02 carries its pattern projector in the housing; a laser galvanometer stereo camera such as the MV-DLS300P-04 scans its own light across the scene.

For volume and logistics lanes rather than dense surface inspection, a time-of-flight sensing camera such as the MV-DT01WMU measures its own modulated light, which puts illumination, exposure and depth in one deterministic chain.

The decision, per case

• Exposure collapses on the dark frames, one camera outdoors: narrow the ROI, shade the lens, then cap or fix the exposure; add attenuation only if the fixed exposure still saturates.
• Exposure collapses on the dark frames, two cameras indoors: synchronize or force continuous projection before touching anything else.
• Exposure flat on the dark frames: emitter, supply or optics — swap components, rerun the capture, and let the CSV name the guilty part.
• Either way the 60-second metadata capture is the first hour of the job, not the last.

Browse the 3D sensors range, open the camera finder, or start the camera configurator. If the CSV shows neither a collapse nor a dropout, send it through the contact page and we will read it with you. More depth-camera troubleshooting lives on the 3D vision topic page.