Sequential depth camera capture removes crosstalk but adds time skew

Two depth cameras aimed at overlapping volumes interfere: each one reads the other's projected pattern, and the depth images grow stripes and holes. The standard cure is to let the cameras capture one at a time — enable one, let it finish, then enable the next. Interference disappears, and a new problem takes its seat: the frames are no longer simultaneous. If your script waits 0.16 s between cameras, the second point cloud sits roughly 160 ms behind the first, and anything that moved in that window shows up as a mismatch. This guide covers the symptom, the arithmetic of the skew, the tests that isolate it, and the decision between sequential capture and simultaneous sync.

The symptom: clean depth, skewed frames

You started with two 3D sensors viewing one work volume, and the depth images striped or holed wherever the views overlapped. You switched to sequential control — one camera's enable line high, then low, then the next camera's — and the stripes vanished. Now the two point clouds disagree on moving parts: edges are displaced, a part present in one cloud is missing in the other, and the frame timestamps sit a fraction of a second apart. Nothing is broken. That time difference is the direct product of the capture sequence you are running.

The effect is not brand-specific. Projected structured-light cameras such as the MV-DPS200P-02, laser galvanometer units like the MV-DLS300P-04 and ToF units like the MV-DT01WMU all interfere when a pair illuminates the same volume at once.

The skew is in your script, not the cameras

The multi-camera sequence used by the vendor setup guides for this camera class goes: drive every enable line low so no camera projects, drive camera 0 high so it projects and captures, hold, drive it low, wait, then drive camera 1 high. The wait exists so the first camera's illumination is fully gone before the second one exposes. That wait is also your skew: the second exposure happens one hold plus one wait after the first.

import time

# Camera0 and Camera1 stand for the GPIO pin objects wired to each
# camera's enable input. GPIO.output, GPIO.HIGH and GPIO.LOW are the
# calls the vendor setup scripts use.
GPIO.output(Camera0, GPIO.LOW)    # every camera idle
GPIO.output(Camera1, GPIO.LOW)
GPIO.output(Camera0, GPIO.HIGH)   # camera 0 projects and captures
time.sleep(0.05)                  # hold: exposure plus readout plus margin
GPIO.output(Camera0, GPIO.LOW)    # its projector is off
time.sleep(0.16)                  # settle gap before camera 1: this is the skew
GPIO.output(Camera1, GPIO.HIGH)   # camera 1 projects and captures
time.sleep(0.05)
GPIO.output(Camera1, GPIO.LOW)

With a 50 ms hold and the 0.16 s gap, camera 1 exposes 0.05 s + 0.16 s = 0.21 s after camera 0. The proof costs one minute: change time.sleep(0.16) to time.sleep(0.32) and the delta between the two frames' timestamps grows by another 0.16 s. The cameras did not change; your sequence did.

Size the hold from the frame period. A 30 fps depth stream has a frame period of 1 ÷ 30 = 33 ms, so the hold must span at least one full frame plus readout margin — 50 ms is a sane floor. The trade inside one camera is covered in ToF camera exposure time and frame rate trade-off, and gating a capture to its exposure window is the same discipline as using exposure time as your shutter for an external flash.

Compute the skew and your motion budget

Three numbers decide everything:

• Skew — how far apart in time the two exposures are.
• Motion error — how far the part travels during the skew; the displacement you will measure between the two clouds.
• Skew budget — the largest skew your tolerance survives. When motion error exceeds tolerance, sequential capture is the wrong tool.

For a static scene, part speed is zero, motion error is zero and the skew is free. For anything that moves, compare budget with skew before you trust an overlay of the two clouds.

Worked example: two cameras over a 150 mm/s conveyor

Take a conveyor at 150 mm/s, an agreement tolerance of 2 mm between the two clouds, hold 0.05 s and wait 0.16 s.

• Skew: 0.05 s + 0.16 s = 0.21 s.
• Motion error: 150 mm/s × 0.21 s = 31.5 mm. The same part edge lands 31.5 mm apart in the two clouds.
• Skew budget: 2 mm ÷ 150 mm/s = 0.013 s, about 13 ms.
• Verdict: the error is 15.75 times the tolerance. Even a stripped round-robin — hold 35 ms for one 33 ms frame plus margin, wait 10 ms — leaves 45 ms of skew and 6.75 mm of error. Sequential capture cannot reach 13 ms on this line; go simultaneous.

One station over, the same arithmetic acquits sequential capture: a slowly drifting part at 10 mm/s with a 5 mm tolerance has a 500 ms budget, and the 0.21 s skew costs 10 mm/s × 0.21 s = 2.1 mm, inside tolerance with margin.

Why interference can survive a sequential script

If stripes remain after sequencing, the gating is not actually gating. Causes in order of likelihood:

1. Pulses too short or out of order. A high pulse shorter than one frame period can be ignored, and both cameras keep projecting on their own timers. Every hold must span exposure plus readout, and the gap must outlast the first camera's illumination.
2. A mode value set but no line driven. Enabling an inter-camera sync setting in a viewer or a robot wrapper only arms the feature. Until a real signal is wired to each enable input and driven high and low, both projectors stay on and interfere.
3. Auto exposure wandering between captures. Two cameras that expose differently will not match even without interference. Lock exposure, gain and laser settings on both — the same discipline that stops camera image brightness drift over time.

Ten-minute tests that isolate the cause

1. Timestamp test. Log the host time at which each camera's frame arrives, for ten cycles. The delta is your real skew. Change the wait from 0.16 s to 0.32 s and rerun; if the delta grows by 0.16 s, the script owns the skew.
2. Both-high test. Drive both enable lines high at once for a few frames. Stripes returning confirms the cameras interfere exactly when their illuminations overlap — and that your sequence was doing its job.
3. Hold test. Lengthen each hold to a full frame period plus 20 ms. Stripes disappearing means your pulses were too short to gate anything.
4. Fixed-exposure test. Set identical manual exposure on both cameras and compare depth noise. This separates exposure mismatch from interference.

Sequential or simultaneous: the decision

Simultaneous sync — one camera as master, the rest slaved to a shared line — aligns the capture windows and shifts the illumination staggering inside the frame period, so frames share a frame index while the projections never collide. The wiring and mode settings are covered in multi depth camera synchronization: slave mode and trigger setup; the shared timing event side in synchronizing multiple depth cameras with a shared timing event.

Respect the simultaneous caps from the vendor guides for this camera class: two cameras may emit together at the higher, roughly 1024-pixel-wide depth resolution, three at 640 x 480. Need more cameras than that? Capture two or three together as one simultaneous slot, and sequence the slots exactly as you sequenced single cameras.

Compensating the skew when you must stay sequential

When camera count or wiring forces sequential capture onto a moving scene, compensate. Measure the motion speed, then translate the second point cloud along the motion direction by speed × skew before comparing — 150 mm/s × 0.21 s = 31.5 mm on the example conveyor. This holds for rigid, constant-velocity motion such as a belt. It fails on vibrating or rotating parts and on accelerating robot motion, where the constant-speed assumption collapses mid-skew. There the honest fixes are simultaneous capture, or one camera with a wider field of view.

Cables, sync wiring and strobed lights

Each camera needs its own driven line to the enable input, so budget one controller output per camera plus common ground. Between the controller and the cameras, use shielded power and I/O cables — drag-chain rated where the harness moves — such as the M8 8-pin LS-CBL-0300.

If the cell also strobes 2D lights between depth captures, sequence those channels on a 4-channel digital unit from the light controllers range, such as the LS-CTL-0322, so light pulses land inside the gaps you already control.

For the depth side, compare models in the 3D camera finder; more timing and capture work sits on the 3D vision topic page. Cameras that fit this job:

Checklist before you change the code

• Log host timestamps for every frame of both cameras; measure the real delta instead of assuming it.
• Compute skew = hold + wait, then budget = tolerance ÷ speed, and compare the two.
• Confirm each hold spans exposure plus readout, with margin.
• Confirm both cameras obey the enable input: run the both-high test once and watch the stripes return.
• Lock exposure, gain and laser settings on all cameras.
• If the budget fails, move to master/slave sync before writing compensation code.
• Keep simultaneous groups inside the vendor caps for your depth resolution.