Synchronizing multiple depth cameras with a shared timing event

Two depth cameras pointed at the same moving part give you two streams of frames with two unrelated sets of timestamps. Each camera counts time on its own internal hardware clock from the moment it powers up, and no SDK feature quietly links those clocks across devices. Within one camera, depth and color frames share a time base — see depth and color synchronization: check the right timestamps — but across cameras the timestamps are not comparable, and host arrival times carry too much queueing jitter to fix it. This guide shows you how to measure the offset between cameras with a shared visual event, when to prefer a hardware trigger, and how to keep multi-camera depth streams from corrupting each other.

Why the timestamps do not line up

A depth camera stamps each frame with the reading of its own hardware clock. That clock starts at zero when the device boots, runs on its own crystal, and drifts against every other camera's crystal. Three problems stack up:

1. Different origins. Camera A and camera B boot at different times, so equal timestamps mean different instants.
2. Different rates. Two crystals differ by tens of parts per million, so an offset measured once decays over minutes.
3. No cross-link. USB and Ethernet carry no shared clock signal to the cameras; only a dedicated trigger line does.

Host-side arrival timestamps do not rescue you: they include driver queueing and bus transfer time, which varies frame to frame by milliseconds. If you need to know that two exposures happened within, say, 5 ms of each other, you have to build the time relation yourself.

The shared sync-event method

The idea is simple: make something visible happen that every camera can see, capture it, and measure where it lands in each stream.

A strobed light filling the scene is the cleanest event. A light controller with a strobe output pulses a light on command, and the pulse lands inside exactly one frame per camera. You find, per camera, the first frame whose brightness jumps, and the difference of those frames' device timestamps is the clock offset.

For stereo cameras with controllable IR emitters there is a zero-extra-hardware variant: disable every emitter, stream all cameras, then flash one camera's emitter for a few frames. The texture change shows up in the other cameras' IR or depth views. The catch is interference, covered below — verify depth quality during the flash before trusting those frames.

Step by step: measure the offset with a strobed light

What you need: the cameras mounted and streaming, one strobed light and a controller that can fire it from your application, and a script that records per-frame device timestamps and mean intensity.

1. Fix exposure. Set every camera to fixed exposure and gain. Auto exposure reacts to the flash and smears the brightness step over several frames, which ruins the measurement. This is the same failure mode described in contour position jitter under automatic exposure.
2. Record a baseline. Stream all cameras for two seconds with the light off, logging device timestamp and mean intensity per frame.
3. Fire the pulse. Pulse the light once, 20–50 ms wide, so it spans at most one frame interval per camera.
4. Find the flash frame. For each camera, mark the first frame whose mean intensity exceeds the baseline median by, say, 20 percent. Call its device timestamp t_flash for that camera.
5. Repeat and take the median. Fire 10–20 pulses a few seconds apart and take the median of t_B(flash) − t_A(flash) per pulse. The median rejects the occasional pulse that straddles a frame boundary.
6. Track drift. Log the offset against wall-clock time over your longest expected run. Fit a slope; that is your drift rate. Extrapolate between calibrations and re-run after any camera power cycle, because a reboot resets that clock's origin.

A 4-channel 24 V digital controller such as the LS-CTL-0322 can drive the sync light and your inspection lights from one unit, with a strobe feature to time the pulse against the acquisition cycle. For how hard you can drive a strobed light without overheating it, see strobe machine vision light: overdrive, duty cycle and timing.

Worked example

Two cameras run at 30 fps (frame period 33.3 ms). You pulse a light 15 times and record, per pulse, the flash frame's device timestamp.

• Camera A, pulse 1: t_A = 10 041 220 µs
• Camera B, pulse 1: t_B = 8 902 905 µs
• Raw difference: 8 902 905 − 10 041 220 = −1 138 315 µs

The 15 pulses give differences of −1 138 315 µs (±1 frame period of quantization); the median is −1 138 340 µs. So:

A frame from camera B stamped 8 905 000 and a frame from camera A stamped 10 043 340 describe the same instant. Now the exposure-time difference the application actually wanted falls out directly: subtract the corrected timestamps of any two frames.

Uncertainty is one frame period, about 33 ms at 30 fps, because you only know which frame caught the flash, not where inside the exposure. Median-filtering many pulses cleans up outliers but cannot beat that floor. If you need millisecond or tighter alignment of the exposures themselves, move to hardware triggering.

When a hardware trigger is the better answer

A shared timing event measures the offset after the fact. A shared electrical edge removes the problem: wire one trigger source to every camera's trigger input and each exposure starts on the same edge, aligned to microseconds rather than to a frame period.

The details that make or break this — a common signal reference, correct line polarity, trigger widths, and what the GenICam line settings should be — are covered in camera trigger common reference: diagnose missing responses and camera hardware trigger frame rate with external pulses. Not every depth camera exposes a trigger input, so check the datasheet before committing to this route; if it does, prefer it over the flash method for anything that moves fast.

When several cameras must also be geometrically related, do the time work first, then the spatial work: multi camera extrinsic calibration in one shared coordinate frame assumes corresponding points were captured at the same instant.

Interference: not every depth technology tolerates a neighbour

Pointing several depth cameras at one surface is only safe for some technologies:

A laser-galvanometer stereo unit like the MV-DLS300P-04 relies on matching rather than decoding one pattern, which tolerates co-viewing better than a fixed-pattern projector. A compact ToF sensing camera such as the MV-DT01WMU needs emission planning when several ToF units share a workspace. Check the 3D sensors finder's type filter before adding a camera to an existing multi-camera cell.

If images from one camera glitch only while another streams, you are likely looking at shared-bus or power trouble rather than optical interference — multi-camera image corruption: isolate shared hardware faults walks the isolation sequence.

Checklist before you trust the alignment

• Fixed exposure and gain on every camera, auto modes off.
• Flash detection threshold well above ambient variation but below saturation.
• At least 10 pulses, median taken; outliers at frame boundaries rejected.
• Offset re-measured after any power cycle, and drift slope known for the run length.
• Depth quality during emitter flashes verified, not assumed.
• Trigger wiring checked against a common reference if you use the hardware route.
• A moving-object validation test: wave a target through the overlap region and confirm both corrected timestamps describe one trajectory.

For applications that also stitch several shots into one result, the same timing discipline feeds multi image capture per trigger: get an image set from one shot. Coordinating events across several viewer windows in a LabVIEW host has its own pitfalls, covered in a knowledgebase article on synchronizing LabVIEW events across multiple views.

Once time and space are both tied down, pick the rest of the parts in the system configurator, or contact us with your frame rates and camera count and we will check the bus, trigger and controller budget with you.