Pair frames from two synced cameras by timestamp

You have two USB3 cameras on one hardwired trigger, a controller and a follower, and both streams deliver frames happily — but the frame IDs never line up, so you cannot tell which image goes with which. The fix is in the frame metadata you already have: every frame carries a timestamp taken when its exposure started. This guide shows you how to pair frames from the two streams by that timestamp, how tight to make the match, and how to set the trigger rate and USB bandwidth so a rolling-shutter pair keeps up.

Why don't matching frame IDs pair up?

Frame IDs are delivered-frame counters, not trigger counters: if either camera drops or corrupts one buffer, every later ID on that camera shifts by one, so controller ID 214 and follower ID 214 are usually two different trigger pulses.

Pair on time instead. Every frame in a GenICam stream also carries a timestamp marking when that frame's exposure started, so two frames fired by one pulse sit within microseconds of each other even when their IDs differ by three or four. In each callback, push a record of frame ID, timestamp and pixel copy onto that camera's queue, and keep the ID only for auditing deliveries.

The gap grows over a run: one bad buffer costs an ID on one camera forever after, and USB retries can reorder arrivals near the host. Timestamps ignore both effects, which is why the pairing rule below never reads the ID.

If the wiring is the part still giving trouble, hardware sync two USB cameras covers the controller/follower cable before capture starts.

Which timestamp marks a frame, and where do you read it?

Pair on the frame timestamp, which the GenICam standard defines as the moment that frame's exposure started, and read it inside the streaming callback together with the frame ID, because the buffer you get there is reused once you requeue it.

The timestamp rides in the image stream on GenICam-conformant cameras (the SFNC feature dictionary calls it Timestamp), so your SDK exposes it beside the ID on every frame object; look up the exact accessor in the SDK reference rather than guessing a method name. Note the tick unit once — most cameras count nanoseconds on their own clock — and do every comparison in that unit.

The two clocks started independently, so controller time 1,000,000,000 ns and follower time 987,654,321 ns can be the same pulse; the offset between the clocks is unknown until the next section estimates it. Absolute times never pair; corrected differences do.

Convert or copy the pixels inside the callback, then requeue the buffer. The queue must own its copy.

How do you pair frames after capture?

Estimate the clock offset once from the first triggered pairs, then match every controller frame to the unpaired follower frame with the smallest corrected time difference and accept the pair only when that difference stays inside your tolerance.

The offset exists because each camera's clock started when that camera powered up, not when your program did. Estimate it from frames you know saw the same pulse, then match by corrected time:

1. Start both streams, then let the trigger run. In each callback push (frame ID, timestamp, pixel copy) onto that camera's queue; keep at least four buffers per stream.
2. Collect the first ten timestamp pairs that land within one trigger period of each other and set offset = median(t_follower − t_controller). One pair starts you off; the median over ten rejects a bad first sample.
3. For each new controller frame, scan the follower queue for the smallest |t_follower − (t_controller + offset)|.
4. Accept when that value is within tolerance: pop both records and write the pair out with both IDs and both timestamps.
5. Hold an unmatched controller frame for the next follower arrivals; if it ages past one trigger period, release it and increment a drop counter.
6. Re-estimate the offset when the residuals of accepted pairs start to grow — a jump means one camera restarted its clock.

If your pair runs a trigger-ready handshake before each capture, frame trigger ready timeout: set up a USB stereo trigger pair covers that side of the set-up.

How tight should the matching tolerance be?

Start at 1 ms: cameras on one trigger line start exposure within tens of microseconds of each other, so 1 ms sits far above the real skew and far below the 33 ms half-period that would let a missing frame pair with the wrong pulse.

Keep two inequalities in mind: tolerance of at least about five times the measured skew, and tolerance below half the trigger period. At 15 fps the period is 66.7 ms, half is 33.3 ms, and a 1 ms tolerance leaves 66.7 ms of distance to the nearest wrong frame — so a dropped frame cannot pull a partner across the gap.

Measure the true skew once with the scope set-up in the verification section below. If the edges sit within 40 µs of each other, tighten the tolerance to 200 µs: every true pair still passes, and anything not from the same pulse fails. Start at 1 ms, tighten to 200 µs, and let the residual histogram tell you when to stop.

What frame rate will a rolling-shutter pair actually hold?

Count on about half the sensor's maximum frame rate under hardware trigger while the camera's current firmware serializes exposure and readout; later firmware raises the triggered rate to nearly the free-running figure on models that document the fix.

A rolling shutter exposes and reads rows at the same time, and a trigger restarts that cycle from row one, so one triggered frame occupies about two free-running slots. Divide, don't hope.

The camera below carries the example numbers.

Its data sheet prints 30 fps at 4000 × 3000 over USB3, rolling shutter. Hardware trigger halves that: 30 fps ÷ 2 = 15 fps, a period of 1000 ms ÷ 15 = 66.7 ms. Readout at the sensor's own cadence takes roughly 1 ÷ 30 ≈ 33 ms, so exposure must fit in 66.7 − 33 ≈ 33 ms — keep it at 30 ms. Set the controller to 15 Hz and the pair delivers 15 synchronized frames per second, per camera.

The 6 MP USB3.0 mono camera below is the same class of rolling-shutter part; buy a twin, wire a single trigger line, and run the pair at half the rated rate the same way.

Global-shutter cameras pay none of this tax, because every row exposes at once. The arithmetic from trigger pulse to delivered rate is worked through in camera hardware trigger frame rate with external pulses.

How much USB bandwidth do two triggered cameras need?

A 4000 × 3000 Mono8 frame is 12 MB, so a 15 fps trigger pair moves 180 MB/s per camera; two cameras on one USB3 controller push 360 MB/s, at the edge of what a 5 Gbit/s port sustains, so split them onto two controllers.

The basis: USB 3.0 carries 5 Gbit/s on the wire, and after line coding and protocol overhead a host controller sustains roughly 350–450 MB/s. Two cameras at 360 MB/s leave nothing for retries, and the failure mode is not a lost frame but an incomplete one.

DeviceLinkThroughputLimit, the GenICam feature that caps a camera's datarate in bytes per second, is the knob. Raise it per camera until incomplete frames stop appearing, then back off about 10 percent. One camera per controller: 180,000,000 B/s serves the 15 fps Mono8 pair. Both cameras on one controller: cut the trigger to 12 fps, where 12 MB × 12 = 144 MB/s each, 288 MB/s total, and the controller breathes again.

On Linux, clear the usbfs memory cap before blaming the camera; USB3 camera low frame rate and latency: fix usbfs memory and exposure walks that setting. With one controller on the motherboard, a quad USB3.0 host card gives each camera its own bus.

How do you prove the pairs are really in sync?

Route each camera's exposure-active signal to a GPIO line set as output and watch both on a two-channel scope: the rising edges show the true trigger skew in microseconds, and in software the residuals of accepted pairs should cluster within a few times that skew.

In GenICam terms: pick the GPIO with LineSelector, set LineMode = Output, and route the exposure-active signal through LineSource — cameras that name that signal Expo take LineSource = Expo. Feed both lines to the scope; the edge gap is the skew your tolerance must clear. If a camera ignores its trigger line instead, hardware trigger source not detected: check GenICam line settings sorts out the usual configuration misses.

Then audit in software: log the residual |t_follower − (t_controller + offset)| for every accepted pair and plot it. Healthy residuals form one tight cluster a few times the scope skew; a drifting mean means a stale offset, and a jump out to the frame period means pairs are slipping by one pulse. The same timestamps double as a frame-rate audit — camera frame rate measurement from timestamped frames shows the method.

Close out with this list:

• Both streams started before the first trigger pulse.
• The offset came from a same-pulse pair, not from program start times.
• Residuals form a single cluster within tens of microseconds.
• The unmatched-controller count equals the follower drop count.
• Scope edges sit within the skew your tolerance assumes.
• Every saved pair carries both IDs and both timestamps.

When is a timestamp pair the wrong answer?

Timestamp pairing synchronizes exposure starts, not sensor rows: if your parts move fast enough to blur across one readout, put global-shutter cameras on the same trigger line and keep timestamps only as the audit trail.

Decide by case:

• Slow or static parts, USB3 rolling-shutter pair: keep this set-up — 15 Hz trigger, 1 ms tolerance, timestamps as the audit trail. Frame-level sync is all the application needs.
• Fast motion or stereo measurement: switch to a global-shutter pair on the same line, with exposure short enough that both frames overlap inside the period. Timestamps become pure bookkeeping.
• Two cameras starving one USB controller: one controller per camera, or move to GigE. USB3 runs are a few metres; GigE reaches 100 m per segment, which decides the layout on its own.
• More than two cameras: the same pairing key works per pair; trigger everything from one buffered line so the skew stays in microseconds.

On GigE, a dual-port PoE NIC runs one camera per port and carries power over the same cable; cards of this kind sit with the frame grabbers.

To choose the pair itself, start in the camera finder or go straight to configuring a camera; the trigger and timing guides hub collects the rest of the sync fixes, and the team on the contact page reviews a specific cell if the numbers above do not close it.