Sync two CSI cameras on Jetson for stereo SLAM

Stereo visual SLAM only works when the left and right frames describe the same instant. Two CSI camera modules free-running on a Jetson do not: each sensor starts frames on its own oscillator, the offset sits anywhere inside the frame period, and it slides for minutes until it wraps around. This guide sets a numeric sync budget from your rig's motion, compares the four sync paths, walks through wiring a shared pulse or a sensor frame-sync line, and finishes with a bench test that proves the offset before the rig moves. The worked example is a 30 fps pair on a robot cornering at 90°/s — the case where timestamp pairing misses the budget by a factor of about 17, and one shared wire holds a tenfold margin.

Set the sync budget from rig motion

Work the budget in pixels, not milliseconds. While the rig yaws at rate ω, the image rotates across the sensor at f_px × ω pixels per second, where f_px is the focal length expressed in pixels. Whatever time separates the left and right exposure mid-points multiplies that rate into geometry error between the two frames, and the stereo matcher sees it as a violation of the epipolar constraint rather than as motion.

Budget one pixel. Feature matchers localize corners to a fraction of a pixel, so once the left–right geometry error passes one pixel, matches start dropping and triangulated depth turns noisy. A 1280 px image across a 90° horizontal FOV gives f_px = 640 px, and the budgets that follow are:

A wider lens lowers f_px and loosens the budget; a longer lens, a higher resolution, or a faster turn tightens it in proportion. Halve the pixel budget and the times halve with it. Compute your own number before touching a wire, then hold every candidate method against it.

Why timestamp pairing alone drifts to milliseconds

Two free-running sensors never share a frame clock. Commodity module oscillators hold ±50 ppm of nominal, so the worst-case relative drift is 100 ppm. At 30 fps the frame period is 33.33 ms, and the offset between the two sensors' frame starts slides by 100 ppm × 33.33 ms = 3.3 µs every frame. A full period passes in 33.33 ms ÷ 3.3 µs ≈ 10,000 frames — about 5.6 minutes at 30 fps.

Pairing frames by timestamp does not shrink that error bar, because the true exposure gap is whatever the clocks leave it: up to half a frame period, or 16.7 ms at 30 fps, at any moment. Host arrival times add millisecond-scale queue jitter on top. Against the table above, a rig turning at 90°/s has a 1.0 ms budget — the pairing worst case is about 17 times too wide, and it moves.

Pick the sync path

Four paths exist on a Jetson-class CSI rig. Price each against the budget from the first section and commit to the cheapest one that clears it with margin:

For modules built on Sony IMX sensors, driver support comes before wiring: IMX219 and IMX477 on Jetson: built-in driver support and stereo sync covers what the stock camera stack handles, which decides whether sensor-level sync features are reachable at all.

What you need

• Two CSI camera modules and the pinout page of the module connector, which shows whether a sync, trigger or strobe line exists and at what logic level.
• A Jetson carrier with a free header GPIO, for the shared-pulse path, or two modules that both support master/slave frame sync.
• Manual control of exposure and gain in the capture stack on both streams.
• A scope or multimeter to confirm pulse level and polarity at the module connector.
• A target that changes faster than your budget, for the bench test.

Wire and configure the shared trigger

1. Read the module pinout once and note what the connector exposes: a frame-sync line, an external trigger input, a strobe output, and the logic level. This one page decides between the two hardware paths.
2. Both modules expose a sync line: configure one as master, free-running, and the other as slave, starting each frame from the master's frame-start pulse. Set identical resolution, exposure and gain on both.
3. Only the carrier GPIO is available: route one GPIO to both modules' external trigger inputs, with a common ground and equal cable lengths. One pulse can drive three or four sensors the same way — the wiring in multi camera exposure synchronization with a shared trigger scales up unchanged.
4. Fix the exposure. Set the same manual exposure time and gain on both streams; per-camera auto exposure shifts each sensor's exposure mid-point and reintroduces exactly the offset you are removing. The depth-rig version of that failure is in set auto exposure per stereo camera in a depth rig.
5. Size the pulse rate below 1/(exposure + readout). With a 10 ms exposure and 15 ms readout the ceiling is 1 ÷ (0.010 s + 0.015 s) = 40 fps, so 30 fps leaves margin; the arithmetic is in maximum frame rate on a hardware trigger: exposure plus readout time.
6. Put both streams into external-trigger or frame-sync acquisition mode. A camera that delivers one frame and then stops is sitting in single-shot acquisition — the fix is in hardware trigger stops after one frame: check acquisition mode.
7. Stream both cameras and log per-frame timestamps and exposure values with each image where the capture stack supports it, as in chunk data on camera frames: carry timestamp and exposure per image. Then run the bench test below.

Worked example: a 30 fps stereo pair cornering at 90°/s

Inputs, with their sources: image width 1280 px across a 90° horizontal FOV, the geometry of a typical wide-angle stereo module (assumed for this case); yaw rate 90°/s = 1.57 rad/s, a robot turning through a corner; 30 fps streaming, so a 33.33 ms frame period; a budget of one pixel of left–right geometry error; oscillator tolerance ±50 ppm per module, giving 100 ppm worst-case relative drift; shared-pulse trigger-to-exposure latency under 100 µs, typical for sensors with a hardware trigger input.

The rule in words: between the left and right exposure mid-points the platform rotates by ω × Δt radians, which displaces the image on the sensor by f_px × ω × Δt pixels. Keep that product at or below one pixel.

• f_px = 1280 ÷ (2 × tan 45°) = 1280 ÷ 2 = 640 px
• Image rotation rate: 640 px × 1.57 rad/s = 1005 px/s
• Budget: 1 px ÷ 1005 px/s = 0.000995 s ≈ 1.0 ms
• Timestamp pairing worst case: 1005 px/s × 16.7 ms = 16.8 px of geometry error — about 17 times the budget
• Shared pulse: 1005 px/s × 100 µs = 0.10 px — a tenfold margin
• Drift: 100 ppm × 33.33 ms = 3.3 µs of slide per frame, a full period in about 10,000 frames, or 5.6 minutes

The result is a 1.0 ms budget. Timestamp pairing fails it by a factor of 17 at worst; a shared pulse passes with ten times the margin. The crossover where timestamps alone would suffice sits at ω = 1 px ÷ (640 px × 16.7 ms) = 0.094 rad/s, or about 5°/s — below that yaw rate, timestamp pairing holds one pixel; above it, hardware sync is not optional. A robot cornering at 90°/s sits an order of magnitude above the crossover, so this rig runs on a shared pulse.

Verify the sync on the bench

1. Point both cameras at something that changes faster than the budget: a spinning pointer on a screen, a flickering lamp, or the rig itself swung through a measured turn.
2. Pull matched frame pairs and check geometry against the known baseline and motion. The flicker test in verify hardware sync between two cameras actually works settles wiring and exposure alignment in under an hour.
3. Plot the left–right timestamp gap across a ten-minute run. A synced pair holds the gap flat within tens of microseconds; free-running clocks draw a slow triangle sweeping a full period about every 5.6 minutes at 30 fps.
4. Random per-frame gaps point at the trigger line rather than the clock — double pulses from noise or contact bounce produce exactly that signature.

Common mistakes that look like sync failures

• Frame-start sync is not simultaneous capture. A rolling-shutter sensor scans row by row, taking roughly 10–15 ms at this class of frame rate, so at 90°/s yaw the first and last rows see poses 5–8 px apart. That shear survives any trigger wiring; on a moving rig, global-shutter sensors remove it at the source.
• Per-camera auto exposure moves each sensor's exposure mid-point even when frame starts align. Fix exposure and gain manually on both.
• A pulse train above 1/(exposure + readout) drops frames; the drop mimics lost sync but is arithmetic.
• Pairing by arrival order at the host adds queue jitter. Use the sensor or device timestamp, the same rule as sync RGB and depth frames by device timestamp, not queue order.
• A GPIO fan-out on long, unterminated wire picks up noise and bounces. Keep runs short and driven; for longer distances, extend camera GPIO trigger cable length without losing sync gives the limits and the line drivers.
• Mixed sensor models carry different trigger-to-exposure latencies, and the difference lands directly as stereo offset — the same physics camera trigger delay timing for offset inspection sensors puts to work on purpose.

When to move the stereo pair off CSI

Commit by case. A bench prototype that barely rotates runs well on timestamp pairing. A moving rig — anything that corners — runs on a shared pulse through the module sync lines or the carrier GPIO. A product build that must survive years of field motion is better off on the industrial pattern: two global-shutter industrial cameras with hardware trigger inputs sharing one pulse, the same wiring as hardware sync two USB cameras: trigger both cameras together, with the pulse design in stereo camera trigger synchronization with a shared pulse.

Global shutter keeps both frames on the same instant, and GigE or USB3 transport trades CSI FPC routing for standard cables. For a triggered industrial pair, pick compact global-shutter models so both sensors integrate the identical instant:

Where the rig's job is dense depth at a fixed station rather than feature tracking on the move — a loading station, a mapped cell — a factory-synchronized structured-light 3D camera removes the sync work entirely:

The 3D sensors category holds these factory-synced, factory-calibrated rigs; builds that need a custom baseline instead follow the same shared-pulse rules at board level, as in board level cameras for stereo photogrammetry: sync and timestamps. Other trigger patterns across camera families are collected on the cameras topic page.

The decision in one line: budget one pixel of motion error, hold the left–right offset to a tenth of it, and prove it on the bench before the rig moves. To size an industrial pair, start at area scan cameras or the camera finder, or go straight to the configurator: