Stereo camera marker tracking: 3D points from two cameras

Two cameras on a fixed baseline can turn passive markers into tracked 3D points for animation, ergonomics and gesture work, without a dedicated capture system. The triangulation math is settled; feasibility lives in four practical decisions: shutter type, marker design, synchronization and calibration. This guide commits to each decision with numbers: the depth-error formula, a worked example at 1.5 m working distance, the position error a free-running pair accumulates, and the exposure a fast hand demands. You will finish able to size the baseline and lens for your volume, choose cameras by rig scale, and know when a third camera is the honest answer.

The short answer

Feasible, and the working recipe fits on one line: two global shutter color cameras, hardware-triggered together, mounted 300-600 mm apart for a desk-scale volume, each lighting the scene steadily, watching passive pads. Every pad that both cameras see becomes one 3D point per frame, so capture rate equals camera frame rate: a 60 fps pair samples each marker 60 times per second.

The scheme that historically failed did the opposite of passive: it flashed one group of LEDs per frame to stretch 24 markers out of a 90 fps sensor, which divided the effective rate by four to 22.5 fps and demanded that every row of every frame share one instant. A rolling shutter sensor cannot give that. Passive color or coded pads remove the multiplexing entirely, and with it the LED drivers, wiring and intra-frame synchronization burden.

The requirements, each with a section below: global shutter on both cameras; a shared hardware trigger; a calibration of both cameras and the pair; pads at least 10 pixels across at the farthest position; steady illumination without dark gaps. Miss any one and accuracy collapses before the math gets a vote.

Global shutter or rolling shutter: what the method decides

A global shutter exposes every pixel at the same instant. A rolling shutter walks down the sensor row by row, so the first and last rows of one frame are milliseconds apart. If marker identity rides on flash timing, those milliseconds destroy it: a lit LED becomes a streak across rows, and the two cameras disagree about when it was on. Per-frame LED interleaving and rolling shutter sensors cannot be combined.

Passive pads change the contract. Every marker is visible in every frame, so nothing inside a frame needs a shared instant; only the two frames of a stereo pair must share one, which is what the trigger section delivers. A rolling shutter sensor therefore still tracks slow, desk-scale motion, but fast pads smear along the readout direction and their centroids drift. Keep global shutter as the default and treat rolling shutter as a near-static special case. The trade-offs are laid out in global versus rolling shutter sensors for moving targets.

Marker design: color pads and coded fiducials

Two marker families work, and both are passive.

Color pads are flat matte disks in saturated hues that appear nowhere else in the scene. Detection is color segmentation plus a blob centroid, and identity is the hue; the count of separable hues is the practical ceiling, which is why desk rigs stop near a dozen. Matte paper or cloth beats glossy print, because a specular highlight walks the centroid as the pad tilts. Where glare resists, cross polarization strips the specular reflection.

Coded fiducials are printed segment or barcode codes with error correction: hundreds of unique identities, tolerant of partial occlusion, decoded by the same pattern-matching machinery used for part localization. The price is pixels, because every code module needs about two pixels of width to decode. Start from color pattern matching for reliable part localization, and when lighting drifts across the volume, edge-based pattern matching holds detection stable.

Either way, size the pad for ten pixels across at the farthest position in both cameras, and twenty or more for the cleanest sub-pixel centroids.

Stereo geometry: baseline, field of view and the depth budget

Three formulas run the whole sizing job:

Here W is sensor width, z the marker depth, f the focal length, b the baseline, d the disparity in pixels, and f_px the focal length in pixels, which is f divided by the pixel pitch. σd is the centroid error; sub-pixel blob fitting on a well-filled, unsaturated pad typically holds it to 0.1-0.3 px. The budget moves exactly as the formula says: double the distance and depth error quadruples; double the baseline or the focal length and it halves. Keep the volume inside the overlap of the two views, roughly FOV minus baseline across for parallel axes; a slight toe-in recenters it.

Size the focal length against your volume with the field-of-view calculator:

Worked example: 0.35 mm depth error at 1.5 m

Take a 1/1.8-inch sensor, 1920 pixels wide: pixel pitch = 7.1 mm / 1920 = 3.7 µm. A 12 mm lens then gives f_px = 12 mm / 0.0037 mm = 3243 px, and FOV at 1.5 m = 7.1 mm × 1500 mm / 12 mm = 888 mm, call it 890 mm. A 400 mm baseline leaves an 890 − 400 = 490 mm overlap band, enough for a desk-scale volume with a slight toe-in.

Disparity of a marker at 1.5 m: d = 3243 × 400 / 1500 = 865 px. With σd = 0.2 px: σz = 1500² × 0.2 / (3243 × 400) = 450,000 / 1,297,200 ≈ 0.35 mm. Lateral error is far kinder: σx ≈ z × σd / f_px = 1500 × 0.2 / 3243 ≈ 0.09 mm. That 0.35 mm is the random detection term; calibration error adds a systematic term on top, which is why the calibration section holds the line at 0.3 px reprojection.

Trigger the pair; do not let it free-run

Free-running cameras drift apart by up to one frame period. At 60 fps that is 16.7 ms; at hand speed, 2 m/s, the subject moves 33 mm between the two exposures. Stereo then matches two different instants and depth becomes noise. The budget rule is simple: shutter skew × subject speed < accuracy target. For 0.5 mm on a 2 m/s hand, the two shutters must fire within 0.25 ms of each other. A shared hardware trigger line does it in microseconds; software time-stamping at millisecond resolution does not.

Wire one controller pulse into both trigger inputs, or chain camera A's exposure-active output into camera B's trigger input. The wiring and timing arithmetic are in stereo camera trigger synchronization with a shared pulse, and the general scheme for faster pairs in synchronizing two cameras at high frame rate.

Exposure, motion blur and lighting

Blur equals speed times exposure. At 2 m/s, a 0.25 ms exposure smears a pad 0.5 mm, about one pixel at the worked example's 0.46 mm/px, while a 2 ms exposure smears it 4 mm, or eight pixels. Keep exposure at or below 0.25 ms for hand-speed capture, and shorter for impacts. Short exposures need photons, so light the volume evenly and check the frame rate does not sag at your exposure time; that interaction is explained in why your GigE camera frame rate drops at longer exposure times.

Flicker is the trap that wastes sessions. A 0.25 ms exposure sits entirely inside the off phase of a 1 kHz dimmer, so pad brightness jumps frame to frame; drive LEDs with steady current, or PWM at 20 kHz or above. The control side is covered in LED overdrive and current control for stable lighting.

At desk range, a diffused ring light per camera is the simplest steady fill: wide, even, glare-free output over the whole volume. A diffused white 120 mm ring light run from 24 V does this job on a bench rig:

Calibrate both cameras, then the pair

Three steps, in order. First, intrinsics per camera: a checkerboard in about twenty poses covering the frame corners gives focal length, principal point and distortion. Second, stereo extrinsics: the relative rotation and translation that set the effective baseline. Keep the baseline bar genuinely rigid, because a millimeter of flex is a millimeter of depth bias. Third, check reprojection error and hold it at or below 0.3 px; every 0.1 px of calibration error feeds straight into the σz formula above.

To turn camera-frame points into floor-frame coordinates, a ground-plane pose from four known points is enough for a single flat floor. The full chain from raw pairs to world coordinates is worked through in multi-camera marker triangulation and world coordinates; the procedure lives in the multi-camera calibration and synchronization workflow, and stereo calibration without a chessboard covers surveyed-point variants.

Camera picks by rig scale

Scale the camera to the volume, not the other way around. All three picks below expose the frame at once, so the trigger scheme applies unchanged; browse the full range in the area scan cameras category or the camera finder.

• Desk animation rig: roughly a 1 m cube at 1.5 m working distance. Pick the MV-CU020-90GC: 2.5 MP, color, global shutter, GigE.
• Measurement-grade or longer distances: step up to the MV-CH120-11UC, a 12 MP color global shutter camera on USB3, high-speed version.
• Big volumes, distant rigs, dense marker sets: the MV-CH250-25TC-F-NF, a 25 MP color global shutter camera on 10GigE with an F-mount.

Depth error scales with pixel pitch at a fixed field of view, so more pixels across the same FOV buy depth precision in direct proportion. Mono sensors see dimmer pads sooner, but color-coded markers need color sensors; coded fiducials in a single code family can run mono.

All 3 share: Sensor type CMOS, global shutter; Exposure mode Off/Once/Continuous exposure mode; Certification CE, RoHS, KC; Mono/color Color. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

When two cameras are not enough

Three honest limits. Occlusion: two cameras give two viewpoints, so a crossing arm can hide its pad from both; a third camera at a different height restores it. Distance: the same optics at 4 m give σz = 4000² × 0.2 / 1,297,200 ≈ 2.5 mm, so move the cameras closer before you stretch the baseline. Marker count: past a dozen hues, move to coded pads; past roughly twenty fast markers, add cameras or resolution and re-budget the data rate, which the high-frame-rate recording planner walks through.

Checks before you order

Run this list before any purchase order:

• Global shutter on both cameras. Rolling shutter breaks LED interleaving outright and skews fast passive pads.
• A hardware trigger input on each camera. Free-run costs up to 16.7 ms of skew at 60 fps, which is 33 mm at hand speed.
• Pads at least 10 px across at the farthest position in both views. Size them with the field-of-view math above.
• Exposure at or below 0.25 ms for 2 m/s motion, with illumination that holds brightness at that exposure.
• No PWM dimming below about 20 kHz on any light the camera sees.
• A calibration plan with a 0.3 px reprojection budget.
• A data budget. Two 640 × 480, 8-bit streams at 60 fps total about 37 MB/s and fit one GigE link, which carries roughly 110 MB/s; larger sensors multiply quickly.

Start with the pair, then grow

Pick the pair first: two global shutter color cameras with a trigger input, then the baseline bar, then the lights. The camera configurator steps through camera, lens and trigger in one pass, and the 3D topic page collects the calibration and triangulation guides around this one, with the rest of the knowledge base beside it. If you send us your volume size, marker count and speed budget through the contact page, we will size the baseline, lens and cameras against the numbers above.