Sparse point cloud in pixels: convert it to metric scale

A sparse point cloud from an essential-matrix reconstruction looks like hundreds of numbers per axis, and none of them is a millimetre. The essential matrix fixes the rotation between your two cameras exactly and leaves the translation known only up to one unknown scale, so the whole cloud is a faithful copy of the scene multiplied by an arbitrary constant. That constant is a single number, and you already own it: the distance between the two cameras. Below you will compute the scale factor from the triangulated camera centers, work a 118.4-unit example that ends in millimetres, and see when a calibrated stereo pair makes the whole fix unnecessary.

Why the point cloud has no units

The essential matrix E packs the relative rotation R and the translation direction t between the two cameras into a 3×3 matrix of rank 2 with five degrees of freedom. Estimated from matched points, its overall magnitude is unobservable: E and any nonzero multiple of it satisfy the same epipolar constraint for every match. Decomposition therefore returns R exactly and t only up to length, and most pipelines normalize the translation to length 1.

Converting the fundamental matrix F to E with the intrinsics, E = K₂ᵀ F K₁, removes the pixel scale of the images but adds nothing metric: the intrinsics hold focal length in pixels, never in millimetres. Triangulation with P1 = K[I | 0] and P2 = K[R | t] rebuilds a scene with the right shape and an arbitrary size. The coordinates land in the same numeric range as your image features, which is why the cloud reads as pixels.

The reconstruction is a similarity of the true scene, so one single factor is wrong everywhere, the same factor for every point and every axis. Directions, angles and relative poses are already correct.

The scale factor: true baseline over reconstructed baseline

Compute the two camera centers in the same frame and normalization as the cloud. For P = K[R | t] the center is C = -R⁻¹t, and P1 = K[I | 0] puts the first camera at the origin. Their separation is the reconstructed baseline B_rec; the caliper distance between the real cameras is B_true.

Under the unit-norm convention B_rec comes out as exactly 1.000, so s equals the measured baseline in millimetres — that is why the folk fix is 'multiply by the baseline'. Measure B_true once, multiply points and centers by the same s, and leave R alone.

The baseline is not the only usable length. If two triangulated points sit 500.0 mm apart on the real part, s = 500.0 mm divided by their reconstructed separation reproduces the same cloud. The baseline is simply the length you can measure most precisely, because the cameras sit on a rail within caliper reach.

Worked example: 118.4 cloud units become 120.0 mm

Two identical global-shutter GigE cameras sit side by side on a rail, and calipers between the C-mount flange centres give B_true = 120.0 mm. The pipeline decomposed E, built P2 = K[R | t], and triangulated the cloud and both camera centers:

• C_L = (0.0, 0.0, 0.0)
• C_R = (-118.3, -4.1, -3.5), in cloud units

Reconstructed baseline and scale factor:

B_rec = √(118.3² + 4.1² + 3.5²) = √(13994.89 + 16.81 + 12.25) = √14023.95 = 118.4 cloud units

s = 120.0 mm / 118.4 units = 1.0135 mm per unit

A matched feature triangulated at (385.2, 244.8, 402.6) becomes:

• X: 385.2 × 1.0135 = 390.4 mm
• Y: 244.8 × 1.0135 = 248.1 mm
• Z: 402.6 × 1.0135 = 408.0 mm

Cross-check the same feature against the disparity equation, with f_x = 1000 px from the left intrinsics and a disparity of 294 px in the rectified pair:

The scaled cloud says 408.0 mm and the disparity route says 408.2 mm — agreement within rounding. A gap of more than a few percent means the baseline measurement or the match set is wrong, not the arithmetic.

Measuring the baseline on the bench

The baseline runs between the two entrance pupils — the projection centers — not between housing edges. Three cases cover most rigs:

• Parallel rig, identical cameras and lenses: both pupils sit at the same depth, so calipers between the two C-mount flange centres give the baseline directly. That is the 120.0 mm of the worked example.
• Verged or angled rig: the baseline is still the straight-line pupil distance, and stereo calibration with angled cameras: what still works covers what else changes. Keep both lenses identical so the equal pupil offsets cancel.
• Mixed lenses: pupil depth differs between designs. If the two pupils sit 5 mm apart in depth on a 120 mm rail, the flange-to-flange reading is off by 5 mm and every depth is scaled 4.2% wrong (5 / 120 = 4.2%). Use a matched pair or measure each lens's pupil position.

How long the baseline should be is its own trade between depth precision at range and matching failures up close; choosing a stereo camera baseline distance works that choice with numbers.

Fix the cloud or calibrate the pair

The calibrated route kills the ambiguity at the source. A planar target of known square size, seen by both cameras, anchors the stereo translation in real units, so triangulation returns millimetres with no post-factor; stereo disparity scale and calibration for depth estimates walks through the depth side. If no chessboard can enter the scene, stereo calibration without a chessboard, from surveyed scene points is the practical alternative.

Whichever route you take, rectify the pair first, and if the epipolar lines are not horizontal, fix camera order and baseline in stereo rectification before triangulating. Dense clouds follow the same scale logic, and depth map to real distance calibration for XYZ point clouds is the dense-case companion. The 3D topic page gathers the remaining stereo-geometry pieces.

Hardware that ships calibrated

If you are specifying hardware rather than rescuing a cloud, an integrated stereo 3D camera fixes both sensors on one rigid base, so the extrinsics hold between runs. Two examples from the shop's 3D sensors range:

The DIY route is two identical global-shutter GigE cameras on a measured rail, fired by one hardware trigger so both frames expose the same instant; one stereo calibration then serves every run. The camera finder filters by shutter type and interface; a typical pair member:

Either way the depth still comes from the same geometry — f_x, baseline and disparity — so the checks below apply unchanged. When you read Z from a depth device instead of your own triangulation, depth value vs true distance: converting a pixel's Z to camera distance is the companion step.

Checks before you trust the millimetres

1. Scale points and camera centers together, reproject the original matches and confirm the pixel residuals are unchanged. Moving residuals mean the centers came from a different decomposition than the cloud.
2. Measure one physical length in the scene and compare it with the scaled cloud. With the baseline read to ±0.5 mm on a 120 mm rig (0.4%) and clean matches, expect agreement within a few percent; a bigger gap indicts the match set, not the ruler.
3. Recompute one depth as f_x · B / d from the rectified pair, as in the worked example, and compare it with the scaled Z.
4. Distrust a wild factor: s near 100 or 0.01 on a bench rig means normalized and unnormalized extrinsics got mixed.
5. Check match spread before trusting any of it. Matches bunched on one plane leave E underdetermined, so demand points spread across the image and across depth.

The decision, case by case

• Cloud already triangulated and millimetres needed today: measure the baseline to ±0.5 mm, compute s = B_true / B_rec, multiply every point and both centers, confirm one scene length. Minutes of work, single-digit-percent accuracy with good matches.
• Pipeline still being written: stereo-calibrate the pair with a printed target and triangulate from the calibrated extrinsics. Millimetres come out directly and scale ambiguity never enters.
• A target can never enter the scene: scale the essential-matrix cloud with the measured baseline, or take the surveyed-points calibration route if you can hand-measure a few scene coordinates.
• Specifying hardware anyway: an integrated stereo 3D camera, or a synchronized calibrated pair on a fixed rail — both return metric depth by construction.

To spec the pair, start at the camera configurator or the area scan cameras category. If a build question survives the arithmetic, contact the applications team with your intrinsics and the reconstructed camera centers — the two inputs needed to check s.