How to stitch point clouds from multiple cameras

Several cameras can reveal surfaces that one viewpoint cannot see, but collecting more points does not automatically create a coherent scan. A useful combined cloud needs matching capture times, calibrated camera poses, consistent coordinate conventions, and independent accuracy checks. This workflow is for fixed multi-view inspection stations and scene-capture rigs. You will define the output, assemble synchronized capture sets, transform each cloud, and verify the geometry before choosing a fusion method.

Define the output and acceptance limits

Start with the measurement or detection task, not the camera count. Decide which surfaces must be visible, what motion occurs during capture, and how much disagreement the downstream application can tolerate.

Three operations are often called stitching, although they solve different problems:

A transformed, concatenated cloud may be sufficient for object detection. Surface inspection may need duplicate removal, confidence filtering, and a carefully selected fusion resolution. A mesh is another processing stage, not an automatic consequence of registration.

Write an acceptance statement before purchasing hardware. For example: overlapping observations of a reference surface must agree within 3 mm across the inspection volume. That is an illustrative application requirement, not a camera specification. Define separately whether the limit applies to average, percentile, or maximum error.

For repeated observations from a single moving viewpoint, the depth-frame merging workflow addresses a related but different capture problem.

Prepare cameras, mounts, and software interfaces

Select 3D sensors by the required measurement volume, surface behavior, capture method, and documented multi-device operation. Use the camera finder to narrow candidates, then verify the integration requirements against manufacturer documentation.

MV-DLS300P-04 and MV-DPS200P-02 are catalog candidates for a structured-light evaluation. Their catalog descriptions do not establish hardware synchronization support, simultaneous-projector compatibility, or a ready-made stitching interface. Confirm those capabilities before selecting either for a multi-camera rig.

Use camera stands or a fixed fixture that preserves the calibrated relationships. LS-MNT-0040 is a camera-stand option for evaluating the mounting arrangement. Check payload, mounting interfaces, adjustment locks, and cable forces for the actual assembly.

The accessories finder helps locate mounting components, but a stand listing is not evidence of a particular calibration stability.

A separate color view may help with texture or identification. MV-CU020-90GC is a color area scan camera with a global shutter. Using it alongside depth capture requires its own calibration and timing assessment; this is not a claim of direct integration with the sensors above.

Before implementation, confirm that your acquisition interface exposes depth calibration, frame timestamps, frame identifiers, and sufficient raw data. Obtain software and documentation only from the manufacturer or its authorized distributors.

Capture and register the views in order

1. Choose a reference frame and record conventions.

   Select a fixture, calibration target, or camera as the reference. Record axis directions, handedness, length units, and the meaning of each transform. A transform named only “camera pose” is ambiguous: state whether it maps camera coordinates into reference coordinates or the reverse.

2. Arrange coverage around the required surfaces.

   Include common visible geometry between neighboring views, or use a calibrated target arrangement that connects them. Broad, well-distributed overlap is more useful than a narrow strip. A single flat surface may leave some registration motions poorly constrained. Check near and far boundaries using the working-distance and positioning guide.

3. Establish capture timing before measuring moving objects.

   Use documented synchronization facilities where available. Determine whether timestamps describe exposure, acquisition completion, or host receipt. Receipt time includes transport and scheduling delays and should not be treated as exposure time without validation. The shared-event synchronization workflow covers capture timing in more detail.

   Active illumination can also create interference between devices. Compare depth quality with each emitter operating alone and with the planned group operating together. If documented operating modes require sequential exposures, budget the resulting time separation against object motion.

4. Calibrate relative camera poses using static observations.

   Use the documented calibration method and collect observations that constrain the full measurement volume. Include different target orientations and positions rather than repeatedly capturing one central pose. Keep an independent set of observations out of the calibration fit.

   Follow the shared-coordinate-frame calibration workflow for the pose-estimation task. Keep those calibration records separate from production captures so that a changed configuration can be traced.

5. Build matched capture sets and transform valid points.

   Group frames by the synchronization event or a validated timestamp window. Reject incomplete sets, or explicitly mark them as partial. Do not silently substitute an older frame when a camera misses a capture.

   Convert invalid-depth codes into excluded observations according to the device documentation. Then apply the correct rigid transform to each valid point. Check optical-frame versus camera-frame transforms before adding any axis conversion.

6. Validate registration before enabling fusion.

   Display each transformed cloud in a different color. Inspect common surfaces at the center, edges, near limit, and far limit. Compare independent reference measurements rather than judging only a visually pleasing overlay.

   Refine alignment only after the initial calibration is credible. Local matching can settle on the wrong solution when overlap is small or geometry repeats. If refinement rejects inputs, use the cloud and pose compatibility checks before changing its search settings.

Calculate the timing and pose-error budget

A rigid transform expresses a point from camera coordinates in the chosen reference frame:

Here, R_reference_camera is the rotation matrix and t_reference_camera is the translation expressed in reference-frame units. Translation and point coordinates must use the same length unit. Normals receive the rotation only, not the translation.

For several cameras, store a separate transform into the same reference frame. If transforms are chained through intermediate frames, document the multiplication order. Applying a valid matrix in the wrong direction still produces plausible-looking numbers.

Two simple estimates expose common limits:

Suppose a surface moves at 0.4 m/s and two exposures are separated by 10 ms:

That timing mismatch alone exceeds the illustrative 3 mm acceptance limit. Increasing cloud density cannot correct it. A 1 mm motion-error allocation would require separation no greater than 0.001 / 0.4 = 0.0025 s, or 2.5 ms, under this constant-speed estimate.

Now consider a rotation error of 0.1° at a distance of 0.8 m:

These are first-order estimates, not a complete uncertainty model. Depth noise, translation error, calibration uncertainty, motion during exposure, and surface-dependent bias also contribute. Set separate budgets and validate the combined behavior experimentally.

Fuse clouds without hiding registration problems

Keep the original clouds and transforms before filtering. This makes it possible to distinguish a capture defect from a fusion artifact.

Begin with transformed concatenation. Inspect overlap thickness, then introduce filtering in small, reversible stages:

• Remove documented invalid measurements and obvious isolated outliers.
• Crop to the physical measurement volume in the reference frame.
• Check overlap disagreement before averaging nearby observations.
• Downsample only after defining the smallest feature that must remain visible.
• Apply surface fusion or meshing only if the downstream task requires it.

A voxel grid can reduce duplicate points, but it can also blur nearby surfaces. As an illustrative starting point, a 2 mm voxel dimension is unsuitable for preserving a 1 mm-wide feature reliably. Evaluate smaller cells against noise, memory, and processing time rather than treating the voxel dimension as measurement accuracy.

Repeated observations are not necessarily independent. Averaging them does not remove a shared calibration bias or a systematic depth error. Weighting should reflect validated measurement quality, not merely the number of points a camera contributes.

When ordinary fusion is the wrong answer, retain separate surfaces or exclude unreliable regions. Transparent parts, reflective edges, and moving flexible surfaces may not support a stable single-surface estimate.

Export a cloud or render an image deliberately

A combined point cloud is not automatically a conventional image. After fusion, points from different views generally no longer belong to one rectangular pixel grid.

For downstream 3D processing, export coordinates with an explicit reference-frame identifier, capture-time meaning, units, and validity policy. Preserve camera identity when traceability matters. If your middleware expects a structured point message, match field names, numeric types, byte layout, and ownership rules. Do not assume that an acquisition SDK includes the required adapter.

If the consumer needs an image, define a virtual camera or projection plane. Project the combined geometry into that view and use depth ordering to resolve occlusion. Leave unobserved pixels marked invalid rather than implying that every gap contains a measured surface.

Color requires its own mapping. Consult depth-to-color registration before assigning texture to points. Raw and rectified images cannot be interchanged without accounting for their calibration.

For a separate texture view spanning 800 mm, suppose a 2 mm feature needs 4 pixels across:

This is a two-dimensional sampling estimate, not a depth-accuracy prediction. Compare area scan cameras against the texture requirement separately from the depth requirement.

Verify seams with independent measurements

Use a fixed reference object with independently known dimensions or positions. Check different depths and orientations, including regions not used to estimate calibration.

Report more than one statistic. Median disagreement describes typical behavior; a high percentile exposes wider spread. Also report valid coverage and signed offsets on suitable reference surfaces. Nearest-neighbor distance alone can look favorable when a surface slides along a plane.

The single-sensor blank-frame checks help isolate an intermittent missing contribution. Do not patch that symptom by reusing stale geometry.

Repeat validation after startup, a controlled power cycle, and any mounting adjustment. The independent extrinsic-calibration checks explain why a small image reprojection residual is not sufficient proof of spatial accuracy.

Check the integration before ordering

Use this short acceptance checklist with your equipment and software requirements:

• Coverage: required surfaces are visible, with a calibration connection between views.
• Timing: capture-time separation is measured and compatible with permitted motion.
• Calibration: transform direction, units, target coverage, and held-out checks are documented.
• Mounting: interfaces, cable loading, and adjustment retention suit the assembly.
• Software: raw data access, timestamps, export fields, and middleware adaptation are confirmed.
• Processing: filtering and fusion preserve the required features without concealing seams.
• Recovery: missing frames and changed configurations produce explicit status, not silent substitutions.

Start the camera configuration workflow once these requirements are defined. For a multi-view review, send your measurement volume, surface samples, motion limits, required accuracy, and intended output format through application support.