Network depth camera alignment: check calibration metadata

Depth and color can look normal individually while their overlay fails only through a network bridge. For integrators, that pattern calls for a calibration-metadata comparison before changing lenses, moving the camera or recalibrating the assembly. A working direct connection provides a useful reference, provided both paths use identical stream profiles.

The goal is to find where the transported camera description diverges from that reference. You will check intrinsics, extrinsics, depth units and numeric precision, then verify a supported correction against stationary targets. Successful discovery and image delivery do not establish that the receiving application has everything needed for geometric alignment.

Start with the direct connection as your reference

First establish that the direct path produces acceptable registration using the same device, mounting and scene. Save the actual active profiles, not just the profiles requested by your application. A bridge may negotiate a different resolution, format or processing mode.

Keep these conditions unchanged between captures:

• Device identity and calibration revision, where exposed.
• Depth and color resolutions, frame rates and pixel formats.
• Cropping, binning, decimation and rectification settings.
• Depth filtering and alignment configuration.
• Target position, mounting and illumination.

Use a stationary scene initially. Motion introduces timing errors that can resemble geometric errors. The depth and color synchronization guide explains how to separate capture timestamps from delivery timing.

Confirm that the reference itself uses the correct image geometry. The raw versus rectified registration guide is the closest starting point if the direct overlay is also wrong.

Compare metadata before adjusting alignment

Registration normally needs depth-camera intrinsics, color-camera intrinsics, the transform between their coordinate frames and the conversion from stored depth values to physical distance. Distortion models and coefficients also matter when either image is unrectified.

Record those values through the documented API immediately before alignment. Compare what the receiving process actually uses, rather than what a configuration file was intended to supply.

These are investigation priorities, not diagnoses by appearance alone. An incorrect principal point can also cause an offset, while distortion errors can grow toward the image edges.

For a structured list of calibration fields to preserve, use the raw stereo calibration metadata guide.

Check intrinsics and their numeric representation

Intrinsics describe the relationship between image coordinates and rays leaving a camera. For a pinhole model, the main values are focal lengths and principal-point coordinates expressed in pixels. Image dimensions and the distortion model complete the interpretation.

Compare the following at both ends of the bridge:

1. Active image width and height.
2. Horizontal and vertical focal lengths.
3. Horizontal and vertical principal-point coordinates.
4. Distortion model and coefficient ordering.
5. Raw or rectified image status.
6. Profile identity associated with each calibration record.

Do not interpret rounded console output as proof that stored values were truncated. First increase logging precision. Then compare the numeric values returned by the receiving API against those captured before transport.

For a binary format, check the declared field types and schema. For a text format, check significant-digit formatting, decimal parsing and any intermediate integer conversion. Locale-dependent decimal separators are another possible failure point.

Precision requirements depend on the error budget. As an illustration, truncating a focal length from 600.75 pixels to 600 pixels changes its projection contribution by 0.375 pixels when X/Z equals 0.5. Truncating a principal point from 640.4 pixels to 640 pixels adds another 0.4-pixel contribution in the same direction. The combined shift is 0.775 pixels in that example.

Preserve the numeric representation required by the documented interface rather than choosing an arbitrary number of decimal places. If the image grid changes, follow the intrinsics after binning guide before comparing calibration values.

Verify extrinsics, direction and units

Extrinsics transform a point from one camera coordinate frame into another. A valid rotation matrix is not enough: the transform must belong to the correct source and destination pair.

Check that the bridge preserves:

• Source and destination stream identities.
• Transform direction, such as depth coordinates into color coordinates.
• Rotation layout and matrix multiplication convention.
• Translation units and axis conventions.
• Calibration association with the active device and profile.

An identity transform can be legitimate for streams sharing the same coordinate frame. It is suspicious when separate viewpoints require a nonzero transform and the direct path reports one.

Do not reverse a transform by negating translation alone. For a rigid transform, the inverse rotation is its transpose, and the inverse translation is the negative translation rotated by that transpose.

A cache must distinguish the relevant source and destination identities. Otherwise, a transform from another stream pair or profile can appear numerically plausible while being geometrically wrong.

Also compare units explicitly. A translation expressed in millimeters cannot be added directly to a point expressed in meters. Convert both into one documented coordinate system before projection.

Validate depth units with a worked example

Stored depth values are not necessarily distances in millimeters. The documented depth convention and scale determine their physical meaning. Some systems report optical-axis depth; others may report range along a viewing ray. Confirm which interpretation your alignment routine expects.

For the following example, assume optical-axis depth, rectified images and a pinhole model. Distortion is therefore omitted. All values are hypothetical application numbers, not specifications for a catalog product.

Here, d is the stored depth sample and s is the distance per count. Suppose d equals 2000 counts and s equals 0.001 meters per count:

• Z = 2000 × 0.001 = 2 meters.
• Let u_depth = 700 pixels, cx_depth = 640 pixels and fx_depth = 600 pixels.
• X = (700 − 640) × 2 / 600 = 0.20 meters.

Assume equal intrinsics for both views, no rotation and a hypothetical positive horizontal translation of 0.05 meters. The transformed horizontal coordinate is 0.25 meters, giving:

• u_color = 600 × 0.25 / 2 + 640 = 715 pixels.

Now suppose the receiver incorrectly uses 0.01 meters per count. It reconstructs Z = 20 meters and X = 2 meters. With the same physical translation, the projected coordinate becomes:

• u_color = 600 × 2.05 / 20 + 640 = 701.5 pixels.
• Projection error = 715 − 701.5 = 13.5 pixels.

This illustrates why an image can remain recognizable while its cross-camera alignment fails. Translation couples physical distance to image displacement. With coincident viewpoints and pure rotation, a uniform depth-scale error may not produce the same visible reprojection error.

The depth unit and precision settings guide explains why storage increments, reported precision and measurement accuracy are different quantities.

Locate the first transport boundary that changes values

Use the smallest supported capture configuration that reproduces the fault. Then inspect the metadata at successive boundaries:

1. Camera-side acquisition: record values returned by the capture API.
2. Server-side serialization: record field names, types, units and profile identifiers before transmission.
3. Client-side parsing: record decoded values before they enter a cache or stream wrapper.
4. Alignment input: record the final metadata attached to the frames being processed.

The first mismatch narrows the investigation. Correct server values followed by incorrect parsed values point toward serialization or decoding. Correct parsed values followed by incorrect alignment inputs suggest caching, profile association or default substitution.

Check whether depth units belong to device configuration, stream metadata or individual frames in the documented interface. A receiver must not silently replace an unavailable value with an unrelated default.

Run the comparison after a clean connection, a reconnect and a supported profile change. Persistent calibration and session-specific frame metadata may follow different initialization paths.

If software compatibility is uncertain, consult the firmware and SDK alignment troubleshooting guide. Use only manufacturer- or distributor-provided packages and documented combinations. Avoid combining unrelated server and client builds merely because image streaming starts.

Apply supported fixes, then measure registration

Prefer a documented bridge update, compatible server-client pair or corrected configuration over application-side substitution of calibration values. Change one component at a time, retaining the reference capture and a rollback route.

Do not treat copied calibration constants as a durable repair. They can become incorrect after a device replacement, resolution change, calibration update or processing-mode change. If documented calibration import is supported, preserve device identity, profile association and units through that workflow.

After correction, repeat both metadata comparisons and image measurements:

• Capture a stationary target at several positions across the shared field of view.
• Repeat at near, middle and far distances within the documented working range.
• Measure corresponding features visible in both modalities.
• Record median error, a high-percentile error and invalid-depth coverage separately.
• Repeat after reconnecting and restarting the application.

For example, collect 20 frames at each of three distances and five image positions. That produces 20 × 3 × 5 = 300 frame-position observations. Compare bridged measurements with the direct baseline using the same method.

An acceptance limit of 1 pixel could suit a particular application, but it is not a universal camera specification. Define the limit from the task and baseline performance before evaluating a correction.

Know when metadata is not the cause

If intrinsics, extrinsics, units and active profiles agree, investigate processing order and timing next. Aligning already registered depth a second time can be wrong. So can combining rectified images with calibration intended for raw images.

Moving objects deserve a separate test. Network buffering can change which frames are paired even when each frame is geometrically valid. A stationary target that aligns correctly while moving features separate points toward synchronization rather than a fixed calibration error.

Delivery faults also need their own evidence. Missing frames, decoding errors and incomplete buffers are not repaired by recalibration. Use the vision camera network troubleshooting guide when the transport itself is unreliable.

For layered surfaces, ordinary viewpoint occlusion may explain the apparent mismatch. The depth-to-color registration guide helps distinguish those limits from incorrect projection.

Check compatibility before ordering replacement hardware

A bridge-only failure does not establish that the camera needs replacement. First ask for documented support covering calibration access, depth units, active profiles, synchronization and the intended transport architecture.

If a new sensor is genuinely required, compare 3D sensors through the camera finder. The MV-DLS300P-04 is listed as a laser galvanometer stereo camera.

The MV-DPS200P-02 is listed as a projected structured-light camera.

These catalog descriptions do not establish network-bridge compatibility, color-stream availability or access to the metadata your application requires. Confirm those capabilities before selection.

If the design uses a separate color camera, examine area scan cameras and the camera configuration starting point. Separate cameras require their own calibration and synchronization strategy; adding one is not a shortcut around corrupted metadata.

Before ordering or approving a software change, confirm:

• The direct and bridged paths use matching active profiles.
• Calibration values retain their required numeric precision.
• Transform direction and translation units are explicit.
• Depth units reach the alignment routine correctly.
• Reconnection does not restore stale metadata.
• Stationary registration returns to the direct-path baseline.
• Motion performance passes a separate synchronization test.

For help selecting a compatible capture arrangement, bring the profile comparison and measured registration errors to applications support.