3D camera unit vector inconsistency after restart

A per-pixel viewing-direction map should be repeatable when calibration, pixel geometry and coordinate conventions remain unchanged. If the map changes after an acquisition process restarts, investigate the data path before adjusting the optics or replacing hardware. A restart can expose an unsuccessful acquisition, borrowed image memory, a different calibration selection or a changed coordinate transform.

This troubleshooting procedure helps you distinguish those cases from a camera-side calculation problem. It also explains why changing a frame timeout can alter the failure rate without identifying the underlying fault.

Decide what actually changed

Separate three operations: reconnecting a subscriber, recreating the acquisition process and restarting the camera. They exercise different initialization paths. Receiving identical data after repeated subscriptions does not prove that acquisition initialization is repeatable; the publisher might simply retain the same map.

Begin with the following investigation order. It prioritizes inexpensive checks and clear evidence, not a claimed ranking of failure rates.

Use the camera intrinsic parameter conversion guide when the changed map belongs to a resized, cropped or different image stream. An intrinsic matrix for one image geometry cannot automatically describe another.

Build a restart test that preserves evidence

Use a stationary setup and one acquisition client initially. Disable application-level coordinate changes, where supported, or record them explicitly. Do not change several settings between runs.

1. Record the device identity, firmware version, acquisition library version and application build before starting acquisition.
2. Save the active operating mode, output dimensions, calibration identifier and requested data components.
3. Start acquisition using the documented initialization sequence. Wait for a successful, valid result rather than merely waiting a fixed delay.
4. Copy the unit-vector array into application-owned storage while the source buffer remains valid. Copy its associated metadata as well.
5. Save the array without reducing its numeric precision. Record dimensions, data type, channel order, row stride, frame identifier and timestamps.
6. Stop acquisition cleanly, release its resources and restart only the acquisition process. Repeat the same capture procedure.
7. Test a camera restart separately. Label those results so they cannot be confused with process-only restarts.

For an initial investigation, collect 20 process restarts under one configuration. This is a practical starting sample, not a reliability qualification. If the fault appears once per several hundred starts, extend the campaign accordingly.

Compare the first valid map with a later valid map during each run. This separates an initialization-only problem from a map that changes continuously. Where the documentation permits it, also record whether the map is generated once per mode or delivered with each frame.

Reject invalid frames and protect buffer ownership

A frame-wait call returning unsuccessfully does not authorize reading new image contents. The destination may still contain an earlier frame, an incomplete result or no valid data. Check the documented return status and component-validity indicators before accessing the map.

Verify that all expected components belong to the same acquisition result. A fresh depth image combined with an old direction map can look plausible while producing incorrect coordinates. The per-frame metadata verification guide explains the broader practice of associating settings and metadata with their actual frames.

Buffer lifetime is the next check. An image object may expose a view into acquisition-managed storage rather than an independent array. Assigning that view to another variable may copy only a pointer or header.

Use this controlled test:

• Acquire a valid map and create a documented deep copy.
• Calculate and save a checksum of the copy and the original view.
• Acquire another result or release the source acquisition object.
• Recheck the owned copy. Inspect the original view only while its documented lifetime still permits access.

If the owned copy stays unchanged but the valid original view changes on reuse, the application was observing reused storage. Never deliberately dereference a view after its lifetime ends.

The image buffer ownership guide covers the related ownership and disposal decisions. The same principle applies across languages: retaining an object reference does not necessarily retain the underlying image contents.

Match calibration identity and pixel layout

Compare more than the displayed camera settings. Record the effective settings reported after initialization, because a saved configuration may not be the configuration actually selected.

Check these items together:

• Device identity and calibration identifier, when exposed.
• Operating mode and selected image component.
• Width, height, crop origin, binning and decimation.
• Distortion model and whether coordinates refer to distorted or rectified pixels.
• Element type, endianness, channel order and row stride.
• Coordinate-frame identifier and any enabled transform.

A map with the correct dimensions can still belong to the wrong geometry. For example, resizing an image preserves its visual content while changing which original ray corresponds to each output pixel. Cropping also shifts the relationship between image coordinates and the calibrated principal point.

For a conventional pinhole model, the direction calculation can be expressed as follows:

Here, K is the intrinsic matrix for the relevant image geometry. Lens distortion must be handled according to the calibration model before treating this as the physical viewing ray. The formula does not replace a manufacturer's documented model for a different camera geometry.

If your pipeline performs rectification, check its output camera matrix and pixel mapping separately. The live stereo rectification guide explains why rectified image coordinates belong to a particular output geometry.

Compare directions with geometric metrics

First check that valid vectors contain finite values and have the expected normalization. Apply the documented invalid-pixel mask; do not normalize zero vectors or silently treat invalid entries as valid directions.

For each pixel valid in both maps, normalize the vectors before computing their angular separation:

Clamping avoids a domain error when floating-point rounding puts the dot product slightly outside its mathematical range. Calculate in sufficient precision, particularly when comparing very small angles.

Report the valid-pixel fraction, median angle, 95th-percentile angle, maximum angle and fraction above your acceptance threshold. Also inspect a spatial difference map. A single maximum can be dominated by an invalid edge pixel, while a median can hide a localized defect.

Choose the angular threshold from the application error budget rather than an arbitrary decimal-place comparison. For a direction change at a known range, the transverse displacement is:

For example, at a range of 2,000 mm, a direction change of 0.05 degrees gives:

If the direction-map allocation is 1 mm at that range, the approximate angular allowance is:

These are illustrative allocations, not camera specifications. Depth error, calibration uncertainty and mechanical movement consume additional parts of the total measurement budget.

Separate coordinate conventions from changed rays

A unit-length check cannot detect an axis swap, sign change or rigid rotation. Those operations can preserve vector length while changing every component.

Check the documented forward axis, handedness, axis order and frame identifier. Confirm whether the output uses the optical frame, a device frame or an application-transformed frame. Also check row and column indexing; comparing a pixel with its transposed location creates a different problem from a coordinate-axis swap.

If a single rotation aligns both maps closely, investigate a changed coordinate transform. Treat this as diagnostic evidence, not permission to insert a corrective rotation without understanding its source. The extrinsic calibration accuracy guide helps distinguish camera-to-world transforms from internal image geometry.

Depth semantics matter when validating rays against reconstructed points:

Applying the radial formula to axial depth causes errors that generally increase away from the optical axis. It can make a correct direction map appear suspect.

Use the fixed image-point depth measurement guide when checking a known pixel. Select a valid location and verify its coordinate meaning rather than assuming the image midpoint lies exactly on the optical axis.

Treat timeouts as test variables, not diagnoses

A timeout controls how long a particular operation waits under its documented API contract. It does not, by itself, establish frame validity, buffer ownership or calibration readiness.

Changing the timeout can change thread scheduling, buffer reuse and which startup result the application consumes. Consequently, fewer failures with one timeout value does not prove that the timeout was the root cause.

Repeat the same capture protocol with several documented, operationally reasonable timeout settings. Log unsuccessful waits separately from valid-but-different maps. Do not combine both outcomes into a single failure percentage.

The intermittent acquisition failure guide is the better path when frames are missing or incomplete rather than geometrically inconsistent.

If valid, owned maps still differ with matching calibration and coordinate conventions, prepare a manufacturer support package. Include saved arrays, validity masks, metadata, configuration exports, version information and a minimal reproducer. Follow the manufacturer's instructions for sharing device-specific diagnostic files.

Test firmware or acquisition-library updates only through the manufacturer or its authorized distributors. Check compatibility, release notes, configuration backup requirements and documented recovery procedures first. After updating, rerun the original test unchanged before testing additional configuration changes.

Check the data contract before choosing hardware

Replacement hardware is not the first remedy for an ownership or coordinate-frame defect. If you are selecting a new system, request sample data and documentation describing direction maps, calibration identity, invalid values and startup behavior.

The 3D sensors category and camera finder provide starting points for matching the sensing method to the task.

The MV-DPS200P-02 is a projected structured-light 3D camera.

The MV-DT01WMU is a ToF sensing camera.

These type labels do not establish whether either product exposes a unit-vector map or a particular software interface. Confirm the required outputs, coordinate conventions and calibration access before selecting a part. Do not assume that behavior observed with another device applies to either listing.

Verify the fix against a written acceptance test

Use the same application-owned captures and angular metrics before and after each change. Keep software corrections, configuration changes and firmware updates separate so the result remains attributable.

Before releasing the application, confirm that:

• Unsuccessful acquisitions cannot enter the geometry-processing path.
• Every retained map has a documented owner and lifetime.
• Device identity, calibration identity and image geometry match expectations.
• Axis conventions and depth semantics are explicit.
• Valid-pixel coverage and angular differences meet the written limits.
• Process restarts and camera restarts are tested separately.
• Startup delays, reconnects and repeated acquisitions do not corrupt saved maps.

Zero failures in a short run do not establish a rare-event reliability target. Choose a longer test campaign based on the observed failure frequency and the consequences of incorrect geometry.

If the map is stable but reconstructed measurements still move, continue with depth repeatability, synchronization, mounting and extrinsic calibration checks rather than forcing a ray-map correction.

For a new system, start with the camera configuration step. For an existing installation, contact applications support with the comparison metrics and configuration record so the next investigation targets the evidence.