Raw stereo image calibration: verify streams and metadata

A stereo camera can deliver useful depth while hiding the original images used to calculate it. That becomes a problem when you need your own correspondence algorithm, calibration checks or geometric processing. A monochrome stream alone does not tell you whether its pixels have already been rectified.

Before choosing hardware or writing an acquisition pipeline, establish two things: which image geometry the device exposes, and whether its calibration describes that exact geometry. The following checks help you separate documented raw access from assumptions based on image appearance, bit depth or stream names.

1. Decide what raw must mean for your application

For stereo geometry, an unrectified image normally means an image that has not undergone lens-distortion correction and stereo rectification. It does not necessarily mean untouched sensor samples. Exposure correction, defective-pixel correction or other processing may already have occurred.

Write down which processing stages your application can accept:

• Photometric processing: changes pixel intensity, potentially affecting correspondence or radiometric analysis.
• Lens undistortion: resamples the image to remove a specified lens-distortion model.
• Stereo rectification: maps both images into virtual camera geometries with compatible epipolar lines.
• Cropping or resizing: changes image coordinates and therefore the applicable intrinsics.
• Depth registration: projects depth into another camera's image geometry rather than exposing an original stereo image.

An unrectified stereo pair needs intrinsics and distortion information for each camera, plus the relative camera pose. A rectified pair instead needs the geometry of the rectified images, often including rectification rotations and projection matrices.

The distinction is explained further in interpreting distortion coefficients for an unrectified image. Establish the image domain before interpreting any coefficient values.

2. Inventory the actual stream profiles

Build a profile inventory using the manufacturer's documented viewer, SDK enumeration functions or supported diagnostic tools. Get software and documentation from the manufacturer or an authorized distributor.

For each left and right stream, record:

• Physical imager identity and stream identifier.
• Width, height, pixel format and requested frame rate.
• Whether the profile is advertised and whether it actually starts.
• Any processing blocks enabled after acquisition.
• Whether the documentation identifies the output as distorted, undistorted or rectified.
• Whether calibration can be queried for the active profile.

Do not infer geometry from the payload representation. An 8-bit grayscale image can be distorted or rectified; the same is true of a 16-bit image. Bit depth describes sample representation, not the geometric operations previously applied.

For the format distinction, use grayscale versus color modes in stereo infrared streams. If a requested profile cannot start, resolve camera pipeline format negotiation before evaluating its calibration.

3. Retrieve calibration as a matched bundle

A list of calibration values is not enough. Each value must be associated with the image profile and coordinate system it describes.

For each camera, obtain the intrinsic matrix, distortion model identifier and distortion coefficients. For the pair, obtain the relative rotation and translation. Record the calibration source and the conditions under which the documentation says it applies.

Query calibration through the active stream objects when the documented API supports that approach. Otherwise, require documentation connecting the returned calibration to the selected profiles.

Missing values in a command-line report do not automatically mean calibration is unavailable. The report may expose fewer fields than the supported API. Conversely, a successful API call does not prove the returned values describe the stream you selected.

4. Identify coordinate and distortion conventions

For a pinhole camera, the intrinsic matrix is commonly written as follows. The example assumes zero skew, which is common but should still be checked against the documented model.

Here, D represents a forward distortion mapping from ideal normalized coordinates to distorted normalized coordinates. Not every calibration interface uses that direction. Some models instead describe the inverse operation, and their coefficients cannot simply be inserted into a forward-distortion routine.

Confirm the coefficient order, coordinate normalization and supported model. The article on inverse distortion coefficients and warped rectified depth covers the consequences of confusing these mappings.

Extrinsics also need an explicit direction:

This expression transforms a point from the left camera frame into the right camera frame. Translation uses the same length unit as the point coordinates. Reversing the transformation requires both a transposed rotation and a transformed translation:

Negating translation alone is only sufficient in special cases. Record axis directions and matrix storage conventions rather than relying on variable names.

5. Work through a crop and transform example

Consider a synthetic calibration for an unrectified image measuring 1280 × 720 pixels. Its focal lengths are 800 px and 810 px, with a principal point at pixel coordinates (640, 360).

Suppose acquisition returns a crop beginning at full-image coordinates (160, 90), with no resizing or other geometric processing. The crop measures 960 × 540 pixels.

The cropped image needs a shifted principal point:

For a pure crop under the same normalized-coordinate distortion model, the focal lengths and distortion coefficients remain unchanged. The principal point moves because the image origin moves.

Resizing introduces additional changes to focal lengths and principal point. Pixel-center conventions can add offsets, so use the documented mapping rather than applying an unexplained scale factor. For profile changes, consult rescaling calibration versus recalibrating after a resolution mismatch.

Now consider a separate synthetic stereo transform. Let the rotation be identity and the left-to-right translation be (-0.060, 0, 0) m. A point at (0.100, 0.020, 1.000) m in the left frame becomes:

These are illustrative application values, not specifications for any listed camera. They show why image origins, transform direction and units must travel with the calibration.

6. Verify metadata without using circular proof

Documentation establishes the intended stream geometry. Image tests help detect mismatches, but no single visual test proves that an image is raw.

A straight grid that looks straight may indicate correction, or simply low lens distortion. Zero-valued distortion coefficients may describe an idealized output camera rather than the physical lens. Likewise, horizontal correspondences alone do not establish the complete processing history.

Use several independent checks:

1. Capture a static calibration target at different image positions and tilts.
2. Project known target points using the claimed image model and estimated target pose.
3. Inspect residuals across the center and edges, not only the overall average.
4. Apply your own undistortion and rectification to the claimed unrectified pair.
5. Measure vertical correspondence error in the resulting rectified images.

For an internal trial, you might choose a median vertical mismatch below 0.5 px and a 95th-percentile mismatch below 1 px. These are example acceptance targets, not universal specifications. Set limits from your depth-error budget and target-detection uncertainty.

Do not demand horizontal correspondences in the original distorted images. Lens distortion can curve their epipolar loci. The procedure for verifying recorded stereo frames are rectified belongs after you have established which image domain you are evaluating.

7. Compare acquisition routes before choosing hardware

Choose hardware around documented access, not around the presence of a depth output. A device can be suitable for depth measurement without exposing the intermediate images your algorithm needs.

Compare 3D sensors by the measurement approach first, then use the camera finder to narrow the hardware options. Request interface documentation before treating any candidate as suitable for raw-image acquisition.

The MV-DLS300P-04 listing identifies a laser galvanometer stereo camera. The supplied catalog context does not establish unrectified-image access or calibration API coverage.

The MV-DPS200P-02 listing identifies a projected structured-light camera. Its suitability for a custom stereo pipeline also requires confirmation of the accessible streams and metadata.

If separate cameras better fit your workflow, explore area scan cameras. That route gives you component choices, but does not automatically provide synchronized exposure or an assembled stereo calibration.

For that approach, plan stereo calibration from paired checkerboard images and confirm the acquisition method before committing to optics or mounting geometry.

8. Build a reproducible acquisition record

Save calibration beside the image data, not in an unrelated project directory. A recording should remain interpretable after the camera configuration changes.

Use a session manifest containing:

• Device identity, firmware version and SDK version.
• Active left and right profiles, including image dimensions and pixel representation.
• Processing stages enabled inside the device and application.
• Intrinsics, distortion models, coefficients and calibration identifiers when available.
• Extrinsic source and destination frames, translation units and matrix conventions.
• Crop, resize, rotation or rectification operations applied after acquisition.
• Timestamp definitions, frame counters and pairing rules.

Store frame-specific timing with each pair. Equal frame rates do not prove simultaneous exposure, and host arrival time includes transport and buffering delays.

If separate cameras require an external trigger, review hardware-triggered image acquisition. Confirm documented trigger behavior for the selected hardware rather than assuming a shared trigger makes all exposure timing identical.

Finally, reopen a saved session without querying the connected camera. If the recording cannot be interpreted independently, its metadata package is incomplete.

9. Know when the usual answer is wrong

Obtaining unrectified images is not always the best route. If your application only needs disparity from a documented rectified pair, using that pair may avoid unnecessary resampling and calibration work.

Similarly, a raw profile with no matching calibration may be less useful than a documented corrected profile. Do not transplant coefficients from another stream merely because its dimensions match.

If supported calibration access is unavailable, the legitimate options are to request manufacturer clarification, use a documented alternative profile, calibrate the accessible image pair yourself, or choose different hardware. Independent calibration still requires stable imaging settings, reliable pairing and a suitable target.

For offline processing, rectification from stored calibration data explains the next stage once the image model and metadata are established.

10. Check these points before ordering

Use this acceptance checklist with the proposed acquisition configuration:

• Exact profiles documented: both cameras expose the required dimensions, format and acquisition rate.
• Geometry identified: the documentation states whether each delivered image is distorted, undistorted or rectified.
• Calibration matched: each intrinsic model belongs to the delivered profile, including any crop or scaling.
• Transform complete: relative rotation, translation, units and direction are available.
• Timing understood: exposure synchronization and frame-pair association meet the motion requirement.
• Sample recording supplied: representative images and metadata can be evaluated together.
• Independent verification passed: target reprojection and rectified correspondence checks meet your chosen tolerances.
• Session reproducible: saved frames retain enough information for processing without the original device attached.

Start the camera configuration workflow when you know which acquisition route fits. If raw access or calibration coverage remains unclear, send the required profiles and metadata checklist through applications support before selecting a camera.