Distortion model unknown: garbage depth camera coefficients explained

Your depth camera publishes intrinsics with distortion_model set to 'UNKNOWN' and a D array like [4.6e-41, -1.127e+30, 4.6e-41, 0, 0], while an identical unit on the next port reports 'None' and five zeros. This guide is for anyone writing ingestion code that parses a camera_info block, a GenICam intrinsics node, or a vendor SDK struct. You will see what those numbers are (uninitialized driver memory, not optics), the arithmetic test that flags them in one line, what they do to undistortion, PnP and point clouds when passed on, and how to sanitize the block once, at the source.

The answer: unwritten memory, not optics

Read the D array as memory nobody wrote and the UNKNOWN string as the middleware admitting it. A depth image from a time-of-flight, solid-state LiDAR or structured-light camera is computed by an onboard depth engine and published on a rectified pixel grid, so the honest intrinsics block reads model 'None' with D = [0, 0, 0, 0, 0] — exactly what the healthy unit reported. The faulty unit published a raw buffer: -1.127e+30 is a sentinel-scale float, and 4.6e-41 is a denormal, the signature of stale bytes reinterpreted as floating point. The driver in that capture predated the fix; later SDK releases initialize the coefficients and report 'None'. Your decision holds regardless of version: if the model string is UNKNOWN or empty, or any coefficient fails a finite-and-inside-±1 test, replace the whole array with zeros before anything downstream reads it.

Why the depth stream has no distortion to report

Distortion coefficients describe the perspective lens of an imaging camera: where its projection bends relative to an ideal pinhole. A depth stream is not that image. The sensor measures per-pixel range — by modulated infrared phase, by a projected pattern, or by a scanned line — and firmware maps the result onto an output grid using factory calibration, lens correction included. What leaves the sensor is already rectified, so any distortion the depth lens has physically is corrected before you see a pixel. The infrared receiver and the color camera sit behind real lenses with real coefficients, but their raw frames are not the depth stream you are parsing.

This covers time-of-flight cameras such as the MV-DT01WMU and projected structured-light cameras such as the MV-DPS200P-02 across the 3D sensors range, and it applies equally to stereo systems, where rectification is explicit. The color side of any of these systems is an area scan camera with a real lens, and that stream keeps its real coefficients.

Depth engines of this class produce the rectified depth grids discussed here:

One arithmetic test separates real coefficients from garbage

Two checks do the work: the model string must name a known model or none, and every coefficient must be finite with magnitude under 1.

The coefficients act on normalized coordinates, x_n = (u - cx) / fx, not on pixel offsets, so the frame corner of a wide lens sits near r² = 0.85. Healthy radial coefficients for machine-vision and consumer lenses fall within about ±0.3, and fisheye parameterizations stay inside ±1; a coefficient beyond ±1 would fold the image past its own diagonal. That makes ±1 a rejection fence with enormous margin: real values cluster far inside it, and the garbage sentinel sits 29 orders of magnitude outside.

The matrix K in the faulty capture was healthy, so K is not automatically suspect when D is garbage — but gate it anyway, because the same unwritten-buffer bug can land one field over.

Worked example: what -1.127e+30 does at the frame corner

Take the healthy intrinsics reported by the second unit: fx = 463.2 px, cx = 348.6 px, cy = 245.3 px. Pick a pixel 355 px right of and 235 px below the principal point — the far corner of the frame.

1. Normalize: x_n = 355 / 463.2 = 0.766, y_n = 235 / 463.2 = 0.507.
2. Radius: r² = 0.766² + 0.507² = 0.587 + 0.257 = 0.844.
3. Garbage scale: s = 1 + k1 × r² = 1 + (-1.127e+30 × 0.844) = -9.5e+29.
4. Pixel fate: 355 px × (-9.5e+29) = -3.4e+32 px. The undistorted corner lands thirty orders of magnitude past the sensor; every float after that step is noise.

The denormal tells the opposite story: 4.6e-41 × 0.844³ = 2.8e-41, a shift of zero for any purpose. For comparison, a genuinely wide lens with k1 = -0.30 gives s = 1 - 0.30 × 0.844 = 0.747, pulling the corner inward by 25 percent. That is what real barrel distortion looks like — a quarter of the frame, not a factor of 1e32.

The likely causes, in order of probability

1. Unwritten driver buffer. The denormal-plus-sentinel mix is its fingerprint: the array was allocated and published without being written. The fix landed in a later SDK release that initializes the coefficients and sets the model to 'None', so a driver update clears it at the source. Test: capture the same frame after the update; both units should then publish identical blocks.
2. Enum mapping gap. A camera_info bridge knows only a few model names (plumb_bob, rational_polynomial, equidistant); anything unrecognized is published as UNKNOWN while D passes through raw. Test: query the sensor with the vendor's own tool; if it reports none and zeros, the bridge is dropping the field, and your sanitizer should treat UNKNOWN as none.
3. Firmware and SDK mismatch. A struct layout change between versions leaves the parser reading offset fields. Test: repeat the capture on a second host with a matched firmware and SDK pair; garbage that follows the software rather than the sensor is a version problem.
4. Corrupt on-device calibration. The rarest case. Test: if the same block appears on two hosts after a power cycle, reflash the firmware through the vendor's recovery path; if it persists, the sensor goes back for repair.

The fix: sanitize once, where the block enters your code

1. Read the model string first. For none, empty or UNKNOWN, set the model to none and D to five zeros; do not parse the array at all.
2. For a named model, gate each coefficient: any NaN, infinity, denormal below 1e-30, or magnitude beyond 1 invalidates the array — replace all five entries with zeros, not just the bad ones.
3. Gate K next: fx and fy within 1 percent of each other, principal point inside the image, fx between one tenth and ten times the width. A K failure is the same bug one field over.
4. Never average your way out. One garbage frame among ten leaves the mean near -1.1e+29; validity comes from the gate, not from statistics.
5. Prove the result: build the undistortion map with the sanitized block. With D = 0 the map must be the identity, every pixel mapping to itself within 1e-6 px.
6. Store one known-good block as a regression test and replay it whenever the ingestion code changes.

What breaks downstream when garbage gets through

Undistortion is the first casualty: the map scales corner pixels by -9.5e+29, so the whole image becomes noise in one call. Pose estimation follows, because choosing PnP intrinsics that disagree with the frames you feed the solver degrades the result even when it converges. Point-cloud projection is quieter but just as dead: back-projection X = (u - cx) / fx × Z consumes K directly, and subtly wrong intrinsics surface as wavy point clouds that no filter fixes. Calibration code that accumulates coefficients across frames compounds the damage — the poisoned mean above is exactly that pattern.

The inverse hazard costs a day too: applying a non-zero model to an already-rectified stream bends correct depth, which is why rectified depth can look warped after a well-meant correction. And if the cloud feeds a robot, this same block decides where parts land in frame coordinates — how robot vision resolves depth into camera coordinates starts from intrinsics exactly this clean.

When zeroing D is the wrong move

• The color stream. Its lens is real, its model is usually radial-tangential, and its coefficients belong in the pipeline; fuse the streams through the factory extrinsics as covered in registering RGB and depth images.
• Fisheye and equidistant lenses. The model must match the routine or the undistortion call fails its own checks — matching the fisheye distortion model walks through that failure.
• Raw or unrectified depth modes. A few SDKs expose the depth grid before lens correction under a named model; values that pass the ±1 gate are measurements, and interpreting distortion coefficients on an unrectified image shows how to read them.
• Rectified stereo pairs. Both views were remapped once already; applying the coefficients again double-corrects the geometry.

Checklist before you trust an intrinsics block

• Model string names a known model or reads none — UNKNOWN means no contract behind the numbers.
• Every coefficient finite and inside ±1 — the fence that separates optics from memory.
• fx and fy agree within 1 percent — square pixels are a property of the sensor.
• Principal point inside the image bounds — a center outside the frame is a layout bug.
• Driver and firmware versions matched — the release that initializes D is the release you want.
• Undistortion map equals identity with D = 0 — the one test that proves the sanitizer works.
• One known-good block stored for regression — replay beats recapture.

Clean intrinsics are a one-function fix; choosing a sensor that publishes them cleanly is the other half of the job. Filter the camera finder by 3D sensor type to compare depth engines, or send us the block you captured and we will read it with you.