Depth value vs true distance: convert a pixel's Z to camera distance

You click a pixel in your machine-vision view, read a depth value, hold a tape measure to the object, and the two numbers disagree — sometimes by centimeters. The reason is almost always the same: the depth image stores Z, the perpendicular distance along the optical axis, while your tape measures the slanted, straight-line distance. This guide shows you how to deproject any pixel into a real 3D point and compute the true Euclidean distance, which physical point on the camera the measurement is referenced to, and how to keep RGB-to-depth alignment from eating your CPU budget. You will finish with a working formula, a verification procedure, and the checks to run before you trust the numbers.

If you are only reading the center pixel, the problem largely disappears — see center pixel depth measurement: read a fixed image point. For mapping whole regions rather than one point, bounding box depth: mapping image pixels to 3D points covers the general case.

What the depth image actually stores

A depth frame is a 2D array where every pixel holds one number: the Z coordinate of the scene point, in millimeters or meters depending on the depth scale (if your point cloud seems clipped at close range, that scale is the first suspect — see depth scale mismatch: why your point cloud is cut off at close range).

Think of the depth sensor as a pinhole camera. Each pixel (u, v) corresponds to a ray leaving the optical center. The depth value is where that ray's scene point lands on a plane perpendicular to the optical axis, at distance Z ahead of the camera. Two pixels can hold identical depth values while lying at very different straight-line distances from the camera, because one is near the center of the image and the other is far off to the side.

The exact image center is the only pixel where the ray is parallel to the optical axis, so Z and the Euclidean distance coincide there. That is why a tape check at the middle of the frame looks accurate and the edges look wrong — nothing is broken, you are just measuring two different quantities.

Deprojecting a pixel to a 3D point

The pinhole model maps pixel coordinates to a direction using the intrinsics: focal lengths fx, fy in pixels and the principal point cx, cy. The deprojection is:

Most 3D camera SDKs expose this directly as a deproject-pixel-to-point call, so you rarely need to write it yourself — but you should know what it does, because feeding it a pixel from the wrong stream silently gives wrong results (more on that below).

The angle effect alone is worth a table. With θ the pixel's angle from the optical axis, the Euclidean distance equals Z / cos θ:

A typical stereo depth camera has a horizontal field of view around 85–90°, so the image corners sit past 40° from the axis. At 1 m of depth that is over 40 cm of difference between Z and the true distance — far more than the sensor's own error.

Where the distance is measured from

Depth is referenced to the depth sensor's optical center, not the front face of the housing and not the RGB lens. On a multi-imager camera (stereo IR pair plus RGB, or structured light plus texture camera), each imager has its own optical center, and they are separated by real millimeters — often centimeters on stereo units with a wide baseline, a factor discussed in wide baseline stereo camera selection for depth mapping.

When you cross-check with a tape:

1. Find the depth imager's optical center in the camera's coordinate documentation, or measure the offset from the housing once against a known target at the image center.
2. Measure from that point, not from the front bezel.
3. Keep the tape along the line to your target point, not perpendicular to the camera — you are verifying the Euclidean distance now.

For work that needs camera-to-robot-frame transforms rather than raw distances, the pinhole-to-base-transform method in robot hand-eye calibration: how nine-point alignment works extends the same math.

Step-by-step: distance to any pixel

What you need: a streaming depth camera with its intrinsics, and the pixel (u, v) in the depth frame (or an aligned color frame).

1. Start the depth stream and grab a frame. Read the depth scale once (meters per raw unit) and convert Z to meters.
2. Read Z at your pixel: Z = depth_frame.get_distance(u, v) or the equivalent SDK call.
3. Deproject the same pixel to 3D with the depth intrinsics: point = deproject_pixel_to_point(depth_intrinsics, (u, v), Z), giving (x, y, z) in the depth sensor's coordinate frame.
4. Compute distance = math.sqrt(x*x + y*y + z*z). This is the true camera-to-point distance from the depth optical center.
5. To verify: aim at a target, tape-measure from the depth imager's optical center to the target, and compare. Repeat at the image center and near two corners; the center should match within the sensor's rated error, and the corners should now match too, unlike raw Z.
6. Average over a small patch (for example 5 × 5 pixels, rejecting zeros) if the surface is textured enough to give noisy single-pixel depth.

import math
# after deproject: p = (x, y, z) in meters
distance = math.sqrt(p[0]**2 + p[1]**2 + p[2]**2)
angle = math.degrees(math.acos(p[2] / distance))  # how far off-axis the pixel is

Printing the off-axis angle alongside the distance is a quick sanity check: if it is large, you now know why Z alone disagreed.

The RGB pixel trap and the cost of alignment

The common failure mode: you click a pixel in the color image and read depth at the same (u, v). The RGB imager and the depth sensor have different fields of view, resolutions and origins, so that pixel in the color frame corresponds to a different depth pixel. Near the center the error is small; toward the edges it grows into exactly the mystery you started with.

Two fixes, with very different costs:

• Full-frame alignment. The SDK resamples the color stream onto the depth frame (or vice versa). Correct for every pixel, and necessary if you want a color-textured point cloud — see point cloud with RGB and IR intensity: fuse depth, color and infrared. But it interpolates every pixel pair, every frame, which is real CPU at 30 fps.
• Per-pixel projection. Project the color pixel into the depth frame, read that depth value, deproject, take the Euclidean length. For a handful of measurement points this is far cheaper and exactly as accurate.

If the depth and texture streams also use different intrinsics, convert between them first as covered in camera intrinsic parameter conversion for depth and texture images.

Choosing hardware when distance accuracy matters

If your application is fundamentally "distance from camera to a point," pick the sensor for it:

• ToF / depth cameras give per-pixel Z directly and are the natural fit. An example from the 3D sensors category:

• Structured-light stereo cameras suit close-range static scenes with fine surfaces:

• Projected structured light works on textureless surfaces where passive stereo fails:

A ToF unit measures Z natively, so your only correction is the deprojection above. A stereo unit adds disparity-matching error that grows with distance, so calibrate and verify at your actual working range. If you also need 2D inspection, an area scan camera alongside the depth sensor gives you independent color data — at the price of the alignment work this guide describes. The camera finder filters by type, and the system configurator walks the full chain.

Checks before you trust the numbers

• Read the depth scale and confirm units; a meter/milli­meter mix-up dwarfs every other error.
• Verify at the image center first — that isolates sensor accuracy from geometry.
• Then verify at a corner; if the center is right and the corner is wrong, you compared Z against Euclidean distance.
• Confirm which stream your pixel coordinates came from, and align or project accordingly.
• Average a small patch on noisy surfaces and reject zero (no-return) pixels.
• Re-check after any firmware or SDK change; see 3D camera header version mismatch: fix firmware and SDK alignment.

When the usual answer is wrong

• The tape is not wrong just because it disagrees. At 30° off-axis, the tape should read about 15 % more than Z. Compute, do not assume sensor fault.
• Moving the camera farther away can look like it improves edge accuracy. The angle to a given scene point shrinks, so the Z-versus-Euclidean gap shrinks too — that is geometry, not better sensing. The sensor's own error usually grows with range.
• Alignment is not calibration. Aligning color to depth fixes pixel correspondence; it does nothing for depth scale or extrinsic offsets between imagers.
• Averaging hides geometry. The mean of raw Z over a patch is not the distance to the patch's center once you are off-axis. Deproject first, then average points.

If you are choosing a depth sensor for this kind of measurement work and want help matching range, field of view and accuracy to your target, contact our applications team with the working distance and the tolerance you need.