Depth camera coordinate origin: where is the imager center

A point cloud only means something once you know where [0, 0, 0] sits. Depth camera documentation describes the origin as the "center of the physical imager", which reads like the front of the lens, the middle of the housing, or the geometric center of the image — and each guess puts your robot picks, volume readings and gauge positions a different distance away. This guide pins the origin where the calibration actually places it, walks through the deprojection arithmetic with a worked example, shows how factory extrinsics move a point between the infrared and RGB frames, and sorts out which sensor types follow the rule and which replace it with a datum.

Where is the origin of a depth camera's coordinate system?

The origin sits at the optical center of the reference imager — the left infrared sensor in a stereo depth camera — so [0, 0, 0] is a point on that sensor's optical axis inside the camera, not a corner of the housing or a pixel of the image.

Streaming APIs attach a coordinate space to each stream, expressed in meters, and the depth values reference one physical sensor. Two consequences follow. First, X and Y are offsets from the optical axis: a point exactly on axis reads X = 0 and Y = 0 at any distance. Second, Z = 0 lies inside the camera, so every depth reading is a distance from the reference imager, and any working distance quoted from the housing front is a different number. When a reading looks wrong by a constant amount, check which reference the number assumes before you touch the math.

Is the origin at the lens front, the window, or the housing?

None of the three: the origin is the geometric point where the reference imager's optical axis passes through the optics, so Z grows forward from a plane near the sensor, and the housing, window and lens barrel are mechanical datums that never appear in the calibration.

Intrinsics and extrinsics describe the geometry behind the glass; they carry no dimension that locates the sensor inside the shell. For a mount that must reference the housing, that dimension comes from the module's mechanical drawing, or from a one-target measurement: press a flat target against the front window, read the depth of the plane, and the reading is the window-to-origin distance to within the depth noise — single-digit millimeters at half a meter, where stereo depth error is smallest. Assume the glass is Z = 0 and every reading shifts by that same unknown amount.

Which imager defines the depth origin — left IR, right IR, or RGB?

The left infrared imager defines it: the depth map is computed in that reference camera's frame, the right imager only feeds the stereo match, and the RGB sensor rides alongside with its own frame, so three sets of intrinsics and a set of extrinsics tie the streams together.

Factory calibration arrives per sensor: a principal point and focal length for the left IR imager, the same pair for the right IR imager and the RGB camera, plus extrinsics — a rotation and a translation — between each pair. The Z-origin question only really matters for the depth stream, because that is the stream whose numbers you turn into points. Rectified streams are already distortion-corrected, so pinhole math applies to them directly; raw IR frames still carry lens distortion, and the intrinsics and distortion parameters guide shows how to read both from the calibration rather than guess them.

How do you turn a depth pixel into X, Y and Z in meters?

Unproject each pixel with the pinhole model: X = (u - cx) × Z / fx and Y = (v - cy) × Z / fy, where (u, v) is the pixel index, Z is that pixel's depth in meters, and fx, fy, cx, cy are the reference imager's intrinsics at the running resolution.

The depth value is the only measured input; everything else comes from the calibration file. Square pixels make fx and fy equal, and when they differ each pairs with its own axis. Deproject only valid depth readings: a zero or dropout value runs the same equation and produces a confident-looking point sitting at or near the origin, which the deprojection-zero troubleshooting guide dissects. Reading the depth under one fixed pixel, as the center pixel depth measurement guide does, is just this equation with u and v held constant.

Worked example: one pixel to a 3D point

Take illustrative intrinsics of the kind a 1280 × 720 depth stream ships with: fx = fy = 612.0 pixels, cx = 639.5 pixels, cy = 359.5 pixels. One target pixel sits at u = 800, v = 400 with a valid depth of Z = 1.500 meters.

X = (800 - 639.5) × 1.500 / 612.0 = 160.5 × 1.500 / 612.0 = 0.393 meters

Y = (400 - 359.5) × 1.500 / 612.0 = 40.5 × 1.500 / 612.0 = 0.099 meters

The point is (0.393, 0.099, 1.500) meters in the reference imager's frame. A cross-check with angles agrees: 160.5 / 612.0 = 0.262 and arctan(0.262) = 14.7°, which is what 0.393 meters of sideways offset at 1.500 meters of depth should look like. At double the depth the same pixel deprojects to (0.787, 0.199, 3.000) meters — X and Y scale with Z, which is why pixel errors hurt more at range.

If a blob center and a deprojected pixel disagree by half a pixel, the two routines are using different coordinate origins; the blob centroid half-pixel offset guide walks that comparison.

Why do the same point's coordinates differ between the depth and RGB streams?

Each stream lives in its own coordinate space, so the same physical point carries different numbers in the depth frame and the RGB frame until you apply the factory extrinsic — a rigid transform whose translation is the baseline between the sensors, tens of millimeters on a typical module.

The extrinsic is a rotation plus a translation that maps a point from one sensor's frame into another's, and the factory data is the way to get it: the values come from per-unit calibration, not from a ruler and a guess. Say the left-IR-to-RGB extrinsic is a 25 millimeter translation along X with identity rotation — an order of magnitude you will meet on real modules. The worked-example point (0.393, 0.099, 1.500) meters becomes (0.368, 0.099, 1.500) meters in the RGB frame, because 0.393 - 0.025 = 0.368. Skip the transform and one stream swears the feature sits 25 millimeters to the right of where the other puts it: overlays still look fine, coordinate reads fail. The aligned-depth intrinsics guide sorts out which camera's values deproject which stream, the optical-frame transform guide covers the axis flips between image and optical conventions, and fusing several depth cameras with extrinsics scales the same discipline to multi-camera cells.

Does the same origin rule hold for ToF, structured-light and laser profilers?

The reference-imager rule covers stereo pairs and projected structured-light cameras, ToF cameras put the origin at their single imager's optical center, and laser profilers mostly report height against a calibrated reference plane, so the point-origin question dissolves into a datum choice.

Stereo and projected structured light share the geometry: a projector or a second imager only adds texture or a matching view, and the depth frame registers to one reference camera. A ToF camera times reflected light with a single imager, so its origin is that imager's optical center and the convention feels the same to your code. A laser profiler sweeps a line across the part and reports height against a reference plane you calibrate during set-up; its second axis comes from travel or an encoder, so the "origin" is a datum you define, not a point inside the housing. The 3D sensors range spans all three families. The MV-DPS200P-02 is a projected structured-light stereo camera of the reference-camera kind, the MV-DT01WMU is a ToF sensing camera, and the MIS-DP3850-01P is a 3D laser profile sensor of the datum-based kind.

How do you move points from the camera frame into your machine frame?

Chain the transforms: factory extrinsics carry a point from the reference IR frame into the right-IR or RGB frame, and your own hand-eye calibration carries it from the camera frame into the robot or machine frame, with meters kept end to end.

Hand-eye calibration applies when the camera rides the wrist; a fixture-based calibration with a known target applies when the camera watches the cell from a fixed post. Either way you add one matrix, and the whole chain stays in meters until a value reaches a report or the teach pendant. Two naming traps account for most failed chains: the reference frame a calibration routine expects is a convention, not a physical object, which the camera pose initial guess guide untangles; and where a stereo world origin sits is a choice you make, as the stereo triangulation world origin guide explains. Before wiring any of this to a robot, run through the 3D camera robot integration checks.

What should you check before you trust Z = 0?

Before any fixture, pallet pick or gauge trusts these coordinates, run five checks: correct stream, intrinsics matched to the running resolution, rectified depth input, extrinsics applied in the stated direction, and a physical target at a known offset to confirm sign and scale.

• Deproject the depth (left IR) stream, and match intrinsics to whatever stream you actually feed the equation.
• Read fx, fy, cx, cy at the resolution you run: halve the resolution and the pixel-based focal lengths halve too, so 612.0 pixels at 1280 × 720 becomes 306.0 pixels at 640 × 360.
• Feed rectified depth into the pinhole equations; undistort raw IR frames first.
• Apply extrinsics in the direction the SDK documents, and prove the direction once with a target on one known side.
• Confirm signs with a physical test: a target 0.100 meters right of the axis at 0.500 meters of depth should read X = +0.100 meters under the right-down-forward convention.

The decisions, case by case: the origin is the reference imager's optical center, so build frames around it and not around the housing; take the window-to-origin offset from the mechanical drawing or one target measurement; deproject depth with the reference imager's intrinsics only; compare streams through the factory extrinsics; and let laser profilers live on a datum, calibrated once at set-up. If you are choosing hardware around these conventions, the camera finder filters by type and resolution, the system configurator builds the chain from camera to mount, and the 3D vision topic page collects the rest of the frame-mapping guides — or contact the applications team with your cell drawing.