Camera coordinate frame vs point cloud x y z: why the axes differ

Log the first points of a Cartesian cloud, open the SDK viewer beside it, and the same corner of the target shows three different numbers. Nothing is broken: the viewer and the cloud report the same measurement in two different coordinate frames. The image-like view follows the optical frame that cameras and image processing use; the cloud structure follows the robotics camera frame, x forward, y left, z up. This guide names the axes of each frame, gives the fixed conversion between them, and shows the row-major pixel mapping that ties cloud->points[i] to its confidence value, so your log becomes something you can check line by line.

Why does the point cloud show different x, y, z values than the viewer?

The two views use two different coordinate frames: the viewer draws the optical frame, where x points right across the image, y points down and z points forward out of the lens, while the point cloud structure stores the robotics camera frame, x forward, y left, z up.

Both frames are right-handed and both describe the same physical point; they differ by a fixed rotation. The robotics layout comes from mobile robotics, where a platform drives along its own x axis, and camera SDKs that grew up feeding robot pipelines kept it. The viewer stays image-first because that is what raw depth data is.

Read the log with this in mind and the 'wrong' values resolve themselves: for a point ahead and right of the lens, the viewer's largest axis is z while the cloud's largest axis is x, and the two in-plane signs flip. Nothing needs recalibrating and no data is corrupt.

Which axes does each coordinate frame use?

The optical frame comes from image geometry: x runs right along the pixel rows, y runs down the columns and z runs forward into the scene. The robotics camera frame puts x forward, y left and z up, so every axis keeps a job but changes direction.

Both frames are right-handed, which is why the conversion below is exact: rotate the set of directions until x looks forward, then correct the two axes whose signs no longer match. A mirrored frame would not map this way; this one does.

Where the frames meet as whole clouds rather than single points, transforming depth point clouds between optical frame and camera frame wraps the same idea in one matrix multiply.

How do you convert a point between the two frames?

Apply one fixed permutation with two sign flips: the cloud's x is the viewer's z, the cloud's y is minus the viewer's x, and the cloud's z is minus the viewer's y. Nothing is rescaled, so distances and units survive the move unchanged.

Run it backwards the same way: z_opt = x_cam, x_opt = -y_cam, y_opt = -z_cam.

Worked example: a corner reflector sits 0.956 m in front of the camera, 0.610 m right of the optical axis and 0.428 m below it. The optical readings are x_opt = 0.610 m, y_opt = 0.428 m, z_opt = 0.956 m. Applying the mapping gives x_cam = 0.956 m, then y_cam = -(0.610 m) = -0.610 m and z_cam = -(0.428 m) = -0.428 m. The signs carry the geometry: right of center is negative in a left-positive frame, below center is negative in an up-positive frame, and the forward distance keeps its value.

How do you find the confidence value for cloud->points[i]?

The cloud is stored row-major in pixel order, so point i belongs to the pixel at column i % width, row i / width: the column is the remainder of i divided by the image width, the row is the integer quotient.

An organized cloud keeps the image shape — width × height points, one per pixel, no gaps — and the confidence image is the same width × height grid with one value per pixel. The lookup is arithmetic, not a search: keep i an integer, divide, take the remainder, read one array element.

Worked example: the camera delivers a 352 × 264 image, and 352 × 264 = 92,928 points per cloud. For point 46,353, divide: 46,353 / 352 = 131 with a remainder of 241, because 131 × 352 = 46,112 and 46,353 - 46,112 = 241. Row 131, column 241 is where the confidence value for cloud->points[46353] lives.

Two pipeline rules follow. Mask with the confidence image before you decimate, since why decimation shifts points covers what filtering does to positions — and it does the same to your indices. And when you need the reverse lookup, a pixel's x, y, z rather than a point's pixel, the deprojection calculation walks it on this same grid, as does mapping image pixels to 3D points for ROI work.

How do you turn confidence values into a clean cloud?

Treat the confidence value as a bitmask, not a level to threshold: one bit marks the pixel valid, other bits say why the camera distrusts the measurement, and you clear a point by zeroing its x, y, z so the array stays width by height and the pixel mapping survives.

A validity bit is the one to filter on; the remaining bits distinguish failure modes such as weak signal or a suspect distance, which matter when you tune illumination and exposure rather than when you reject points. The bit layout for your camera family ships in the SDK manual, and decoding confidence bitflags and masking bad pixels walks a real value bit by bit and builds the mask around it.

Zero the rejected coordinates (or write NaN, whichever your pipeline expects) instead of erasing elements. Erasing renumbers every later point, and the moment the array is shorter than width × height the arithmetic above points at the wrong pixels.

Which 3D sensors deliver a confidence image with the cloud?

Any depth camera that ships a confidence (or amplitude) image alongside the Cartesian cloud behaves this way: time-of-flight cameras and structured-light stereo cameras both do, because the convention lives in the SDK layer rather than in the ranging technology.

Time-of-flight pixels time returned light and grade each timing; structured-light pixels match a projected pattern and grade the correlation. Both grade per pixel, so a per-pixel quality channel comes naturally. Laser profilers are the different case: they deliver height profiles line by line and carry point quality inside the profile data, so the width × height lookup belongs to area depth cameras.

One time-of-flight camera in the shop's 3D sensors category:

And one projected structured-light camera:

How do you confirm which frame and unit your camera uses?

Confirm both conventions with one target and a rule: place a block forward and right of the camera, read the same point in the viewer and in the cloud, and check the sign of each column; a rule at a known distance fixes the unit.

Then run this list once and write the results into your code as constants:

• Viewer: the block sits right of center, below center, roughly a meter out.
• Cloud: x carries the meter value, y is negative (right of center), z is negative (below center).
• y comes back positive instead? Your SDK flips a different pair — note it once rather than rediscovering it quarterly.
• Unit: a target at 1.000 m reads 1.0 on the forward axis (meters) or 1000 (millimeters).
• Provenance: log a frame or timestamp identifier with every cloud-and-confidence pair.

With the camera frame pinned down, the hand-eye transform carries the cloud into robot coordinates. If the cloud arrives rotated inside the robot program instead, that is the companion problem, and fixing rotated camera axes in a robot vision program takes it from there.

When is the mismatch not the frame convention?

A uniform swap-and-flip means frames; a cloud that is rotated 90 degrees, mirrored, upside down or shifted only after hand-eye calibration points to a calibration or axis-order problem instead — fix that at the transform, not by negating columns in your reader.

Three other culprits produce similar symptoms. A cloud rotated exactly 90 degrees after hand-eye calibration usually has a matrix applied in the wrong order: point clouds rotated 90 degrees after hand-eye calibration. A cloud where the floor sits above the ceiling is a frame fault in the region of interest: diagnose an inverted point cloud. And when an exported file looks different from the live render for reasons unrelated to axes — colors, decimation, hidden points — the rendering path is the suspect: why exported point clouds differ from the viewer render.

The decision in one line: if viewer and cloud disagree by exactly the permutation above, encode the conversion once and move on; if they disagree any other way, chase the transform and the grab timing, not the reader loop. When the next step is hardware rather than code, the 3D camera finder narrows the depth-camera line by ranging type, and the configurator takes over from there:

More fixes in the same workflow sit on the 3D camera guides page, and the contact page reaches the applications team when you want a second pair of eyes on a log.