Point cloud rotated 90 degrees after hand-eye calibration

You calibrated the camera against the robot, published the extrinsic, and the point cloud arrives in the robot frame tipped onto its side: rotated about 90 degrees on two axes, with the shape of the scene intact. Nothing about the scene is wrong, and the calibration itself is probably fine. The transform chain mixes two axis conventions, and the mismatch is one fixed rotation. This guide is for anyone wiring a depth camera into a robot over a TF tree. You will name the two conventions, prove the mismatch with a three-line matrix calculation, fix it by republishing the extrinsic between matching frames or inserting one static rotation frame, and verify against a taught point in minutes.

What the 90 degree symptom tells you

The signature is specific. The cloud keeps its shape and its proportions, every surface sits at a plausible depth, but the whole scene is re-oriented: the table top stands upright, the camera's viewing direction runs along a robot axis it never pointed along. A rotation that maps each axis onto a different axis — not a small tilt, not a jitter — points at the frame conventions, not at the measurements.

Two rows decide where you start. Ninety degrees on two axes is this article. A constant offset with correct orientation means the calibration output never reached the tree — jump straight to cause 2 before touching any numbers.

The two conventions, and the 90 degrees between them

Robot frames and camera link frames run x forward, y left, z up. Depth drivers publish their clouds in the optical convention: z forward out of the lens, x right in the image, y down. Neither convention is wrong; one describes the device in the cell, the other describes the images on the sensor. A transform tree that mixes them without carrying the offset between them corrupts every cloud that passes through the joint.

The offset between the two conventions is fixed and deterministic — two 90 degree turns in sequence:

Read R by its columns: forward (body x) lands on optical z, left (body y) lands on optical −x, up (body z) lands on optical −y. Composed, the two turns equal a single 120 degree turn about the diagonal, which is why the faulty cloud reads as tipped onto a different face rather than spun in place. For the day-to-day bookkeeping between a depth message's optical frame and the camera's own frame, the optical-frame transform explainer works the same multiplication in both directions.

Worked example: an eye-to-hand output published under the cloud's optical frame

The inputs, with their sources:

• Robot base frame: x forward, y left, z up — the controller-side convention.
• Point cloud frame: z forward, x right, y down — the depth driver's optical convention.
• Solver output: one 4×4 transform from base to camera, solved with the camera pose taken from the link (body) frame.
• Observed symptom: the cloud rotated about 90 degrees on two axes, shape intact.

The rule: a transform chain composes correctly only when every hop speaks one convention. Publishing a body-convention transform as the parent of an optical-convention cloud makes the tree read the cloud's coordinates as body coordinates, so the whole scene misses exactly the hop R.

The arithmetic. Optical coordinates read as body coordinates move each physical direction onto another axis:

• forward arrives as (0, 0, 1) in the optical frame; read as body coordinates, that is up.
• left arrives as (−1, 0, 0); read as body coordinates, that is backward.
• up arrives as (0, −1, 0); read as body coordinates, that is right.

Every direction lands on a different axis — two 90 degree turns' worth of re-orientation, matching the symptom exactly.

The result: the fault is one constant rotation, not bad data and not a bad solve.

The decision. Insert the exact hop R once, or republish the transform between frames of one convention, and the visible rotation goes to zero. Then touch a taught point: a residual within a few millimetres closes the case. A leftover tilt of degrees or an offset of centimetres means a second fault in the chain. Calibration faults that large do occur — published datasets for learning-based hand-eye calibration inject perturbations up to 150 mm and 90 degrees into their data — but you settle the convention first and judge the solve second.

The likely causes, and the test for each

Work down the list; each test takes minutes.

1. The output was published under the wrong frame. The calibration edge attaches to a frame name that uses a different convention than the cloud carries. Test: dump the TF tree with your middleware's standard frame-dump tool and compare the frame name on the calibration edge with the frame name carried by the cloud message. Different conventions on the two ends means you have found it. Fix: the next section.
2. The calibration output never went live. The tree still carries the dummy camera pose from calibrate.launch because the stack was never restarted with publish.launch, the launch that publishes the stored result. Test: jog the robot a few hundred millimetres and watch the cloud; a stale or dummy edge shows the scene swimming against the motion. Fix: restart the whole stack from the publishing launch.
3. Marker frame versus camera frame. The solver consumed the tracking target's pose on the robot side and one camera frame on the other side, while you published the edge to a different camera frame — the optical centre and the link frame are the classic pair. Test: display both camera frames in the visualizer and confirm the published edge connects exactly the two frames the solver sampled. Fix: republish between those exact frames; the missing calibration target frame checklist sorts out a frame that refuses to appear.
4. A robot-side frame configured with a rotation. A rotated base frame, or a tool frame written for a rotated mount, injects its own fixed rotation into the chain. Test: move the robot to a taught Cartesian pose and compare the pendant readout with the expected numbers; a fixed angular mismatch follows the configuration, not the camera. The write-up on configuring a rotated robot base frame shows what to check controller-side.
5. The solve itself. Poses too similar, a marker smeared in the images, or the target bumped mid-capture. Test: clear causes 1 to 4 first, then run the taught-point check below; a systematic residual of degrees or centimetres convicts the solve. Fix: recapture with wider pose diversity — the eye-in-hand pose selection guide lists positions that carry information.

Causes 1 and 2 between them explain most field cases, which is why the tests are ordered this way.

The fix: republish between matching frames, or insert one rotation frame

Option A — republish. Attach the solver's transform between the frames it actually consumed. Most depth drivers already publish their own static edge between link and optical frames, so the two edges compose on their own and no new frame is needed. This is the clean fix, and the static eye-to-hand publish guide shows the publishing side step by step.

Option B — one intermediate frame. If you cannot change which frames the two sides use, publish a single static transform between them that carries exactly R: 90 degrees and 90 degrees, entered as exact values, not tuned by eye.

With the frames settled, the same chain now drives picks directly: the robot-base pick computation from eye-to-hand calibration results walks the composed transform end to end.

Verify in five minutes

1. Plane check. Sweep a flat, known surface such as the table or the pallet base. The fitted plane should sit at its taught height within a couple of millimetres and read horizontal in the robot frame.
2. Taught point. Touch a fixed corner with a pointer, read the cloud's coordinate at that pixel — the center-pixel depth read shows the readout side — and compare against the robot's numbers. A few millimetres passes; the pixel-to-robot arithmetic behind this check is worked through in a pixel-to-robot coordinate transform walkthrough.
3. Axis check. Command the robot 100 mm along +X. The scene in the visualizer must translate along the robot's +X, not along −Y or +Z; then one jog per axis.
4. Round trip. Compute one pick from the cloud, command the robot to it, and measure the approach error on a repeated target. The eye-to-hand object pose transform for picks covers that last hop.

Rotation gone, axes honest, taught point within a few millimetres: the chain is done.

Prevention

• Put the convention into the frame name itself, so a mismatch is visible when you dump the tree instead of buried in a matrix.
• Keep the calibration launch and the publishing launch separate, and never run the production stack from calibrate.launch.
• Record in the cell file which frame each side of the solve consumed; the answer stays one comment away when this returns.
• Re-run the taught-point check after any camera re-mount; a bumped bracket recreates cause 5, and no convention fix will hide it.
• If you are building the solve from scratch, the nine-point hand-eye alignment explainer shows what the solver is actually fitting, so the frame questions above get asked before the first capture.

Parts behind the transform tree

The fault is software, but it lands on hardware you may be speccing right now. A depth camera mounted over the cell does the measuring:

and the machine that runs the driver, the publisher and the robot interface needs the ports to keep every stream on its own wire — five or more LAN ports and four built-in light channels are the practical floor for a cell like this:

Browse the 3D sensors and vision controllers categories, or filter by type in the camera finder.

If a residual survives every check above, send the TF tree and the two frame names through the contact page and we will read the chain with you. Speccing the cell from scratch: the system configurator walks camera, lens, light and controller in order.