Depth camera orientation relative to a fixed frame

Tracking a moving depth camera is straightforward: the driver's tracking stack estimates motion and publishes a pose. The catch is which frame that pose refers to. Most stacks publish orientation against the odometry frame created at startup, so a reading of "zero" only means the camera has not moved since launch. This guide shows you how to express that pose against a fixed world frame you choose, and how to run two depth cameras at once without their topics and frames colliding.

It is written for anyone working with a ROS-style driver stack on a mobile robot, a gimbal, or a manually moved camera. You will need the camera driver running, the tracking or SLAM node publishing a pose topic, TF tools available on the machine, and RViz or an equivalent viewer for checking the result. If you are still choosing hardware, the 3D sensors range covers stereo, structured-light and ToF cameras, and the camera finder filters them by type and interface.

Why the pose reads zero at launch

A visual-inertial or SLAM node has no prior knowledge of the room. When it starts, it defines its odometry frame (odom, or a camera-local frame) at the camera's current position and orientation. The first pose it publishes is therefore the identity: zero translation, zero rotation. From then on, every pose means "relative to where I was when the node started".

That is often exactly what you want for SLAM and mapping. It is not what you want when you need to know which way the camera points towards a conveyor, a pallet, or the robot's base frame. The fix is not to re-zero the tracker; it is to add a static transform between the tracker's frame and your fixed frame, and to ask for the pose in the right frame.

Publish a static transform to the fixed frame

The transformation between the startup frame and your world frame does not change while the tracker runs, so a static transform is the right tool. If your world frame is the robot base and the tracker started while the camera was mounted 0.5 m ahead of the base and level, the static transform is exactly that offset with zero rotation. If the camera was tilted at launch, the static rotation must include that tilt.

In practice:

1. Start the driver and the tracking node as usual and confirm the pose topic is publishing.
2. Measure or estimate the transform from the tracker's startup frame to your fixed frame: translation in x, y, z and rotation as roll, pitch, yaw.
3. Publish it with a static transform publisher, attached to the frame tree with a clear name such as world as parent and the tracker's odometry frame as child.
4. Verify with rosrun tf tf_echo world odom_frame while moving the camera: the numbers should now be stable when the camera is still, and change only when it actually moves.
5. In RViz, set the fixed frame to world so the camera model and point cloud render in your frame, not the startup frame.

Get the orientation in the frame you want

With the transform in the tree, a lookup converts for you:

1. Use the TF lookup interface (lookupTransform(target_frame, camera_frame, latest_time)) with your fixed frame as the target.
2. Read the returned quaternion, or convert to roll-pitch-yaw if your downstream code prefers Euler angles.
3. If you subscribe to the odometry topic instead, transform the message's pose yourself using the same static transform; do not assume the topic's frame is your world frame.

A frequent mistake is reading the orientation topic directly and treating its values as world orientation. Check the header.frame_id of every message you consume. If it names the odometry frame, the values are relative to startup.

This frame discipline matters in every part of a 3D pipeline, not just tracking. The same thinking appears in the guide on bounding box depth: mapping image pixels to 3D points, where pixel coordinates only become useful once you know which frame the depth values live in.

Run two depth cameras without collisions

Two cameras on one machine fail in two ways: duplicated node names, and identical TF frame names. Both are solved by namespacing.

1. Launch each driver node with its own camera name or namespace, for example cam_front and cam_rear.
2. Confirm in the topic list that images, point clouds and poses carry the prefixes (/cam_front/..., /cam_rear/...).
3. Check the TF tree: each camera should own its own link names. If both publish a camera_link, rename one at the driver level.
4. In RViz, add one display per camera and pick the namespaced topics; set the single fixed frame once, shared by both.

Now each camera reports its own odometry and orientation, readable in your fixed frame using the lookup from the previous section.

When the cameras have a known mounting relation

If both cameras are rigidly mounted, you know (or can calibrate) the transform between them. Add one more static transform linking the two camera frames. The tree then reads: world → odom_front → cam_front → cam_rear. Both camera poses are now expressible in the same fixed frame, and a single lookup per camera gives consistent orientations.

Estimating that inter-camera transform is its own task. The article on multi camera extrinsic calibration in one shared coordinate frame shows how to recover it from a shared scene, and the guide on multi-camera point cloud synchronization for mobile robots covers keeping the captured data itself aligned in time. When rigid mounting is not possible, a projected structured-light 3D camera such as the MV-DPS200P-02 can simplify the geometry, since its pattern projection does not depend on texture in the scene the way stereo matching does.

For a fixed surveillance or inspection pose that only needs a one-off orientation, you may not need odometry at all: a single calibration against a known target, done once and stored, replaces live tracking. That is often the more robust answer, and it is the same principle behind tool-mounted setups, where the mounting offset between a camera and the tool flange must be resolved before the pose data is usable; the article on resolving tool-mounted camera orientation frame offsets goes into that case in detail at resolving a tool-mounted camera's orientation frame offsets.

Verify before you trust the numbers

Run through these checks once per set-up:

• Stillness test. With the camera untouched, the pose in your fixed frame should hold steady to within the tracker's stated jitter. Drift while stationary means the static transform is wrong or the tracker is slipping.
• Known-move test. Rotate the camera by a measured angle, for example 90 degrees about one axis, and confirm the reported orientation matches.
• Frame check. Print header.frame_id on every subscribed topic; anything still naming the odometry frame is not yet in your world frame.
• Second-camera cross-check. With both cameras watching a common target, compare their reported target positions in the fixed frame; disagreement points to a wrong inter-camera transform.

If timestamps rather than frames are the problem, the article on depth and color synchronization: check the right timestamps explains which stamp to compare, and the guide on multi-camera RGB synchronization: verify exposure timing covers the multi-camera timing side.

Choosing hardware for a moving-camera set-up

If tracking accuracy matters, pick the camera technology to match the scene. Stereo needs texture; structured light works on bland surfaces at short range; ToF trades detail for range and robustness. A compact ToF sensing camera such as the MV-DT01WMU suits larger volumes where millimetre detail is not required, while structured-light units suit close-range dimensional work.

Quick checklist

• World frame defined and named before launch.
• Static transform published between the odometry frame and the world frame.
• Pose looked up via TF in the fixed frame, not read raw from the odometry topic.
• Each camera namespaced; TF tree checked for duplicate link names.
• Inter-camera static transform added only when the mount is rigid and measured.
• Stillness and known-move tests passed before the data feeds anything downstream.

To size the camera and lens for the scene your moving camera must cover, start from the camera resolution calculator, then work through the system configurator to match camera, mount and cabling. If the pose data feeds a picking or alignment step, the article on rectified image pose estimation: choosing PnP intrinsics is the natural next read.