Stereo camera_info frame_id on the right camera: intentional or a bug

You echo the right infrared stream of a stereo depth camera in ROS 2 and its camera_info reads frame_id: camera_infra1_optical_frame — the left camera's frame. Your detector, which pairs the camera_info topic with the image topic, now sees two different frames on one stream and wants an exception case. Before you write one or file a bug: many stereo wrappers do this deliberately while both streams run, because the rectified pair lives in a shared image plane and the baseline travels in the projection matrix. This guide gives you the decision rule that separates design from a real fault, a five-minute check you can run today, the arithmetic that shows the shared frame costs no depth accuracy, and the consumer-side fix that removes the exception case for good.

Why does the right camera's camera_info report the left camera's optical frame?

Because the wrapper publishes it that way on purpose when both infrared streams run: the two rectified images share the left camera's image plane, and the right stream's camera_info is stamped with the left camera's optical frame to match it.

Rectification is the reason. A raw stereo pair has two camera coordinate systems with epipolar geometry between them; rectification reprojects both images onto one common plane so that matching points fall on the same rows and disparity becomes a plain column offset. After that step, the geometry of the right rectified image is most naturally expressed in the left camera's frame, shifted along one axis by the baseline.

The message contract leaves room for that reading. The sensor_msgs/msg/CameraInfo definition says the header "frame_id should be optical frame of camera", with the origin at the optical center, +x to the right in the image and +z pointing into the scene. For a rectified right image, the operative optical frame is the shared one.

The practical driver is older stereo code. Packages that rebuild depth from a pair of camera_info messages required the two frame_ids to match; handed two different strings, they rejected the pair or went looking for a tf transform the depth math never needed. The wrapper's authors chose the shared frame to keep those packages working and recorded the rationale in the wrapper's issue tracker. Meanwhile the image topic header on the right stream keeps camera_infra2_optical_frame — the disagreement between the two topics of one stream is the tell that you are looking at this design, not at corruption.

Is the shared frame_id a bug or an intentional shortcut?

It is intentional as long as three conditions hold: both streams run, the frame_id reverts to the right camera's own optical frame when the left stream is disabled, and tf still carries both optical frames with the baseline between them.

The substitution is conditional code, not a corrupted header. Corruption does not depend on which streams you launched; this behavior does, because the wrapper sets the right optical frame equal to the left one only while the pair runs. The strict reading of the message definition would put camera_infra2_optical_frame in that header, so calling it a contract violation is defensible — but the choice is deliberate and version-tracked. Treat it as design and adapt your consumer; escalate as a bug only under the patterns in the real-fault section below.

How do you verify it in five minutes?

Run six checks in order: echo both camera_info topics, read the tf transform between the two optical frames, disable the left stream and re-echo, read the image header, and compare the baseline from P with tf.

1. Launch the wrapper with both streams on — in the launch file, enable_infra1:=true enable_infra2:=true — then run ros2 topic echo /camera/camera/infra2/camera_info. Note frame_id: camera_infra1_optical_frame.
2. Echo /camera/camera/infra1/camera_info. The left stream names its own frame, so the pair now shares one frame_id.
3. Ask tf for the bridge: ros2 run tf tf_echo camera_infra2_optical_frame camera_infra1_optical_frame. Expect translation on one axis at your baseline magnitude and near-zero rotation; the sign follows lookup direction. Optical frames are x right, y down, z forward, so a horizontal pair moves on x.
4. Relaunch with the left stream off — enable_infra1:=false enable_infra2:=true — and echo the right camera_info again. camera_infra2_optical_frame in that header proves the substitution is conditional.
5. Echo the header of the right stream's image topic in the same paired launch. It keeps camera_infra2_optical_frame, which is exactly the mismatch your detector trips over.
6. Recover the baseline as |P[0][3]| / fx from the right camera_info and compare with the tf translation from step 3. Agreement within rounding closes the case: design, geometry intact.

If step 3 already fails, jump to the real-fault section — a missing transform is a different problem from a shared frame string.

Where does the baseline live when both frame_ids match?

In the right stream's projection matrix: the rectified right camera_info carries the baseline inside P as fx × B in its P[0][3] element, which is exactly why the frame label can be shared without losing geometry.

Depth from a rectified pair never asks tf for the baseline. It multiplies focal length by baseline and divides by disparity, and all three numbers come from the camera_info messages themselves. Which intrinsics apply to which stream after alignment is its own question — aligned depth streams: data layout and which intrinsics apply covers it.

Z is depth, fx the horizontal focal length in pixels, B the baseline in the calibration's length unit, and d the disparity in pixels.

The frame_id appears nowhere in that arithmetic. It names the reference frame for anything consuming the message downstream; the metric stereo model rides entirely in K and P. When you later push depth data into a point cloud, frame conventions matter again — transforming depth point clouds between the optical frame and the camera frame shows which axes to expect.

How do you stop your own detector needing a special case?

Read the frame from the image header, not from camera_info: on this class of wrapper the image message keeps the right camera's own optical frame, so a detector that pairs image and camera_info should anchor its transform to the image header and treat camera_info purely as intrinsics.

Three exception-free patterns, in order of preference:

• Anchor on the image header. Publish your transform into image.header.frame_id and use camera_info for K and P whatever its frame string says. infra1-only, infra2-only and paired launches then behave identically, and the shared frame stops being your problem.
• Bridge through tf. When the two headers disagree, look up the transform between camera_info.header.frame_id and the image header's frame. On this pair that lookup is exactly the right-to-left baseline you validated in step 3, delivered as one tf call.
• Compose the pose yourself. A checkerboard detector that publishes per-camera transforms can compute the right camera's pose as the left pose composed with the P-derived baseline. That is the general form; no stream gets special treatment.

Resist forking the wrapper to re-stamp the right camera_info with its own frame. The shared frame exists for the depth-consuming packages; re-stamping moves the exception out of your node and into every depth node downstream. Fix the consumer, leave the publisher alone.

When is the frame_id actually wrong?

Treat it as a real fault when the frame is wrong with only one stream running, when the string is empty or garbled, when tf cannot chain the two optical frames, or when depth scaled against a known distance is off by more than a few percent.

Causes, ordered by how often they show up on the bench:

• Camera-name parameter or topic remapping. The frame string is assembled from the camera name at launch; a wrong parameter or a remapped topic name yields an empty or stale frame. Test: print the camera-name parameter and echo the topic path your node actually subscribes to.
• Missing static transform. If tf_echo fails between the optical frames, nothing is publishing the baseline — the URDF is not loaded, the description publisher is down, or the wrapper's own static publisher is disabled. Fix: restore that publisher so tf carries the baseline again; the value is the one you already recovered in step 6.
• Stale extrinsics. Depth with a clean frame_id but a scale error against a measured distance points at calibration, not at labels. Recalibrate the pair — stereo depth camera calibration: target capture and stream setup covers capture, and ROS 2 camera calibration: validate and publish camera_info covers validating what you publish.
• Wrapper regression. After an update, the conditional logic can change; the issue tracker is where the maintainers recorded the shared-frame rationale, so check the version you run before assuming your memory of the behavior still holds.

If the pair is even in doubt as rectified, verify it yourself before blaming any frame string: record both raw streams, apply the distortion model from each camera_info, and check that matching points fall on the same image rows — only a row-aligned pair earns the shared-frame reading above. Depth that jitters frame to frame or drifts with motion is a different failure family; stereo depth jitter and IMU drift troubleshooting works through its causes.

Which stereo cameras and cables does this apply to?

Any stereo device whose ROS wrapper publishes rectified infrared pairs can show it — binocular structured-light heads, laser-galvanometer stereo units, or a two-camera rig you rectify yourself — because the behavior lives in the wrapper, not in the sensor.

For a ready-made head, browse stereo and structured-light 3D cameras: the MV-DLS300P-04 laser galvanometer stereo camera and the MV-DPS200P-02 projected structured-light unit both deliver pairs whose ROS integration is exactly this discussion. Building the pair yourself from two area scan cameras puts the wrapper decisions in your hands — and the first requirement is synchronized shutters, which hardware sync two USB cameras: trigger both cameras together works through.

On cabling, short bench runs suit USB3: the LS-CBL-0219 is a 3 m USB3.0 camera cable, high-flex and drag-chain rated. Longer runs on a GigE head want the LS-CBL-0199, a 3 m GigE camera cable with screw locks and the same drag-chain rating — browse camera cables for other lengths.

What should the integration checklist look like?

Six checks before you call the integration done: both streams on at launch, both camera_info topics echoed, tf_echo returning the baseline, the single-stream flip confirmed, the P-matrix baseline matching tf, and your node reading the image header for its frame.

• Both infrared streams on — the shared frame appears only with the pair running, so test the paired case, never a single stream alone.
• Both camera_info headers read side by side — one echo per topic, not one inferred from the other.
• ros2 run tf tf_echo camera_infra2_optical_frame camera_infra1_optical_frame returns one axis of translation at the baseline magnitude with near-zero rotation.
• Single-stream relaunch flips the right frame_id to camera_infra2_optical_frame — the proof the substitution is conditional.
• Baseline from |P[0][3]| / fx matches the tf translation — P and tf telling the same story is your geometry audit.
• Your node takes its pose frame from image.header.frame_id — the habit that removes every exception case downstream.

If the camera side is still open, the camera finder filters stereo heads by type, and the system configurator walks camera, lens, light and cabling in one pass. For a wrapper-level question on a head you already run, reach us through the contact page. More build-level write-ups sit under the systems topic page.