Chain pairwise camera extrinsics into one valid TF tree

Your cameras barely share a view, so you calibrated them in pairs — and the saved results put every camera under a /world frame that moved between runs. Lookups through /cam2 fail, RViz draws floating fragments, and nothing downstream trusts the poses. The fix is to stop publishing what the calibrator saved and publish the chain instead: one root camera, N-1 parent-child links, each camera appearing exactly once. This guide walks through the composition math, a publisher node you can adapt in an afternoon, and the checks that prove both the tree and the numbers.

What do you need to chain pairwise camera extrinsics?

You need two or more rigidly mounted cameras, one calibration target each pair can see at the same time, one extrinsic calibration per adjacent pair, N-1 runs for N cameras, and a small publisher node that broadcasts the composed links as static transforms.

The rig is two or more cameras — area scan cameras for a 2D chain, 3D sensors for a depth chain — mounted on one beam or frame, plus a checkerboard target that fills a good part of each pair's shared view. Trigger the cameras together or hold a still scene with fixed exposure, so both cameras of a pair record identical board poses. Touch nothing between the two captures of a pair: each result is only as good as the rig's stability during that run.

Shortlist hardware in the camera finder. Typical choices across common resolutions, all GigE with C-mount:

For a 3D rig the same chain rule applies to depth hardware:

If the rig is several depth cameras on one host, select each by serial number rather than by array index; the pattern is in Multiple depth cameras on one host: select each by serial number.

Why does a chained pairwise calibration break the TF tree?

TF requires every frame to have exactly one parent, and each pairwise calibration saved /cam2 against a different, arbitrary /world, so after two runs /cam2 has two conflicting parents and every lookup through it fails.

Each run defines its own /world — the board pose or first camera pose of that run — so /world is a different physical frame every time. The saved transforms are individually correct and mutually inconsistent:

tf2 answers a query by walking one path up and down a tree; a frame with two parents has no unique path, the tree is invalid by construction, and queries through it fail rather than return a stale or averaged pose. The failure is hard: nothing downstream through /cam2 resolves, not a subtle offset you could tune around.

A frame with no path at all is the sibling failure; its diagnosis lives in Missing camera frame tf transform: fix the frame tree.

Which camera should be the root of the chain?

Mathematically any camera can be the root, so pick the one whose pose other systems depend on most — usually the camera bolted to the machine frame or the one nearest the robot — and direct every link away from it.

Two rules fix the details. First, direction: walking outward from the root, each calibrated edge points parent to child, so every camera except the root is published as a child exactly once. In the cam1–cam2–cam3 chain, cam1 as root gives cam1→cam2 and cam2→cam3; cam2 as root gives cam2→cam1 (the inverse of calibration 1's relative pose) and cam2→cam3. Both are valid trees; pick the root your machine-frame anchor will attach to later.

Second, the calibration graph itself must be a tree. If you also calibrated cam1–cam3, you hold three links for three frames — publishing all three gives some camera two parents. Publish N-1 of them, drop the link with the worst residual, and keep the dropped pair as a cross-check for the verification step below.

How do you remove /world from a pairwise calibration result?

For each pair the calibrator saved world→cam_i and world→cam_j against its own private /world, and the relative pose you need is T_cam_i_cam_j = inverse(T_world_cam_i) × T_world_cam_j, so the arbitrary /world cancels out of the product.

A basis change cancels in a product of transforms — that is the whole trick. The inverse of a rigid transform needs no matrix inversion: the rotation block transposes and the translation becomes -R^T t.

The camera_pose_calibration package targets the simultaneous case — one checkerboard visible to all cameras — and its calibration_tf_publisher writes each result against /world. You have two ways out. Modify the publisher or the calibrator to store the relative pose directly, or leave upstream code alone and subscribe to the camera_calibration result topic, on which the finished poses arrive, and publish any TF from it that you like. The steps below assume the second path.

If you have multiplied transform chains for a hand-eye rig, this is the same move at rig scale: Eye-in-hand pixel to world coordinates: chain the transforms applies it to pin a pixel down in the world.

How do you publish the chain so every camera has one parent?

Write one small node that turns each saved pairwise result into exactly one parent-child broadcast — cam1→cam2 and cam2→cam3 — with a static transform broadcaster, and never publishes anything against /world.

Work outward through the graph:

1. Draw the graph. Nodes are cameras, edges are the pairs you calibrated. Confirm it is a tree; drop the weakest edge of any cycle and keep it as a cross-check.
2. Choose the root (previous section) and direct every edge away from it, inverting a relative pose where the calibrated direction opposes the tree direction.
3. Compose each link with the formulas above, offline in a script that writes parent, child, translation and quaternion into a small YAML file.
4. Broadcast each retained link once from a single static broadcaster, as in the snippet below.
5. Restart the node after any recalibration — static transforms persist until rebroadcast, so an edited link file changes nothing until the node re-sends it.

# Publisher skeleton: documented tf2_ros and geometry_msgs API names.
# xyz is the composed translation in metres; quat is the composed unit
# quaternion as [x, y, z, w]; values come from your link file.
from geometry_msgs.msg import TransformStamped
from tf2_ros import StaticTransformBroadcaster

def build_link(parent: str, child: str, xyz: list, quat: list) -> TransformStamped:
    t = TransformStamped()            # one message per retained edge
    t.header.frame_id = parent        # "cam1": the root camera
    t.child_frame_id = child          # "cam2": a child exactly once
    t.transform.translation.x = xyz[0]
    t.transform.translation.y = xyz[1]
    t.transform.translation.z = xyz[2]
    t.transform.rotation.x = quat[0]
    t.transform.rotation.y = quat[1]
    t.transform.rotation.z = quat[2]
    t.transform.rotation.w = quat[3]
    return t

broadcaster = StaticTransformBroadcaster(node)  # node: your rclpy Node
broadcaster.sendTransform(build_link("cam1", "cam2", [0.800, -0.150, 0.000], [0.0, 0.0, 0.0, 1.0]))
broadcaster.sendTransform(build_link("cam2", "cam3", [1.250, 0.050, 0.400], [0.0, 0.0, 0.0, 1.0]))

If the same node also serves image streams, validate the intrinsics side before you trust the chain: ROS 2 camera calibration: validate and publish camera_info.

How do you verify the tree and the composed numbers?

Two checks close the job: the tree dump must show one root, N-1 links and no frame with two parents, and a hand-composed transform must come back from lookup_transform to within 1 mm.

Run the tree dump from tf2_tools (view_frames) and read its summary line by line: exactly one root, one link per retained pair, and a parent count of one for every camera frame. In RViz, set the fixed frame to the root camera and confirm every camera frame resolves — when the fixed frame itself refuses to resolve, Rviz fixed frame world does not exist: pick a real TF frame walks that failure.

Then check numbers by hand on the three-camera beam example, axes parallel, identity rotations:

• Calibration 1 gives T_cam1_cam2: translation (0.800, -0.150, 0.000) m
• Calibration 2 gives T_cam2_cam3: translation (1.250, 0.050, 0.400) m
• Composed T_cam1_cam3: (0.800 + 1.250, -0.150 + 0.050, 0.000 + 0.400) = (2.050, -0.100, 0.400) m

With identity rotations the translations add. When the mid rotation R_A is not identity, rotate before you add: t_cam1_cam3 = t_A + R_A × t_B. If cam2 were yawed 90° about z, R_A × (1.250, 0.050, 0.400) m = (-0.050, 1.250, 0.400) m and cam3 lands at (0.750, 1.100, 0.400) m in cam1. Rotations compose as quaternion products: q_total = q_A ⊗ q_B.

# Verification: documented tf2_ros API; expected values from the worked example.
from tf2_ros import Buffer, TransformListener
from rclpy.time import Time

buffer = Buffer()
TransformListener(buffer, node)                     # node: your rclpy Node
result = buffer.lookup_transform("cam1", "cam3", Time())
# Pass: translation (2.050, -0.100, 0.400) m, rotation (0.0, 0.0, 0.0, 1.0)

Assert the returned translation within 1 mm of the hand value. The dropped cycle link pays for itself here: compare its direct cam1–cam3 calibration against the chained lookup, and agreement within your per-link budget confirms the chain.

Mistakes that cost hours:

• Editing the link file without restarting the publisher — static transforms persist, so the tree silently keeps stale numbers.
• Leaving the stock publisher running next to yours, quietly re-adding /world parents.
• Trusting the chain after a bump: every calibrated pair that included the moved camera is stale; pairs between untouched cameras keep their transforms.

How much accuracy does chaining cost over one shared calibration?

Each link adds its own calibration error, so budget translations in quadrature and rotations as a lever arm: two links at 0.5 mm each give about 0.7 mm at the far end, and 0.1° of rotation error at a link moves a camera 1.5 m downstream by roughly 2.6 mm.

The 0.5 mm is a planning number, not a constant: measure your own by re-running one pair calibration twice and differencing the results. Two links then give sqrt(0.5^2 + 0.5^2) = 0.71 mm at the end of the chain; five links give sqrt(5 × 0.25) = 1.1 mm. Rotation dominates on real rigs: 0.1° at a link moves a camera 1.5 m away by 1.5 × tan(0.1°) = 1.5 × 0.00175 = 2.6 mm, and the angular error of every earlier link also shifts every later link's reference.

Spend the effort where the formula points: a target that fills both views, even lighting, the rig untouched between a pair's two captures, and residuals accepted only under a fixed threshold. Two good pairwise calibrations typically land within a millimetre or two end to end — ample for robot guidance and gross inspection, and exactly why a shared-view simultaneous calibration stays the better tool when one pair must agree to a few tenths of a millimetre.

When should you not chain pairwise camera extrinsics?

Chain only when the calibration graph is a tree; if three or more cameras share one target view, a simultaneous multi-camera calibration is better, and if poses must reach the robot, anchor the root with one robot-frame calibration instead of stacking camera links.

The shared-view case is what the stock tooling was designed for: Multi camera extrinsic calibration in one shared coordinate frame covers the simultaneous setup, and Pairwise camera frame alignment from calibration observations covers solving relative poses from shared observations — the math your composed links replace.

For robot delivery, publish exactly one anchor: /world or the robot base to your root camera, from an eye-to-hand calibration (Publish eye-to-hand calibration as a static camera transform). The measurement-free workflow behind that anchor is in Camera to robot transform calibration: stop measuring by hand.

And if the cameras move relative to one another — separate axes, separate robots — no static chain is valid. A static tree describes a rigid rig; recalibrate after relative motion or derive the moving link from the machine kinematics instead.

Everything reduces to one habit: publish each calibrated pair as a single parent-child link, direct every link away from the root, and let the tree dump plus a hand-computed lookup prove it before the line runs. To pick the cameras, start a build at the camera configurator, or contact the applications team with your layout sketch. More integration guides sit under systems.