3D visionTroubleshooting

External odometry with RGBD mapping: why exact sync fails

External odometry breaks exact sync in RGBD mapping: the tracker stamps on a second clock. Set approx_sync true and drop the odometry-info input.

LumaSpec Vision10 min read Share on LinkedIn

On this page 9 sections
  1. 01Why does an RGBD map stay empty once external odometry is added?
  2. 02What does the "Did not receive data since 5 seconds!" warning tell you?
  3. 03How far apart can timestamps be before exact sync fails?
  4. 04Which launch arguments fix the pairing, and what do they cost?
  5. 05Why is the node still silent after approx_sync is true?
  6. 06Why does the map jump or reject loop closures once data flows?
  7. 07How do you verify the pipeline before trusting the map?
  8. 08When is exact sync still the right choice?
  9. 09Questions

Your RGBD mapping node ran happily on its own visual odometry. Then you added a tracking camera to supply external odometry, the 3D view stopped updating, and the log filled with one repeated warning: no data received for five seconds, and a note that the inputs are subscribed in exact-sync mode. That combination — a second odometry source plus exact sync — is the whole fault. This guide shows how to read the synchronizer's subscription list, work out how far the stamps are apart, set approx_sync and subscribe_odom_info so the callback actually fires, and verify every input with rostopic before you trust the map.

Why does an RGBD map stay empty once external odometry is added?

The callback that builds the map never fires, because exact sync compares header stamps across all subscribed inputs and requires them to be identical, and odometry from a second device cannot carry a stamp identical to the RGBD frames it is meant to guide.

A mapping cycle assembles one odometry sample, one color frame, one registered depth frame, one camera calibration and one odometry-info message. The node's time synchronizer releases that cycle only when every input carries the same header stamp. ROS stamps hold seconds plus nanoseconds, and "exact" means equal to the nanosecond. One mismatched input holds back the whole cycle, every cycle, forever.

The internal-odometry configuration ran, and that is the clue. One driver produced the color frame, the registered depth frame and the calibration, so one grab stamp went onto all three and the synchronizer matched them without effort. External odometry adds streams stamped by a second device — the startup log's list of odometry, RGB, depth, camera info and odometry info is exactly the set that must now agree — and that invariant is gone.

The odometry data itself is usually healthy. Echoing the topic shows position, orientation and a covariance array arriving at full rate. The data is not the fault; the pairing policy is.

What does the "Did not receive data since 5 seconds!" warning tell you?

It tells you the pairing policy and the full input list: a subscription line that reads "exact sync" and includes the odometry topic describes a match that can never complete, so treat the warning as the diagnosis rather than as a driver fault.

The warning prints once the node has gone five seconds without a released cycle, and it carries its own checklist: run rostopic hz on every topic, check that the header stamps are set, and synchronize the clocks when the topics come from different computers. Work that checklist in order, because it splits the failure into two classes.

If every topic publishes at its expected rate and the warning still repeats, the streams are healthy and pairing is the fault, which is the common case once a second odometry device joins. If a topic is quiet, the node starves on that input alone. A synchronizer is a conjunction: every input must arrive, so a silent camera_info topic looks identical to a clock mismatch.

In order of probability: stamp pairing across two clocks first, the odometry-info subscription second, a genuinely silent input third.

How far apart can timestamps be before exact sync fails?

At 30 Hz images and 200 Hz odometry the chance of an exact stamp match is 1 in 5,000,000 per pairing attempt, which works out to one lucky match roughly every 46 hours — exact sync with external odometry never fires on a real run.

The rates below are example inputs; compute your own from your configuration. The color and registered depth streams run at 30 Hz, so one frame period is 1/30 = 0.0333 s = 33.3 ms. The tracker publishes odometry at 200 Hz, so its period is 1/200 = 0.005 s = 5.0 ms. Registered depth carries the color grab's stamp and camera_info carries it too, so three of the five inputs already agree exactly; only the odometry streams are in question.

Case one: the tracker stamps in its own device clock. An image pair goes out stamped 168.204000 s, while the nearest odometry message reads 172.551000 s. The difference is 172.551 − 168.204 = 4.347 s, or 4,347,000,000 ns away from equality. Exact sync demands zero, so it never fires, no matter how healthy both streams are.

Case two: give the devices perfectly aligned clocks, which no two independently started devices have. Exact equality then needs an odometry stamp landing on the image stamp to the nanosecond. The odometry period is 5.0 ms = 5,000,000 ns, so one attempt matches with probability 1/5,000,000. At 30 pairing attempts per second, the expected wait is 5,000,000 / 30 = 166,667 s ≈ 46.3 h of continuous running — the callback count stays at zero forever.

p_exact = 1 ns / T_odom E[wait] ≈ T_odom / f_image e_pair ≈ T_odom / 2 + Δclock

T_odom is the odometry period, f_image the image rate, Δclock the residual offset between the two stamp sources, and e_pair the time by which a paired odometry stamp can misdate the pose.

Which launch arguments fix the pairing, and what do they cost?

Set approx_sync to true and the approximate-time policy pairs inputs whose stamps are close instead of identical; with everything stamped on one host, the paired odometry stamp sits within about 2.5 ms of the image stamp, half the 5.0 ms odometry period of the worked example.

roslaunch rtabmap_ros rtabmap.launch \
  rtabmap_args:="--delete_db_on_start" \
  rgb_topic:=/camera/color/image_raw \
  depth_topic:=/camera/aligned_depth_to_color/image_raw \
  camera_info_topic:=/camera/color/camera_info \
  approx_sync:=true \
  visual_odometry:=false \
  odom_topic:=/tracker/odom \
  frame_id:=odom

The topic names in the command are examples; pass the ones your driver publishes. Set frame_id to the tracker's odometry frame, which is the header.frame_id written into its odometry messages, because the node places every depth cloud in that frame.

The cost is bounded, and it is small. The paired odometry stamp can misdate the pose by up to half the odometry period plus the clock residual: 2.5 ms plus near zero on a single host. Motion converts that to position error of 2.5 ms × 1 m/s = 2.5 mm per frame at 1 m/s, comfortably inside mapping tolerance.

Two computers change the arithmetic. Keep approx_sync true and discipline both clocks with NTP or chrony; on a switched LAN the residual offset then stays well under 10 ms, small against a 33.3 ms image period. Do not try to bridge a multi-second offset: a 4.347 s gap at 30 Hz means buffering about 4.347 × 30 ≈ 130 frames per stream, and the synchronizer's queue is bounded, so inputs drop. A shared hardware trigger aligns capture moments at the source when both cameras accept one — see sync a depth camera and a third-party camera on one external pulse.

Interpolating the odometry pose onto the exact image stamp tightens the residual further; IMU and depth image timestamps: interpolation for alignment shows the method.

Why is the node still silent after approx_sync is true?

The synchronizer is waiting for a fifth input: with the odometry-info subscription still active, the node needs a topic only its own visual odometry produces, so set subscribe_odom_info to false and relaunch — a full ROS master restart is the clean way to rebuild the connection set.

Recent rtabmap.launch versions expose subscribe_odom_info as a launch argument; older releases declare it in several places inside the file, so change the defaults there instead. Subscriptions are built once at node start, so relaunch after the change, and restart the whole ROS master if stale connections linger.

The subscription list should then hold four inputs: odometry, RGB, registered depth and camera info. Uncertainty handling survives the change, because every odometry message carries a 36-value covariance array that the tracker fills on each sample.

Give both devices a minute to stream before judging: the five-second warning should stay gone, and the point cloud should appear in the tracker's odometry frame.

Why does the map jump or reject loop closures once data flows?

The tracker's own re-localization is the usual cause: when it re-locks to a place it recognizes, it rebases its odometry frame mid-run, the map tears along the jump, and loop closures come back as "Not enough inliers".

Disable re-localization on the tracking module before the run, so the odometry frame is fixed at power-up; the setting lives in the tracker driver's tracking-module namespace. If a session has already torn, rebuild with --delete_db_on_start rather than stitching a broken trajectory.

Why bother with a tracker at all? Published benchmark evaluations of RGB-D visual odometry put indoor drift on the order of 1% of the distance travelled, and a dedicated tracking camera exists to beat that — but only while its odometry frame stays put.

Start-up order matters as well. Launch the mapping node only after both devices stream; a tracker needs a few seconds to initialize, and a node that starts first sits on an empty input and prints the same five-second warning.

How do you verify the pipeline before trusting the map?

Verify in this order: every input publishes at its expected rate, the paired stamps agree to single-digit milliseconds, the subscription list matches what you configured, and the point cloud follows the rig — each check isolates one fault class.

  1. Run rostopic hz on each input: the image topics at their configured rate — 30 Hz in the worked example — and odometry at its own, 200 Hz there. One quiet input freezes the node.
  2. Compare one odometry stamp with one image stamp using rostopic echo. On a single host the paired stamps differ by at most about 2.5 ms; a difference of seconds means a second clock is still in play.
  3. Read the node's startup subscription line: odometry present, odometry info absent once subscribe_odom_info is false.
  4. Let both devices stream for a minute and confirm the five-second warning stays gone.
  5. Walk the rig and watch the point cloud track it; frozen or jumping stamps surface here, and the clock-source fix is the next stop.

When is exact sync still the right choice?

Keep approx_sync false when every input is stamped by one source: visual odometry computed from the same RGBD pair, with depth registered to color so RGB, depth and camera_info share the color grab's stamp. Zero pairing error beats a bounded one.

The rule: one stamp source, exact sync; two stamp sources, approximate. Hardware timestamping silently changes a device's stamp source, so re-check the policy after enabling it — depth camera hardware timestamps vs software frame sync covers that switch. If depth lags color by one grab on a single driver, pair by sequence number as color lags depth by one frame describes.

None of this changes the camera side: the fix lives in launch arguments and clocks. When you do specify the depth camera itself, start in the shop's 3D sensors range:

3D sensorsHikrobot MV-DPS200P-02 Projected structured-light 3D cameraMV-DPS200P-02
Type
Stereo / structured light
$8,829.00 · quotedView and quote
3D sensorsHikrobot MV-DLS300P-04 Laser galvanometer stereo cameraMV-DLS300P-04
Type
Stereo / structured light
$16,579.00 · quotedView and quote
3D sensorsHikrobot MV-DT01WMU ToF sensing cameraMV-DT01WMU
Type
Depth / ToF camera
Price on requestView and quote

For the software side, the RGBD SLAM setup guide walks a complete mapping configuration and pairs with this article. To pick hardware, the camera finder filters depth cameras by type and series, and the build-a-system configurator assembles the rest of the rig; more guides sit on the 3D camera guides page.

Questions

Does a faster odometry rate shrink the pairing error?

Yes. The worst-case misdate is bounded by half the odometry period, so doubling a 100 Hz tracker to 200 Hz halves the bound from 5.0 ms to 2.5 ms. Once odometry is denser than the image stream, the image rate no longer sets the bound; the residual clock offset dominates whatever is left.

Does the depth stream have to be registered to the color frame?

For RGBD mapping, yes. Registration puts depth and color on one pixel grid and lets the depth topic carry the color grab's stamp, which removes a stamp source from the synchronizer. Pass the registered depth topic to depth_topic and the color camera's calibration to camera_info_topic, exactly as the example command does.

Can the tracker and the depth camera run on different computers?

Yes, with approx_sync true and both clocks disciplined by NTP or chrony. On a switched LAN the residual offset stays well under 10 ms, small against a 33.3 ms image period. Keep the odometry stream off Wi-Fi: bursty delays arrive as stamp jitter the synchronizer's bounded queue must absorb.

The node maps fine now, but the viewer window stays empty — why?

The viewer is a separate subscriber with its own frame settings. Set its fixed frame to the tracker's odometry frame — the same frame name you passed to frame_id — and match the reference frame to it, or the display stays blank while mapping works. Judge data flow from the node's startup log, not the viewer.