RGB-D odometry stuck waiting for IMU: why no camera feed appears

You launched a handheld RGB-D scan, rviz shows the color and depth streams, and the rtabmapviz window stays blank. The console prints the reason every few hundred milliseconds: odometry: waiting imu (/rtabmap/imu) to initialize orientation (wait_imu_to_init=true). Nothing is wrong with the camera. The odometry node is holding every synced RGB-D pair until one IMU message with a usable orientation arrives on the topic you named, and nothing is arriving there. This guide pins down that gate, walks the three causes in order of likelihood, gives the command that proves each one, and closes with the decision: remap and filter the IMU topic, or drop the wait and start without gravity alignment.

The symptom: raw topics stream, odometry stays silent

rviz displaying /camera/color/image_raw is your control experiment: driver, USB link and timestamps are all healthy. The failure sits downstream, in rgbd_odometry.

The node did subscribe and sync correctly — its own log line lists the three topics it matched: color, aligned depth and camera info. Approximate sync is working. What follows the subscription is the waiting warning, printed once per synced pair, about every 60–80 ms in this log — roughly 15 pairs per second — and each pair is dropped, not queued.

rtabmapviz renders what odometry publishes. While odometry publishes nothing, both rtabmapviz and the rtabmap node (which syncs on /rtabmap/odom and /rtabmap/odom_info) have nothing to render, so the view stays blank no matter how good the camera output is.

What wait_imu_to_init gates, and why

The IMU supplies the gravity vector for the first keyframe. The launch sets Optimizer/GravitySigma to 0.3, so the graph optimizer holds gravity as a constraint at that sigma and keeps the map's z axis vertical. A handheld camera cannot be assumed to start level, so the first pose must take its roll and pitch from the IMU.

wait_imu_to_init:=true makes rgbd_odometry discard frames until that orientation estimate exists: one message carrying a non-zero quaternion starts the pipeline. After that, the IMU keeps feeding the inertial-visual fusion loop inside odometry — the same sensor-fusion structure described in Kalman filter fusion of IMU and camera data.

The flag trades feed availability for a correctly initialized map. When the IMU chain is broken you get neither: no feed and no map.

Cause 1: the IMU topic has no publisher

The failing launch passes imu_topic:=/rtabmap/imu. rtabmap.launch starts three nodes — odometry, rtabmap and rtabmapviz — and no IMU driver. Unless something remaps a live IMU stream onto that exact name, the topic has zero publishers.

Stereo depth cameras with a built-in IMU publish gyro and accelerometer as separate streams; a single combined sensor_msgs/Imu topic appears only when the driver is told to unite them. In the camera driver launch, set:

enable_gyro:=true
enable_accel:=true
unite_imu_method:=linear_interpolation
align_depth:=true

That publishes /camera/imu at close to the gyro rate, about 200 Hz, with the accelerometer near 250 Hz. The align_depth flag is what produces the aligned_depth_to_color topic you already pass; the full alignment chain is covered in depth image registration to RGB.

Now test the topic you actually named:

rostopic info /rtabmap/imu
rostopic hz /rtabmap/imu
rostopic list | grep imu

No publishers on the first command, or a silent second command, confirms the cause. The third line shows the names that really exist.

Cause 2: messages arrive with an empty orientation

Remapping alone may leave you waiting. The united topic fills angular_velocity and linear_acceleration and leaves orientation at zero — orientation is computed by a Madgwick or Mahony filter, not by the camera driver, and the wait ends only when a quaternion shows up.

Check with:

rostopic echo -n 1 /camera/imu

An orientation of 0, 0, 0, 0 means odometry will wait forever at any publish rate.

Fix it by placing the filter between driver and odometry, and keep the flag:

<node pkg="nodelet" type="nodelet" name="imu_filter"
      args="load imu_filter_madgwick/ImuFilterNodelet camera_nodelet_manager">
  <remap from="imu/data_raw" to="/camera/imu"/>
  <remap from="imu/data" to="/rtabmap/imu"/>
</node>

The filter publishes onto /rtabmap/imu, so the name odometry subscribed to never changes — this one nodelet solves cause 1 and cause 2 together. Hold the camera still for the first second or two after launch: Madgwick needs a few hundred samples at 200 Hz, roughly 1–2 s, before its quaternion is trustworthy.

Cause 3: the IMU frame does not reach camera_link

Once IMU messages flow, odometry looks up the transform from the IMU's frame (typically camera_imu_optical_frame) to your frame_id — camera_link here — within wait_for_transform_duration, 0.2 s in this log. A missing link produces its own warning naming both frames, and odometry resets instead of waiting quietly, so the log tells the two conditions apart.

Test:

rosrun tf2_tools view_frames.py
rosrun tf tf_echo camera_imu_optical_frame camera_link

The frame graph must contain a chain from camera_link down to the IMU's optical frame. Let the camera driver publish its own static transforms, and do not override frame_id in odometry with a frame the driver does not know. The axis conventions that trip people up — optical versus body frames — are laid out in the IMU optical frame transform guide.

Fix it or disable it: the decision

With the flag false, odometry starts immediately and takes the first frame as the map reference: start the camera close to level and move slowly for the first few seconds, because the roll and pitch of node one come from that frame. Set the flag false and drop imu_topic together — a true flag with an unnamed topic is cause 1 all over again. When IMU messages do arrive later in the run, odometry uses them from that point on.

The arithmetic: what the wait costs, what the fix needs

A worked example with the numbers above, a 30 Hz camera and a united IMU topic at 200 Hz:

• 200 / 30 = 6.7 — six to seven IMU samples arrive between consecutive frames, dense enough that odometry can interpolate attitude across the frame interval.
• The warning fires about 15 times per second. Five minutes of waiting drops 15 × 300 = 4,500 synced RGB-D pairs, and with them 4,500 × 6.7 ≈ 30,000 IMU samples.
• Filter settle time before the quaternion is solid: 2 s × 200 Hz = 400 samples.

A two-second stationary hold after launch buys the entire pipeline. A misnamed topic costs the entire session — that asymmetry is the whole argument for the checks below.

Pre-launch checks and the one-minute confirmation

Before the next launch:

• rostopic hz on the exact topic you will pass shows a live rate — 100 Hz or better, 200 Hz typical.
• rostopic echo -n 1 on the same topic shows a non-zero orientation quaternion.
• Driver flags pinned in the launch file: enable_gyro, enable_accel, unite_imu_method, align_depth.
• The filter nodelet remaps onto the topic name odometry subscribes to.
• The tf graph connects camera_link to the IMU optical frame.
• The camera driver starts first; rtabmap.launch starts only after the rate shows.

After the fix, confirmation takes under a minute: the waiting warning stops within one sync cycle, rostopic hz /rtabmap/odom shows a live rate, and the cloud appears in rtabmapviz. If the feed runs but the map leans or swims over minutes, the gate is gone and you are chasing IMU quality instead — depth jitter and IMU drift troubleshooting covers that layer.

The same habit — check the topic before the algorithm — applies one step earlier when the camera itself publishes nothing; depth camera stuck at startup: check topic publishing first walks that case, and ROS stream enable settings covers driver outputs that stay silent until enabled.

When a fixed 3D sensor is the better answer

If the part can come to the camera, handheld SLAM buys mobility and pays for it with exactly the failure modes above: IMU gating, drift, loop closure. A stationary scan has no odometry loop to block. Structured-light and ToF sensors in our 3D sensor range capture a full point cloud per exposure from a tripod or gantry, so you choose by working distance and field of view rather than SLAM features. The 3D scanning method selection guide compares the methods, and 3D camera working distance and positioning sets the geometry.

Two type examples from the range: MV-DT01WMU, a ToF sensing camera, and MV-DPS200P-02, a projected structured-light 3D camera.

For more on this stack, the 3D vision topic page collects the guides, and the contact page reaches our applications team for help with your next build.