Read IMU data from a 3D camera through its SDK, not ROS

Many fixed-mount 3D cameras carry two sensors nobody reads: an accelerometer and a gyro. They sample the same shocks, vibration and rotation the image sensor sees, which makes them useful for pose smoothing, mount-stability checks and robot-cell diagnostics. Most teams never read them because a middleware driver node gets installed first, and that node hands out motion data on its own schedule with its own timestamps. This guide shows the direct path in five steps: enumerate the motion streams, enable them, read them at native rate from your own thread, stamp each sample with the device clock, and prove the pipeline with a ten-second count. The pattern fits any depth camera whose SDK exposes motion streams.

Why read motion through the SDK instead of a driver node

The decision is simple: enable the motion streams in your own process and read them like image frames, only faster. Motion data typically runs at 100-400 Hz per stream - roughly ten to thirty times a 30 fps camera - and Step 1 reads the exact rate each stream declares. A node built around frame-rate publishing either resamples that firehose or queues it behind images, and either way you receive someone else's timing. Reading direct buys three things: the native sample rate, the device timestamp assigned at the sensor, and single ownership of the camera.

If you later fuse motion with frames, the sensor-side timestamp is what keeps the pair aligned; when ordering goes wrong anyway, the fix starts in frames and IMU data arriving out of order.

What you need

• The camera SDK your image code already uses, installed and able to open the device.
• A camera with motion sensors. Stereo, structured-light and ToF models differ here, and a profiling sensor streams profiles and encoder positions instead, so confirm by enumeration (Step 1). If the list shows no accelerometer or gyro, no SDK call will create one.
• Exclusive ownership: stop any driver node before your process opens the device.

The fixed-mount structured-light and ToF classes below are the camera side of this pipeline; both stream through their SDK the way the steps assume:

The whole job is five steps:

1. Enumerate the device's streams and find the accelerometer and gyro entries.
2. Enable both motion streams next to your depth stream.
3. Read motion on its own thread, or through a per-stream queue sized to the rate ratio.
4. Stamp each sample with the device timestamp at arrival.
5. Verify with a ten-second callback count.

Step 1 - Enumerate the streams before enabling anything

Query the device's stream list the same way you would before configuring a depth stream. Beside the depth and color entries you will find, when the hardware carries motion sensors, an accelerometer entry and a gyro entry. Read what each motion profile declares - sample rate, units, sample width - before writing any reading code, because the declared rate is nominal and Step 5 measures the real one. The enumeration is also where you learn what a profiling sensor offers: its stream list is built around profiles and encoder ticks rather than inertial samples.

Step 2 - Enable the accelerometer and gyro like any other stream

Motion streams are enabled on the same pipeline object as your depth stream, as two additional named profiles; the depth configuration itself does not change. Enable both sensors even if you think you need one: a gyro without its accelerometer loses the gravity reference that limits drift. Data volume stays small - a motion sample is a few dozen bytes, so a few hundred samples per second is noise next to a depth stream. If the depth frame rate dips after you enable motion streams, suspect trigger mode and data-path settings, not the motion data; the frame-rate guide to trigger mode and data streams walks that fault down.

Step 3 - Read motion at its own rate, not the image rate

This is the step teams get wrong. Motion and images run on different clocks: depth at 30 fps means a frame every 33.3 ms, while a 200 Hz gyro delivers a sample every 5.0 ms (1000 / 200). Read motion only inside the depth callback and you service the motion queue 30 times per second against 200 arrivals: you drop 1 - 30/200 = 85 percent of the gyro and 1 - 30/100 = 70 percent of the accelerometer, silently, because nothing raises an error.

Read motion on a dedicated thread, or poll a per-stream queue sized to the ratio. The thread is simpler and the load is trivial: gyro 200 + accelerometer 100 + depth 30 = 330 stream events per second. Polling from the main loop instead works if you size the queue honestly - integer math floors the ratio (200 // 30 = 6), so the margin matters:

# Illustrative names - your SDK's calls differ; the pattern does not.
gyro_hz = 200 # measured rate, see Step 5
loop_hz = 30 # your depth loop
queue_len = gyro_hz // loop_hz + 2 # 200 // 30 = 6, + 2 -> 8 samples of headroom
counts = {"gyro": 0, "accel": 0, "depth": 0} # test counters, start at 0

Step 4 - Stamp each sample with the device clock

Every motion sample carries a timestamp assigned at the sensor next to its data. Use it, not your process's arrival time. Host arrival carries operating-system scheduling jitter of milliseconds, and at 200 Hz the entire sample period is 5 ms - one millisecond of jitter is a fifth of a period, enough to scramble sample order and poison alignment with frames. The same rule holds for images, which is why frame pairing belongs on device timestamps rather than queue order: sync RGB and depth frames by device timestamp shows the data layout. From there, aligning samples to frame times is interpolation work - the IMU and depth timestamp alignment guide has the math - and fusing both into a pose is state estimation, where a worked Kalman filter for IMU and camera fusion earns its keep. To tune offline, record streams to file: H5 recording for offline processing keeps frames and motion together in one place.

Step 5 - Prove it with a ten-second count

Run the pipeline for 10.0 s with the counters from Step 3 and check the numbers:

1. Gyro at 200 Hz: 200 × 10 = 2,000 callbacks; accept 1,980-2,020, because ±1 percent covers clock tolerance.
2. Accelerometer at 100 Hz: 100 × 10 = 1,000 callbacks.
3. Depth at 30 fps: 30 × 10 = 300 frames.
4. Mean gyro spacing: 10.0 s / 2,000 = 5.0 ms; if you measure 5.1 ms, your real rate is 1000 / 5.1 = 196 Hz, and that measured value is what your queue and filter math should use.

Then check physics on the bench. Camera flat and still, the vertical accelerometer axis reads about 9.81 m/s² (or 1.0 g, per the profile's declared unit) and the gyro reads near zero. Rotate the camera through 90° in about a second: the gyro integral, the sum of ω·Δt, should land near 1.57 rad (90° × π / 180). A gyro counter stuck near 300 means you are counting depth frames - the motion thread is not running. Counts near zero mean the stream never enabled. If counts wander between runs, start every run from a known device state; the flaky-test fix shows the reset sequence.

Common mistakes

• Reading motion inside the image callback. The 85 percent gyro loss produces smooth-looking but aliased angular rates; the symptom is a vibration signature that appears and disappears with your loop timing.
• Timestamping at host arrival. Order scrambles and alignment drifts; Step 4 is the whole fix.
• Trusting the nominal rate. Queue depths and filter gains come from the measured mean spacing.
• Leaving the driver node half-running. Two claimants on one device produce intermittent open failures that look like hardware faults.
• Killing the process without stopping streams. Streams left open across runs are why an SDK hang on exit happens; close them in the documented order.
• Chasing jitter that is not in your reader. If depth still jitters while orientation drifts on a clean pipeline, stereo depth jitter and IMU drift guide works the problem from the sensor side.

When the middleware node is still the right choice

Keep the node when your control stack already consumes its messages at a rate the application accepts and you have no use for sensor-side timestamps. If your pose filter runs at 30 Hz anyway, the extra 170 gyro samples per second (200 published minus 30 used) buy nothing. The SDK path earns its keep when you need every sample, honest timestamps, or a camera shared with non-middleware code, which describes most precision and robot-mount work. And when you keep the node and it will not connect, that is a separate fault with its own fix list: the ROS node handshake guide covers discovery and stream enables.

Pick the camera, then start the pipeline

The pattern needs one thing from the hardware: a depth camera that streams through an SDK and, for pose work, motion sensors behind it. The 3D sensors category holds the fixed-mount stereo, structured-light and ToF classes, the camera finder filters them by type and series, and the configurator below matches a camera to the rest of the cell. Tell us the model you are considering through the contact page and we will read its data sheet with you before you order. More depth-camera fixes sit on the 3D topic page.