Unstable IMU readings from a depth camera: calibrate and filter

You bolted a 3D camera with a built-in IMU to a bench, touched nothing, and watched the z value climb every second. The camera is almost certainly healthy: a MEMS inertial sensor always reports gravity plus a small constant bias plus noise, and any integration of that bias becomes a straight-line ramp. This guide works the diagnosis in order of probability: confirm what the climbing number really is, run the vendor's IMU calibration, switch on motion correction, filter what remains, and mount the camera so the environment stops feeding it. The arithmetic behind the drift, the residual to expect after calibration, and the acceptance checks are all here, with numbers you can reproduce on your own bench.

The decision, up front

A climbing z value on a still camera is bias being integrated, not a failing sensor. Calibrate the IMU once with the vendor's tool, keep motion correction enabled, high-pass or fuse before you integrate anything, and hold the camera on a rigid mount. After a good calibration, plan for 0.1 to 0.5 m/s of z-velocity error per minute of integration, which is the 0.2 to 0.8 mg of residual bias a good fit leaves behind. Raw, uncalibrated data from the same production line drifts 10 to 60 times faster. If the climbing stream is the gyroscope rather than the accelerometer, the identical arithmetic lands in angle: a 0.01 rad/s bias is 0.57°/s, which is 34° per minute of attitude error.

Why a still sensor cannot read zero

An accelerometer reports specific force: gravity, plus a constant bias from manufacturing tolerance, plus broadband noise. Held perfectly still, the ideal reading is gravity alone — 9.81 m/s² on whichever axis points down, with the sign set by the frame convention. Noise averages toward zero over seconds; bias never does. Integrate it and you get a ramp in velocity and a parabola in position:

Consumer MEMS units ship with anything from a few mg to tens of mg of bias, and the value moves with temperature. Even research systems that recalibrate inertial sensors online still model a slowly varying bias term, so treat yours as time-varying too, not as a constant you removed once. The gyroscope follows the same law in angle: angle_error = gyro_bias · t.

Causes in order of probability

Work this table top to bottom; the first two rows cover most cameras that arrive with this symptom.

The first row is the most common by far: code integrates acceleration to get velocity, sees a clean linear climb, and calls the sensor unstable. The sensor is doing its job; the arithmetic is the fault.

Run the IMU calibration

1. Clamp the camera rigidly, on the mount you will actually use in production.
2. Run the vendor's Python tool, rs-imu-calibration.py, from the tools folder of the SDK source tree; the vendor's IMU calibration whitepaper documents the method behind it.
3. Follow the on-screen prompts: hold the camera still in each orientation the tool asks for — several static positions in total — and let it average gravity in each one.
4. Accept the fitted coefficients. The tool writes per-sensor text files with the accelerometer and gyroscope numbers; file them with the machine's documentation.
5. Re-run the 60 s static capture from the checks below before changing anything else.

When the tool reports a good fit, stop. Re-running a successful calibration rarely improves the coefficients, and a rushed session can produce a worse set than the one you had.

Switch on motion correction

Calibration is half the fix; the stream has to use it. In the vendor viewer, open the options side-panel, find the Motion Module section, expand the Controls category, and set Enable Motion Correction so the option is active. With it enabled, the stored IMU calibration is applied to the motion frames before they reach you. Calibrate and then read the stream with the option left off, and you are looking at the same raw climb you started with.

Re-check the toggle after every viewer or SDK update; settings do not always survive an update.

Worked example: what 0.5 mg costs

Take a well-calibrated camera with a residual z bias of 0.5 mg — mid-range for a good consumer unit:

Now the uncalibrated sibling off the same line, with a 30 mg bias:

That is the whole symptom in four lines: one second of uncorrected bias already adds 0.29 m/s.

Filter what calibration cannot remove

Calibration removes the constant part of the bias. It cannot remove noise, nor a bias that keeps shifting with temperature, so filter for what you actually need:

High-pass for velocity and vibration. Any metric built on integrating acceleration — vibration energy, shock detection, conveyor tremor — should pass the accelerometer through a high-pass with a cutoff between 0.2 and 1 Hz first. A bias is a DC term; the filter removes it before it can become a ramp.

Complementary filter for tilt. The gyro is accurate over milliseconds and drifts over minutes; the accelerometer's gravity vector is the opposite. Combine them:

With a 200 Hz motion stream, dt = 5 ms. For a 1 Hz crossover, τ = 1 / (2π × 1) = 0.159 s, so α = 0.159 / (0.159 + 0.005) = 0.97: the gyro carries fast rotation, the accelerometer slowly pulls the angle back to true gravity. Filter after calibration, never instead of it. For full pose fusion, implementing a Kalman filter for IMU and camera fusion is the next step up from this one-liner.

Make the mount part of the fix

Bias is physics; vibration is environment. A camera clamped to a sheet-metal guard or a bracket on a running conveyor reads fan and motor motion that no calibration removes, because it is real acceleration. Set the camera on a rigid, heavy stand — the camera stands range is built for exactly this. The LS-MNT-0040 machine vision stand has a height of 700 mm and a 400 x 450 mm base, a footprint floor tremor cannot rock. Set the position against the 3D camera working distance and positioning guide so mechanical stability and framing do not fight each other.

Keep the cable out of the robot envelope: a struck camera turns a calibration session into downtime. For the full mechanical set-up, start at the mounting step of the system configurator or work through the accessories finder.

When the usual answer is wrong

Calibration done, motion correction on, and the stream still misbehaves? The bias was never the whole story:

• The value jumps or repeats instead of climbing smoothly. That is dropped frames or timestamp glitches, not bias. Read camera IMU timestamp clock domains at lower frame rates and check for missed motion frames.
• An axis or a sign flips after a software update. The stream may now be expressed in a different frame; settle the IMU optical frame transform before touching calibration again.
• Calibration is good but fused poses are still off. The camera-to-IMU extrinsics dominate the error budget; camera-IMU extrinsics: factory values versus your own calibration covers when the stored values suffice and when to measure your own.
• The depth image jitters along with the IMU. Treat it as a scene problem: depth noise removal filter not supported covers one common failure, and stereo depth jitter and IMU drift troubleshooting goes deeper on the pairing.

Checks before you trust the stream

• 60 s static capture at operating temperature: each axis mean within about 1 mg (0.01 m/s²) of its gravity component.
• Integrated z velocity climbs by no more than 0.5 m/s over that minute; 0.1 to 0.5 m/s is the band a good calibration leaves.
• Gyroscope mean below 0.01 rad/s on every axis at rest.
• Motion correction confirmed active after every viewer or SDK update.
• The capture repeated after 20 min of warm-up shifts each mean by no more than a few mg.
• Camera on the rigid stand, machinery running, noise floor unchanged from the quiet bench.

Whatever ranging method sits in front of the IMU — laser galvanometer stereo, projected structured light, or time of flight — the diagnostics above are identical. The 3D camera finder filters 3D sensors by type, and models such as the MV-DLS300P-04, the MV-DPS200P-02 and the MV-DT01WMU span those three types:

If the climb survives calibration, motion correction, a high-pass and a rigid stand, capture 60 s of raw motion data together with the calibration files and send them through the contact page — free expert support with every order. More depth-camera set-up guides sit on the 3D vision topic page.