Indoor navigation approach: stereo camera on old embedded hardware

You have a small vehicle, an aging embedded GPU board and a stereo depth camera, and one job: drive a fixed indoor route — start point, through a yellow door if it is open, down an aisle, turn left, stop at the elevator. No exploring, no new buildings, no printed codes. The question is whether to spend months on deep learning or to wire up a localization stack you fear is mostly glue. This comparison gives the decision up front, the compute and depth numbers behind it, a worked example that sizes the camera to the route, and the waypoint logic that reaches the elevator. You will leave able to choose the approach, the camera and the lens from arithmetic rather than from whichever library looked more interesting.

The decision: odometry and landmarks, not a CNN

Use sparse stereo visual odometry fused with the vehicle's IMU and wheel odometry, an HSV color check for the yellow door, and depth thresholds for the open/closed test and obstacle stops. Do not train a CNN for this route. The rule behind it: a known map and a repeatable route reduce navigation to dead reckoning between engineered landmarks, and that is a CPU job. Deep learning earns its cost only when the scene is unknown or the landmarks cannot be engineered.

Three cases, three answers:

• Known map, fixed route (this project): stereo odometry plus landmark re-fixes, all on the CPU at reduced resolution.
• Known map, routes that change: localization against a saved occupancy map — particle-filter localization for crowded indoor halls is the upgrade, not a new approach.
• Unknown environment, unlabeled clutter: learned detection, on hardware with the GPU for it.

Retire one fear early: the classical route is not just middleware plumbing. A waypoint loop is a couple of hundred lines of code, and inside it you do real vision work — calibration, rectification, feature matching, filtering. The mapping half of a full SLAM stack is the part you skip.

The compute budget that decides it

Compute, not cleverness, settles the CNN question. Published stereo visual-inertial odometry systems run in real time on a single desktop-class CPU; one such system reports real-time operation on a 2.9 GHz server processor. A 2014-era mobile CPU delivers a fraction of that throughput — but a fraction is enough when the pipeline is sparse. Current deep-learning sample collections tell the opposite story: they target the current GPU generation, on a first-generation module many of them will not run, and a compact detector that does run lands in the low single-digit frames per second.

Run the pixel arithmetic on the route's own frame size:

• Full frames: 1280 × 1024 = 1.31 MP per sensor, 2.62 MP per stereo frame, 78.6 MP/s of rectify-and-match work at 30 fps. Not on this board.
• Quarter size: 640 × 512 = 0.33 MP per sensor, 0.66 MP per frame, 13.1 MP/s at 20 Hz — one sixth of the load, and the door still spans 72 px at 8 m, as shown below.

Commit to 640-wide input at 15–20 Hz and the old GPU sits idle, which is what you want on a board its GPU cannot help. The CNN frame rate ends the debate on its own: at 1 fps and 0.5 m/s the vehicle travels 0.5 m between detections. It cannot stop at a door, let alone thread one. If a later stage needs detection on CPU-class compute, object detection without a GPU covers what survives there; this route does not need it.

What odometry costs you: drift

Stereo odometry drifts as a fraction of distance travelled; plan on 0.5–2 % over a short indoor run. On a 30 m route that is 0.15–0.6 m of error by the elevator, and 3–12 cm over the 6 m aisle leg. Two consequences shape the state machine below: no leg may end on an odometry count alone, and every leg ends on a sensor event instead — a color blob, a depth plane, a wall entering the frame. For what each camera type contributes to a pose estimate, see visual odometry vs depth mapping. Fuse the camera's translation with the attitude and wheel speed your autopilot board already streams over its serial link; when the fused track wanders, stereo depth jitter paired with IMU drift is the combination to debug first.

Sizing the camera to the route: a worked example

Worked inputs you can swap for your own parts: a 1280 × 1024 stereo pair with 3.45 μm pixels, a 2.2 mm f/2.0 lens giving about 90° horizontal field of view, a 60 mm baseline, a 900 mm wide yellow door, a 30 m route and a 0.5 m/s speed cap.

• Focal length in pixels: 640 / tan 45° = 640 px. On 3.45 μm pixels that is 640 × 3.45 μm = 2.21 mm, so a 2.2 mm lens. The lens focal length calculator runs this trade in both directions.

• Scene width: at 90° horizontal FOV the visible width is twice the stand-off — 8 m of corridor at a 4 m approach, with both door jambs in frame and margin to spare.

• The door in pixels: 900 × 640 / 4000 = 144 px wide at 4 m; at first sight from 8 m, 900 × 640 / 8000 = 72 px. Both are solid color blobs, not thin lines.
• Depth and its error at 4 m: disparity d = 640 × 60 / 4000 = 9.6 px; with 0.2 px of sub-pixel matching error, ΔZ = 4000² × 0.2 / (640 × 60) = 83 mm. At 8 m the same sum gives 333 mm. Make every stop-and-pass decision inside about 4 m, where depth error stays under a decimetre.
• The approach stop at 1.5 m: ΔZ = 1500² × 0.2 / 38 400 = 12 mm. Centimetre-grade, enough to stop square at a doorway.
• Motion blur: at 0.5 m/s with a 5 ms exposure and 3 m look-ahead, blur = 0.5 × 0.005 × 640 / 3 = 0.53 px. Fix the exposure at 5 ms, hold f/2.0, cap the gain.
• Drift budget: 30 m × 1 % = 0.3 m — already priced into the leg logic above.

Scaling in one line each: halve the baseline and every ΔZ doubles; double the distance and ΔZ quadruples. That quadratic is what long-range stereo depth accuracy with a wide baseline works through, and it is why distance deforms stereo depth edges at the door frame.

The route as a state machine

Six states, and every transition is a sensor test — never a timer:

1. SEEK: drive forward on the fused heading until a yellow blob passes the color, size and depth gates.
2. APPROACH: steer to center the blob; stop when the door plane's median depth reads 1.5 m.
3. CHECK: median depth inside the door rectangle under 5 m means closed — wait. Readings of 6 m and beyond across most of the rectangle mean open. Ignore the outer columns of the rectangle: at an open door's edge, matches average door and corridor into a false mid-range, the flying pixels between object and background artifact.
4. CROSS: track the two depth edges of the doorway and keep them symmetric; the passage walls correct the heading continuously through the crossing.
5. AISLE: count 6.0 m of travel; while counting, stop if median forward depth falls inside 0.8 m — the rule depth change detection with a moving camera formalizes for clutter between waypoints.
6. TURN: 90° left at reduced speed on the fused heading; end the route when the facing wall rises into the lower depth band. The wall is the last landmark — the stop comes from depth, not from the odometer.

The approaches compared

Read the table as a build order: row one is the route, row two is the first upgrade when routes multiply, row five is a per-leg supplement. Rows three and four cost setup months this project does not have.

When deep learning is the right answer

Learned perception is correct when the environment is unknown, the clutter is unlabeled, or landmarks cannot be engineered into the scene. A published study of safe visual navigation demonstrated novelty-aware learned driving on a fast RC car — with a desktop-class GPU aboard. Two of those conditions fail here, and the third is the hardware itself. The division of labor survives every later upgrade: networks classify what is in the scene, odometry still says where you are. When a rebuild moves to current-generation compute, revisit choosing CPU or GPU for a deep learning set-up — the pose backbone stays classical either way.

Choosing the stereo hardware

The LS-BCM-0043 binocular module is the one-board route: 1.3 MP at 1280 x 1024, global shutter, 1/2" sensors, M12 lenses, USB3.

One board fixes the pair's geometry and shares one exposure domain — the two things a hand-built rig gets wrong.

To roll your own, use a pair of LS-BCM-0003 boards: 3.1 MP, 1/1.8" global shutter, 120 fps, C-mount, USB3.

Wire the trigger lines together so both sensors expose on the same edge, set the 60 mm baseline on a machined spacer, and let rectification calibrate out the residual mounting error.

For halls of glass and bare concrete where passive stereo starves, active sensing closes the texture gap: the MV-DLS300P-04 laser galvanometer stereo camera and the MV-DPS200P-02 projected structured-light 3D camera.

Match each unit's rated range against the 4 m decision distance before making one the primary sensor; this class is typically built for close, dense profiling rather than corridor-length ranging. Browse stereo and structured-light 3D sensors and board-level cameras, or start from the camera finder — it spans area scan cameras and board-level parts alike — then run the system configurator from the camera step.

Checks before you build

• Global shutter with hardware sync on both sensors; rolling shutter smears matched features at 0.5 m/s.
• A 60 mm or wider baseline if door decisions run at 4 m — halving the baseline doubles every depth error.
• A lens near 90° horizontal FOV so the doorway stays framed through the turn; f/2.0 with the fixed 5 ms exposure.
• Intrinsics and stereo rectification calibrated before anything moves — geometric camera calibration — because every number above assumes a rectified pair.
• Recorded runs to replay offline: capture with a stereo camera recording workflow for SLAM, then apply scale-aware SLAM parameter tuning.
• The autopilot's IMU and wheel odometry fused with camera translation, never vision-only scale.
• If one printed marker is ever allowed, a 100 mm fiducial at 2 m spans 100 × 640 / 2000 = 32 px — centimetre re-fixes for an otherwise unmarked leg.
• The 0.5 m/s cap carried into the control code, and a physical stop switch within reach while the vehicle runs.