Cross-compile a depth camera SDK for an embedded ARM Linux board

A depth camera runs anywhere a Linux kernel runs, and that includes the ARM64 system-on-chip boards behind FPGA carriers, smart cabinets and compact cells. The blocker is never the silicon: it is that the SDK ships built for x86-64, and the board cannot run x86 binaries. This guide walks the cross-compile from source to board: pick the toolchain, write one CMake toolchain file, install into a staging tree, copy it over, and run a first depth capture. The flow fits any CMake-based depth camera SDK, including the 3D sensors we ship, and it ends with the bandwidth arithmetic that tells you whether the board can hold the stream.

What do you need before you start the cross-compile?

You need four things on the build host before the first configure: an aarch64 cross-compiler such as aarch64-linux-gnu-gcc, the kernel headers matching the board's kernel, the camera SDK's source from its vendor, and CMake 3.x with make or ninja installed.

Get each piece from the side that owns it. The cross toolchain comes from the build host's package manager: on a Debian-family host that is gcc-aarch64-linux-gnu and g++-aarch64-linux-gnu, one apt command. The kernel source comes from the board vendor, because the target kernel is not the host's. The SDK source comes from the camera vendor's download, and libusb should already exist in the target image; if it does not, the SDK can build its own copy, as the backend section below shows.

Still choosing the board itself? The guide on choosing an ARM board for USB3 camera acquisition puts USB 3, RAM and kernel version at the top of the list, which are the same three items this build needs.

The devices at the end of this chain are the depth cameras in the shop's 3D sensors range. The MV-DT01WMU ToF sensing camera and the MV-DP4180-03P 3D laser profile sensor both ship with vendor SDKs, so the same cross-build carries either one onto the aarch64 host.

Which USB backend does your kernel need?

The choice is set by the kernel version: on kernel 4.4 or newer the SDK's V4L2 backend works as shipped, while kernels older than 4.4 need the LibUVC backend, which you select at configure time with -DFORCE_LIBUVC=true.

Read the number off the board itself: uname -r prints it. A board reporting 4.19, 5.4 or 5.10 clears the bar; anything under 4.4 forces the LibUVC route. The two backends differ in where the USB work happens. V4L2 uses the kernel's UVC driver plus a small vendor patch that exposes per-frame metadata; LibUVC talks to the camera over libusb with no kernel driver involved.

When you do apply that kernel patch, it edits the target kernel's UVC driver source, so it belongs in the kernel build that produces the board image. It also applies cleanly only when the source tree matches the kernel version the patch expects; the UVC patch kernel mismatch guide walks that exact failure.

If the target image lacks libusb, add -DUSE_SYSTEM_LIBUSB=false and the SDK builds and bundles its own copy, still for aarch64 because it compiles inside the same project. Older depth camera generations carry kernel quirks of their own, and the legacy depth camera compatibility guide shows how to test the stack before you commit.

Which CMake settings point the build at the ARM toolchain?

A toolchain file is the clean answer: it declares CMAKE_SYSTEM_NAME Linux, CMAKE_SYSTEM_PROCESSOR aarch64 and the aarch64-linux-gnu-gcc / aarch64-linux-gnu-g++ pair as compilers, and CMake then uses them for every check and target in the project.

# arm64-toolchain.cmake
set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR aarch64)
set(CMAKE_C_COMPILER   aarch64-linux-gnu-gcc)
set(CMAKE_CXX_COMPILER aarch64-linux-gnu-g++)
set(CMAKE_FIND_ROOT_PATH /home/you/sysroot)  # a copy of the board's root filesystem
set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

The two SYSTEM lines flip CMake into cross mode, and the compiler lines name the binaries from the package you installed. CMAKE_FIND_ROOT_PATH confines every find_library and find_path search to the target sysroot, so the host's x86 OpenCV can never sneak into an ARM link. Copy the board's root filesystem to the host once with rsync -a root@board-ip:/ /home/you/sysroot and point the root path at it; dependency searches then resolve against exactly what the board has.

The same file drives any C++ SDK on this target, not only camera code; this walkthrough of configuring a C++ SDK for an ARM Linux target applies the identical variables to another vendor's source tree.

How do you keep the install out of the host and in a staging tree?

Set CMAKE_INSTALL_PREFIX to a staging directory such as /home/you/staging/usr/local, never to /usr, so make install lays the library, headers and examples out under that tree for copying later instead of scattering them into the build host's system paths.

The default is exactly what you do not want. Install without a prefix and the ARM binaries land in /usr/local on the x86 build host, where they are useless and hard to remove completely. A staging tree is one directory you can inspect, wipe and copy, and the install step fills it in the layout the board expects:

cmake --install ~/sdk-src/build-arm64
# fills ~/staging/usr/local with: bin  include  lib  share

With the Makefile generator you can reach the same layout from the other side: leave the prefix at /usr/local and run make install DESTDIR=/home/you/staging, which stages everything under staging/usr/local for a straight overlay onto / on the board. Either way, one rm -rf of the staging directory wipes a bad attempt cleanly, which a botched host /usr/local never allows.

What are the steps, in order, from source to board?

The whole path is six commands on the host and one copy to the board: install the toolchain, write the toolchain file, configure with it and the backend flag, build, install into the staging tree, then copy that tree into the board's root filesystem.

1. Install the cross toolchain on the build host: sudo apt install gcc-aarch64-linux-gnu g++-aarch64-linux-gnu.
2. Unpack the SDK source from the camera vendor's download into ~/sdk-src.
3. Read the target kernel on the board: uname -r. Kernel 4.4 or newer runs the V4L2 backend; anything older means adding -DFORCE_LIBUVC=true in step 5.
4. Write the toolchain file above to /home/you/arm64-toolchain.cmake.
5. Configure: cmake -S ~/sdk-src -B ~/sdk-src/build-arm64 -DCMAKE_TOOLCHAIN_FILE=/home/you/arm64-toolchain.cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/staging/usr/local
6. Build: cmake --build ~/sdk-src/build-arm64 -j$(nproc)
7. Install into the staging tree: cmake --install ~/sdk-src/build-arm64
8. Prove the architecture: file ~/staging/usr/local/lib/lib*.so*. Every line must report ARM aarch64; an x86-64 line means one dependency escaped the cross build.
9. Deploy: rsync -a ~/staging/ root@<board-ip>:/, then ssh root@<board-ip> ldconfig.

Two finishing moves on the board: run ldconfig so the linker cache picks up the new library, and if the image does not search /usr/local/lib, write that one path into a file under /etc/ld.so.conf.d and run ldconfig again. If the SDK installed udev rules inside the staging tree, copy those to /etc/udev/rules.d the same way. On a four-core desktop the library itself builds in minutes; the build takes longer when OpenCV or another heavy dependency must be cross-built first.

How much USB bandwidth does the depth stream need?

Budget roughly 55 MB/s for a 1280×720 depth stream at 30 fps with 16-bit pixels, which is why the camera must sit on a USB 3 port: USB 2 tops out near 35-40 MB/s in practice and will drop frames.

Worked example, depth plus color. Width times height: 1280 × 720 = 921,600 pixels. Two bytes each: 1,843,200 bytes per frame, about 1.84 MB. Thirty frames a second: 55,296,000 bytes per second, which is 55.3 MB/s or 442 Mbit/s. A color stream at 1280×720 with 3 bytes per pixel adds 921,600 × 3 × 30 = 82.9 MB/s, for about 138 MB/s in total.

Now compare against the ports. USB 2 offers 480 Mbit/s on paper and delivers roughly 35-40 MB/s for bulk streams, so the combined stream does not fit and even depth alone at 55.3 MB/s does not. USB 3 delivers about 400 MB/s in practice, leaving a factor of three of headroom. So the rule: mount the camera on a controller-native USB 3 port, not behind a hub sharing a USB 2 lane, and if only USB 2 exists, drop to 848×480 at 15 fps, which is 848 × 480 × 2 × 15 ÷ 1,000,000 ≈ 12.2 MB/s and fits.

When frames drop or the image goes black mid-stream anyway, suspect the port and its power delivery rather than your build; the depth camera black image guide covers the USB 2 bandwidth and power traps in full.

How do you verify the examples run on the board?

Confirm three things on the board before you trust a capture: uname -r prints 4.4 or newer, lsusb lists the camera's vendor and product ID, and one depth example runs for a minute without a 'no device' error.

Run the checks in that order. Plug the camera in and read dmesg: the kernel should log the device enumerating as UVC. lsusb prints the vendor and product ID pair your udev rule will need. Start one of the SDK's depth examples from ~/staging/usr/local/bin and watch the depth map move; a frame-rate readout near 30 fps confirms the port delivers the bandwidth the arithmetic above demands.

If lsusb sees the camera but the SDK insists there is no device, the cause is access, not detection: the user cannot open the device nodes. The USB camera not detected guide covers that split, and the SDK access errors guide isolates the permission layer underneath it.

Which build failures show up first on ARM?

Expect three recurring failures on an ARM build: undefined CMake targets for Boost when the toolchain file hides the target's libraries, -fPIC relocation errors when a static archive is linked into a shared library, and an OpenCV config path that CMake cannot find.

Each has a guide with the fix in full. Undefined Boost targets: cross mode hides the host's packages, so Boost must be built for aarch64 into the sysroot and found there; the CMake Boost targets guide untangles it. Relocation errors: a dependency was compiled without -fPIC and then linked into the shared library; the fPIC relocation guide shows the rebuild. A missing OpenCV config: install OpenCV into the sysroot and point CMAKE_FIND_ROOT_PATH at it; the OpenCV config path guide names the files CMake searches for.

One habit shortens all three hunts: build the SDK once for the host, unmodified, before the first cross build. If the host build fails the same way, the problem is a dependency, not your toolchain file.

When should an ARM board hand the job to a vision controller?

Hand the job over when the numbers stop closing: if the frame budget, RAM or sustained CPU load leaves no headroom at the line rate, the same camera moves to a fanless x86 vision controller that builds the SDK natively and adds built-in light channels and camera ports.

Put your own numbers on that test. At 30 fps the whole pipeline, capture through inspection, owns one frame for 1000 ÷ 30 = 33 ms; at 60 fps it is 16.7 ms. When processing eats 30 of the 33 ms, any scheduler hiccup drops a frame, and no build flag fixes a CPU that is out of time. RAM tells the same story: 2 GB boards fill fast once point clouds enter the pipeline.

The x86 exit costs no new code. Every SDK that cross-compiles with a toolchain file also builds natively on an x86-64 host, and the vision controllers range packages such a host with camera-facing I/O: the MV-VC2140-128G40-1T pairs an Intel N-series CPU with 8 GB memory, three or four LAN ports and 4 built-in light channels in a fanless box.

Choose the board when the cell needs the camera mounted far from the compute, when power draw is capped, or when the product is an embedded device in its own right. Choose the controller when the frame budget above is failing and the camera can sit next to the compute. If the target is not Linux at all but an RTOS, the porting path changes; the GigE Vision RTOS port guide covers cross-compiling and device discovery on that side.

Ready to pick the hardware? Filter depth cameras by type in the camera finder, browse more system-building guides on the systems topic page, or send the frame budget and target board through the contact page and we will size the pairing with you.