Depth camera node process died with exit code 127: fix it

A depth camera node that dies at launch with exit code 127 has usually not reached your code: the Linux dynamic loader stopped it while it resolved the shared libraries it needs. The launcher folds the whole failure into one line and hides the loader's message, so the instinct is to reinstall blindly and lose an afternoon. Work in this order instead: confirm what 127 means in your case, run the node binary by hand to surface the real error, trace the missing library or undefined symbol with ldd and nm, then fix the mismatch by matching SDK and wrapper versions on the controller. Every command here runs on a stock Linux shell, on x86 towers and ARM boards alike, and each one returns in seconds — an ldd run on a node with 50 dependencies finishes in well under 1 s.

What does exit code 127 mean when a camera node process dies at launch?

On Linux, 127 from a process the launcher actually started is the dynamic loader's failure code: either a needed shared library was missing at load time, or the first call hit an undefined symbol and the process aborted with a symbol lookup error.

Two messages cover the failure:

depth_node: error while loading shared libraries: libdepth_sdk.so.2:
  cannot open shared object file: No such file or directory

depth_node: symbol lookup error: depth_node: undefined symbol:
  _ZSt16__throw_bad_array_new_lengthv

Both names are illustrative; the loader prints your library's soname and your binary's path. The first message means the file was never found: the binary needs that soname and no installed library provides it. The second means the file loaded but the first call into it failed, the classic signature of two builds that disagree about an interface. A third case exists: a plain shell returns 127 for "command not found", covered near the end. The number alone cannot separate the three, so the error line decides.

Why does the launcher only print "process has died" with exit code 127?

The launch system captures the child process's standard error and folds it into that one-line exit report, so the loader's real message never reaches your terminal; run the node binary itself from a shell with the same environment and the cause prints on the first line.

The report quotes the full command it tried, binary path included. Open a terminal as the same user, load the same environment your launch file loads, and run that path by hand. The loader resolves libraries before the program's first line executes, so a bare run with no arguments reproduces the failure immediately — typically within 100 ms of pressing Enter. Add the launch arguments only when the loader passes and the node then fails on its own settings.

How do you find the missing library with ldd?

Run ldd on the node binary from the same shell you launch from: every dependency that prints "not found" is a library the loader could not resolve, and one such line is enough to explain the 127.

$ ldd ~/ws/install/depth_camera/lib/depth_camera/depth_node | grep "not found"
	libdepth_sdk.so.2 => not found

The binary name and path are illustrative; use the path from your launcher report. On a camera node, dozens of dependencies resolve and the one or two "not found" lines explain the launch failure. Cross-check the loader cache before installing anything: ldconfig -p | grep libdepth_sdk shows whether the system already knows a file under that soname. If the cache lists it but ldd misses it, your environment is at fault; if neither finds it, the SDK runtime was never installed or an update removed it. The Linux vision shared library errors guide walks this workflow for any vision SDK.

Why did a node that worked yesterday fail with an undefined symbol today?

Something in the environment moved: an SDK or OS update replaced a library the wrapper was built against, or another program's library path now shadows the system one, so the loader binds an old file that lacks the symbol the new code calls.

Four movers cover almost every case; check them in this order:

1. One side was updated. The camera SDK gained a version and the wrapper still calls the old interface, or a rebuilt wrapper calls symbols the installed SDK lacks. If you touched either side this week, rebuild the other to match.
2. A system update replaced a runtime. Distribution and board-support upgrades swap the C++ runtime and framework libraries underneath every existing build.
3. Another library shadows the system one. A stale LD_LIBRARY_PATH entry pointing into an old build directory, or an activated conda environment exporting its own older libstdc++.so.6, feeds the loader a file without the new symbols. Check echo $LD_LIBRARY_PATH in the exact shell that launches the node.
4. The loader cache is stale. An SDK installed into /usr/local/lib without sudo ldconfig stays invisible even though the file sits on disk.

Version alignment between SDK, wrapper and firmware is its own discipline; the SDK and firmware alignment guide covers the pairing, and the leftover-settings article shows how half-updated environments keep producing flaky runs.

How do you trace an undefined symbol back to the library that is too old?

The name in the error tells you which library family exported the missing symbol: demangle it, grep the loader cache and the candidate libraries with nm -D, and compare the highest version the installed library exports against the one the error asks for.

The most common real case is a C++ runtime tag. The node fails with version 'GLIBCXX_3.4.29' not found (required by ./depth_node), so ask the installed runtime for its highest exported tag:

$ nm -D /usr/lib/x86_64-linux-gnu/libstdc++.so.6 | grep GLIBCXX | sort | tail -1
GLIBCXX_3.4.28

The node wants 3.4.29 and the controller exports 3.4.28: 3.4.29 − 3.4.28 = 1 minor revision behind the compiler that built the node. That single revision is the entire failure. Update the distribution's C++ runtime package, run sudo ldconfig, and relaunch.

For a mangled C++ symbol, pipe it through c++filt to read it in plain form, then run nm -D on each candidate library and grep the fragment to find which installed file exports it. No exporter anywhere means the library is missing rather than old: install instead of update. On ARM boards the system path carries an aarch64 triplet instead of x86_64; the commands do not change.

How do you fix the mismatch so the node launches again?

Match the two sides: install the missing library from its official package, refresh the loader cache with sudo ldconfig, point LD_LIBRARY_PATH at the SDK you actually built against, and when the wrapper predates the SDK, rebuild the wrapper from source in the same workspace.

Work down this list until the launch sticks:

1. Stale cache: sudo ldconfig, then relaunch. It costs seconds and is worth trying first after any manual install.
2. Missing library: install the SDK runtime package built for your OS version, from the SDK's official distribution, then sudo ldconfig.
3. Shadowing: correct the export in the shell profile or launch file so LD_LIBRARY_PATH points at the SDK you built against, and deactivate conda in the controller user's shell.
4. Wrapper older than the SDK: rebuild the wrapper and its workspace, colcon build in a ROS 2 workspace, so the compiled calls match the installed symbols.
5. System upgraded under the build: reinstall the SDK release built for the new OS, then rebuild the wrapper against it.

If the rebuild itself fails with undefined references at link time, that is the build-time twin of this launch-time failure; the C++ linker error guide covers that side.

When is exit code 127 not a library problem?

When the binary was never found: a shell returns 127 for "command not found" and 126 for "found but not executable", so a cmd path that does not exist or lost its execute bit produces the same number for a completely different reason.

Check the simple possibilities before touching libraries. Run ls -l on the path from the report: one line answers existence and the execute bit, and a lost x after a file copy shows up immediately. After a manual run that fails, echo $? prints the number yourself. One subtlety remains: a program chooses its own exit codes, so an SDK node can exit 127 for internal reasons of its own. The stderr line, never the number, is the evidence. The same number reaches other industrial software with unrelated causes; why a WinSCP transfer reports exit code 127 on a panel PC is one worked example.

How do you keep the node launching after every SDK update?

Rebuild the entire workspace whenever the SDK changes, keep one version of each camera library on the controller, and save the ldd output of a healthy build as a baseline so the next mismatch is a one-line diff instead of a morning of guessing.

• After any SDK or wrapper change, rebuild the whole workspace, not only the package that changed; mixed build artifacts are what produce missing symbols.
• Keep exactly one version of each camera runtime installed: remove the old release before installing the new one, then run sudo ldconfig.
• Keep conda out of the controller user's profile; develop inside the environment, deploy on the system runtime.
• Save the baseline right after a good launch, ldd ~/ws/install/depth_camera/lib/depth_camera/depth_node > ~/ldd-baseline.txt, and diff it after the next update; one changed line is a five-minute fix, and a diff of the two files runs in about 1 s even for a 60-line dependency list.
• Gate every launch: ldd ~/ws/install/depth_camera/lib/depth_camera/depth_node 2>&1 | grep -c "not found" must print 0 before the node starts.

Once the node launches, controller-level reachability is the next common wall, and the not-pingable 3D camera guide covers subnet and link faults from there.

If library faults keep costing you mornings, or you are standardizing the station anyway, choose the hardware deliberately: browse the 3D sensors category, size the compute side through the vision controllers category, and contact us for a second pair of eyes on your ldd output — expert support comes with every order.