3D visionHow-to

Build a 3D camera SDK with CUDA in Docker: what the GPU covers

What CUDA accelerates in a 3D camera SDK, why x86 packages drop it, and how to build the SDK with the GPU flag and pass the camera into Docker.

LumaSpec Vision9 min read Share on LinkedIn

On this page 11 sections
  1. 01What CUDA accelerates — and what it never touches
  2. 02Why the packaged x86 SDK has no CUDA
  3. 03The bytes the GPU saves you
  4. 04What you need
  5. 05Build the SDK with the GPU flag
  6. 06Run the container: GPU and camera, no privileged mode
  7. 07Check that the CUDA path is live
  8. 08Common mistakes
  9. 09When the CUDA build is the wrong answer
  10. 10Pre-flight checklist
  11. 11Questions

You containerized the SDK for a stereo 3D camera and hit the question that decides the whole Dockerfile: what does CUDA actually accelerate, and is the packaged SDK good enough? On x86 hosts the packaged install has no GPU support at all, so you compile the SDK from source inside the image with the GPU flag on, then run the container with the GPU runtime and the camera's device nodes mapped in — without privileged mode. This how-to gives the three stages CUDA covers, the build flags, the run command, and the checks that prove the kernels are live before the station ships.

What CUDA accelerates — and what it never touches

Stereo matching runs on the camera's own processor. Depth frames arrive over USB or GigE whether your host has a GPU or not, and no host-side build changes the camera's depth rate. CUDA support in the SDK replaces three conversions that otherwise run on the host CPU:

  • Point-cloud generation — turning each depth pixel into an XYZ point from the intrinsics.
  • Depth-to-color alignment — projecting the depth frame onto the color camera's viewpoint.
  • YUY-to-RGB conversion — unpacking the color stream's YUY format into RGB pixels.

Nothing else in the pipeline changes. If you fuse depth with color and infrared intensity into one point cloud, alignment is the stage where you feel the difference, because it touches every pixel of both frames on every frame.

Why the packaged x86 SDK has no CUDA

The SDK's apt repository for x86 and x64 hosts ships its utility and development packages without CUDA support. Everything still works after that install, which is what makes it easy to miss: your point clouds arrive, but the CPU does all three conversions. On embedded GPU module boards the situation is inverted — the packaged install has CUDA enabled by default, so a source rebuild there buys nothing. For GPU-accelerated image and video work on those boards, FFmpeg GPU hardware acceleration setup covers the same host-side principle.

The bytes the GPU saves you

Take a mid-range stereo depth mode: 1280 × 720 depth plus a color stream, 30 frames per second. Point-cloud generation turns each depth frame into three 32-bit floats per pixel.

points = width × height xyz_bytes = points × 3 channels × 4 bytes MB_per_s = xyz_bytes × fps ÷ 1,000,000
  • 1280 × 720 = 921,600 points per frame
  • 921,600 × 12 B = 11,059,200 B ≈ 11.1 MB per frame
  • 11.06 MB × 30 fps ≈ 332 MB/s of XYZ data the CPU writes with no GPU help
  • The color stream adds 921,600 × 2 B = 1.84 MB of YUY in and 921,600 × 3 B = 2.76 MB of RGB out per frame, ≈ 83 MB/s written at 30 fps

Roughly 0.4 GB per second of memory traffic sits in those two conversions. The CUDA build does not shrink a byte of it; it moves the work off the CPU. Decision: if your application consumes aligned point clouds continuously, build with CUDA. If it reads depth frames only, a CPU build is enough and the container gets simpler.

What you need

  • A host with a CUDA-capable GPU and a driver new enough for the CUDA version in your base image
  • A CUDA devel base image — devel, not runtime, because the build needs the compiler
  • The SDK source tree at a pinned release
  • The camera, a strain-relieved cable and, for laser models, guarding that keeps the beam inside the machine

Structured-light, stereo and ToF cameras from our 3D sensors range pass into a container the same way, because the SDK stage is host-side — the camera choice stays free:

3D sensorsHikrobot MV-DLS300P-04 Laser galvanometer stereo cameraMV-DLS300P-04
Type
Stereo / structured light
$16,579.00 · quotedView and quote
3D sensorsHikrobot MV-DT01WMU ToF sensing cameraMV-DT01WMU
Type
Depth / ToF camera
Price on requestView and quote

Build the SDK with the GPU flag

  1. Pin the base image to a CUDA devel tag on a release the SDK supports, for example nvidia/cuda:12.5.0-devel-ubuntu22.04. Keep the container on that base even when the host runs a newer distribution: forced-USB-backend source builds have failed on 24.04 hosts while building clean on the 22.04 base.
  2. Install the build prerequisites: apt-get install -y cmake build-essential git pkg-config wget unzip libusb-1.0-0-dev libgtk-3-dev libssl-dev libopencv-dev
  3. Clone the SDK source and check out a tagged release, so the headers, the library you link against and any host-side install agree. Mixed versions inside and outside the image cause exactly the failures covered when you pin one library version everywhere.
  4. Configure with the GPU flag on:
cd <sdk-source>
mkdir build && cd build
cmake .. -DBUILD_WITH_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DFORCE_RSUSB_BACKEND=ON -DBUILD_EXAMPLES=ON

-DBUILD_WITH_CUDA=ON compiles the kernels. -DCMAKE_CUDA_ARCHITECTURES=86 matches a common desktop compute capability; set it to your card's value. -DFORCE_RSUSB_BACKEND=ON forces the SDK's own userspace USB backend instead of the system one — the right switch when your distribution is newer than the packaged backends support.

  1. Build and install: make -j$(nproc) && make install && ldconfig. Parallel builds inside a memory-capped container can die halfway through; if that happens, lower the job count — fixing SDK parallel build failures in Docker walks through the same failure on another SDK.

Run the container: GPU and camera, no privileged mode

  1. Start the container with both GPU switches:
docker run --rm -it --runtime=nvidia --gpus all --device /dev/bus/usb <image>

--gpus all schedules the cards; --runtime=nvidia mounts the driver stack into the container. When the card does not show up, make CUDA visible to a depth camera container covers the missing runtime piece.

  1. Map the camera by device, not by privilege. --device /dev/bus/usb maps the whole USB subtree, so the camera survives re-enumeration. For per-device control, write a rule that gives the camera a stable symlink, as in udev rules for USB cameras on Linux, and map that node — or declare the camera in a CDI spec and let the runtime wire it up.
  2. For the color stream's V4L2 device, pass the V4L2 media nodes, not just /dev/video0; mapping the video node alone drops the linked media controllers.
  3. On Docker Desktop hosts, USB devices cannot be passed through directly at all: run a small host-side service that remaps the device node, and bind the container to that node.
  4. Privileged mode does make the camera appear with zero configuration, and it hands the container every device, bus and kernel interface on the host. Refuse it anywhere the image leaves a developer's desk.

Check that the CUDA path is live

  • nvidia-smi inside the container lists the card and the driver version.
  • The SDK's device enumeration reports the CUDA compute capability next to the camera.
  • Run the point-cloud example against a recorded frame with the GPU build, then with a CPU-only build of the same release. Identical pixel values tell you quality is untouched; host CPU utilization during aligned-color streaming drops with the kernels on.

If the container's point cloud renders black while the host pipeline works, the GPU runtime passthrough is the layer to fix: point cloud black in a container is that exact case.

Common mistakes

  • Runtime base image. The runtime CUDA tags carry no compiler, and the configure step fails on the missing nvcc. Use a devel tag.
  • Packaged SDK shadowing the source build. An apt install of the SDK's packages inside the image puts a CPU-only library on the default linker path. Remove it before make install, or link explicitly against your build.
  • Camera disappears after hours. A --device flag bound to one /dev/video node breaks the moment the bus re-enumerates. Map the subtree, turn USB autosuspend off for the camera on the host, and take the mechanical side seriously: a drag-chain-rated cable holds the connector still while the machine moves. The LS-CBL-0219 is a 3 m USB3.0 camera cable rated for high-flex drag-chain use. For a GigE camera, the LS-CBL-0199 is a 3 m GigE camera cable with screw locks and the same high-flex rating. More options in cables.
  • Architecture mismatch. Kernels built for the wrong CMAKE_CUDA_ARCHITECTURES value either fail at load or fall back to a slow compatibility path. Match the card.

When the CUDA build is the wrong answer

  • Your application consumes depth frames only: the camera's processor already did the stereo work, and the host conversions you skip are small. Ship the CPU build.
  • The host is an embedded GPU module board: the packaged SDK there has CUDA enabled by default, so a source rebuild adds risk for no gain.
  • The complaint is frame rate: the ceiling is the camera's depth mode and the link bandwidth, not the host. Fix the mode or the cable before you touch the GPU flag.
  • You need a high-rate color stream for inspection: give that job to a dedicated 2D camera from area scan cameras and skip depth-color alignment entirely — two simple pipelines beat one strained one.

Pre-flight checklist

  • Devel CUDA base image pinned to a release the SDK supports
  • -DBUILD_WITH_CUDA=ON and -DCMAKE_CUDA_ARCHITECTURES matched to the card
  • SDK source at one pinned version inside and outside the image
  • --runtime=nvidia --gpus all on every docker run
  • Camera mapped through a stable node or a CDI spec, never privileged
  • USB autosuspend disabled for the camera on the host
  • Point-cloud example runs and reports the compute capability before integration

If you want the camera, cable and controller matched as one system before you containerize it, run the system configurator or send us the frame-rate and resolution target.

Questions

Does CUDA speed up the stereo depth computation itself?

No. Stereo depth is computed on the camera's own processor and arrives as finished depth frames over the cable. CUDA in the SDK accelerates three host-side stages only — point-cloud generation, depth-to-color alignment and YUY-to-RGB conversion — so it frees host cores and cuts conversion latency. The camera's frame-rate ceiling and the pixel values stay exactly where they were.

Do the SDK's packaged Linux installs include CUDA on x86?

No. The utility and development packages that the SDK's apt repository serves for x86 and x64 hosts ship without CUDA, and everything still works, so the missing GPU support is easy to miss. On embedded GPU module boards the same packages have CUDA enabled by default. On a desktop or server GPU, compile from source with -DBUILD_WITH_CUDA=ON.

Is privileged mode the only way to give the container the camera?

No, and in production it is the wrong tool: privileged mode hands the container every device and kernel interface on the host. Map the USB subtree with --device /dev/bus/usb, write a udev rule that gives the camera a stable symlink and map that node, or declare the camera in a CDI spec. Docker Desktop hosts cannot pass USB devices directly at all and need a host-side remap service.

Why does the 3D camera vanish from the container after a few hours?

Almost always USB re-enumeration: the container was bound to a single /dev node that changed after a hub reset or power event, or the host suspended the camera to save power. Map the /dev/bus/usb subtree instead of one node, disable USB autosuspend for the camera on the host, check dmesg for disconnect events, and strain-relieve the connector so vibration cannot interrupt it.

What does -DCMAKE_CUDA_ARCHITECTURES=86 mean?

It tells the CUDA compiler which compute capability to build the kernels for; 86 is a common desktop-GPU value. Match it to the card, because a mismatch either fails at kernel load or falls back to a slower compatibility path. The SDK's device enumeration prints the compute capability it detects, so compare that against your cmake line before committing the image.