OpenCL clBuildProgram failed in a container: fix the GPU compute stack

Your 3D camera node finds the camera, connects, and then dies before the first frame: "An OpenCL error occurred: clBuildProgram failed", the node exits with code 1, and the launcher tears everything down. This guide is for teams running a 3D camera driver in a Docker container on a GPU host, where the same stack worked on another machine or before an update. You will pull the version numbers that decide the case from clinfo and nvidia-smi, learn why nothing installed inside the image can repair it, and pin the one component that is actually broken: the host GPU driver's OpenCL compiler. The field case behind this signature: driver 520.56.06 with CUDA 11.8 refused to build kernels that driver 510.85.02 with CUDA 11.6 compiles without complaint.

What the failure already tells you

Read the start-up log in order. The node loads, prints its SDK build, finds exactly one camera, reads the camera's model and firmware, and reaches the connect step. Then the exception. Discovery, enumeration and the camera's network link all worked before the failure, so the camera is not the problem.

The failure lands one step later, when the SDK initializes its processing pipeline and compiles compute kernels to the GPU. The exception names its requirement, OpenCL 1.2, and clinfo on the failing machine reports "Device OpenCL C Version 1.2": the requirement is met, and the build fails anyway. When a stated requirement is satisfied and compilation still fails, the compiler is the suspect. A node that dies at start-up with the GPU missing entirely is a different fault — stereo depth node won't start: verify GPU runtime libraries first covers that one; here the runtime is present and its compiler is broken.

The image does not ship the compiler

The Dockerfile from the failing case installs exactly what an image needs on the dispatch side, and nothing more:

RUN apt-get install -y ocl-icd-libopencl1 opencl-headers clinfo
RUN mkdir -p /etc/OpenCL/vendors && \
    echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility

The loader finds the GPU through /etc/OpenCL/vendors/nvidia.icd, which names libnvidia-opencl.so.1 — a library the image does not contain. The GPU container toolkit on the host injects it, with the rest of the driver stack, at container start, because NVIDIA_DRIVER_CAPABILITIES requests compute. That injection is the whole arrangement, and it carries the consequence that decides this case: the OpenCL compiler inside the container belongs to the host driver. Its version, its bugs and its fixes arrive from the host on every start.

Two things can go wrong in a GPU container: the injection itself, as when ROS 2 point cloud black in Docker: fix GPU runtime passthrough, or the injected stack being broken — which is where clBuildProgram lives.

The numbers that decide it

Two commands, five numbers. Print them before touching anything:

The CUDA version in the nvidia-smi header names the newest toolkit that driver branch supports, so it is the fastest label for the branch you are on. The OpenCL compiler ships with that branch, which is why the loader row staying identical between the two stacks proves the image was never the variable.

One arithmetic check shows the device side is healthy. Global memory reports 6222970880 bytes and maximum allocation 1555742720 bytes:

Exactly one quarter — the allocation cap this driver family reports. The device answers queries consistently; only compilation fails.

The error codes agree with that reading. The SDK's exception is CL_BUILD_PROGRAM_FAILURE: the build failed. The clinfo build log goes further with "create kernel : error -45" — and -45 is CL_INVALID_PROGRAM_EXECUTABLE, meaning no successfully built executable exists. clinfo's own probe kernel cannot be created, so the compiler is broken for every program on that stack, not just the camera SDK's kernels.

Tests in order of probability

1. Platform count. Run clinfo in the container. "Number of platforms 1" with your GPU listed means the injection works: ICD file, loader, compute capability, host toolkit. Zero platforms is a plumbing fault — missing ICD file, missing compute capability, no GPU container toolkit on the host — so fix that layer first; this article's signature has the platform visible.
2. Build log. Search the clinfo output for the CL_PROGRAM_BUILD_LOG section. "Preferred work group size multiple" printing a number means kernels compile on that stack. An error -45 line there means the compiler fails for everything — stop editing the Dockerfile; the host is the patient.
3. Driver pair. Read the nvidia-smi header inside the container. A driver of 520.56.06 with CUDA 11.8, against an SDK build proven on an older branch, is your answer with no further test needed.
4. Host reproduction. Run the SDK's capture tool directly on the host, same driver. It fails there too, and the container is exonerated. This single command separates container faults from driver faults.

The fix: pin the host driver

1. Install the 510.85.02 driver branch on the host. Its own nvidia-smi header reads CUDA 11.6 — confirm the branch by that header, not by package names.
2. Leave the image untouched. Loader, headers and ICD file already work, and the compiler never lived in the image.
3. Restart the container so the new driver stack is injected.
4. Run clinfo: platform listed, "Compiler Available Yes", and the build log prints a number for "Preferred work group size multiple" — 32 on this GPU class, matching the warp size the device reports.
5. Run the SDK capture tool once, then start the node. Driver initialization now completes past the point where it died.
6. Record the triple — driver 510.85.02, CUDA 11.6, SDK build — and pin it the way you pin library versions, as edge inference device crashes: fix with SDK version pinning argues for the whole stack.

Worked example: the failing station, number by number

The case: an Ubuntu 20.04 image, a ROS 1 Noetic driver node, one structured-light 3D camera, and a 6 GB-class workstation GPU reporting 30 compute units.

• Node start: SDK build prints, one camera found, camera model and firmware read, connect attempted.
• 2.4 seconds later (log timestamps 793.137 to 795.557): "clBuildProgram failed", CL_BUILD_PROGRAM_FAILURE, node exit code 1, all processes down.
• clinfo: platform "NVIDIA CUDA", one device, "Compiler Available Yes", driver 520.56.06, device OpenCL C 1.2, ICD loader 2.2.11 reporting OpenCL 2.1.
• Build log: "create kernel : error -45" — the probe build left no executable.
• Memory check: 6222970880 / 4 = 1555742720, internally consistent, so the device reports sane limits.
• nvidia-smi: driver 520.56.06, CUDA 11.8.
• Action: host driver pinned to 510.85.02, header reading CUDA 11.6; container restarted; image unchanged.
• Result: kernels compile, the node initializes the driver, capture proceeds.

The entire repair is one version number on the host. Nothing in the Dockerfile, the SDK packages or the camera changed.

Hardware that keeps the stack supportable

The driver pin belongs in the station record whatever sensor you run, because structured-light cameras and laser profilers alike do their point-cloud math on the host GPU. A projected structured-light camera such as the MV-DPS200P-02 reaches this code path the moment its driver initializes.

Laser galvanometer and profiler sensors like the MV-DLS300P-04 and MV-DP4180-03P ride the same host stack, so the same record entry covers them.

The host carries the container, the GPU and the camera link, so it deserves the same attention as the sensor. The MV-VC5525-256G70-NN supplies PoE camera ports, 8 built-in light channels, 32 GB of memory and PCIe expansion; the MV-VC4712-256G10-2TH brings PCIe expansion with PoE camera ports and 6 built-in light channels.

All 2 share: Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

Browse vision controllers when compute, camera ports and light channels need sizing in one box.

When the usual answer is wrong

The exception text tells you to update your drivers. In this signature the update is the disease: 520.56.06 is the newer driver and the one that fails the build. Commit to the downgrade instead of hunting for a newer driver because the message suggested it.

The clinfo footer warning — loader OpenCL 2.1 against a 3.0 platform — is noise here. The SDK's kernels are OpenCL C 1.2, and a 2.1 loader dispatches 1.2 builds without trouble. Keep ocl-icd-libopencl1 current and ignore the note until you use 3.0 features.

Installing a CPU OpenCL platform into the image trades a five-minute host pin for a configuration the camera SDK was never qualified on; its supported path is the GPU. If the host must stay on a newer branch because other software needs it, the honest move is a newer SDK build proven on that branch — the situation run a 3D camera SDK on an unsupported compute device walks through — and then the clinfo build check again before the station runs.

Decide by case: keeping the validated SDK means pinning 510.85.02 with CUDA 11.6. Moving the driver forward means requalifying SDK and driver together and recording the new triple.

Checklist before you call it fixed

• clinfo shows "Number of platforms 1", your GPU, and "Compiler Available Yes".
• The clinfo build log prints a number for "Preferred work group size multiple" (32 on this GPU class) with no error -45.
• nvidia-smi inside the container matches the recorded driver and CUDA pair.
• The SDK capture tool runs on the host and inside the container.
• The node starts, driver initialization completes, and the first capture stores.
• Driver version, CUDA branch and SDK build are written into the station record.

For the next station, start from 3D sensors, narrow candidates in the camera finder, and bring camera-and-controller pairings to us on the contact page so the compute stack is aligned before the first container runs.