Build OpenCV with CUDA for depth camera streams

You wire the depth camera into your SDK, the sample code compiles against its bundled OpenCV, and the first cv::cuda call fails: the module was never in that binary. That is the normal state of every prebuilt OpenCV — SDK bundles, installers, pip wheels and most embedded board images ship with CUDA compiled out. This page gives you the one-call check that proves what your build contains, a frame-budget calculation that decides whether a CUDA build pays for itself, the CMake steps that build OpenCV with CUDA, and the way to point an existing depth-camera project at the new library without breaking the SDK.

Why doesn't the prebuilt OpenCV see your GPU?

Because every prebuilt OpenCV package is compiled with WITH_CUDA=OFF and OPENCV_DNN_CUDA=OFF — SDK bundles, Windows installers, pip wheels and most embedded board images included. The CUDA module and the DNN GPU back end are absent from the binary, and no driver, path or environment change can add them.

CUDA support must be compiled in, per GPU generation, and it brings the toolkit plus large per-architecture kernel binaries with it. That cost is why official packages leave it out and why requests for CUDA-included prebuilts remain open. The copy a camera SDK ships with, or links against, is the same CPU-only package under a different install path. And the depth functions you most want on the GPU — stereo disparity and optical flow — live in opencv_contrib, which the standard packages do not include at all.

Do you actually need a CUDA build for your depth stream?

Build OpenCV with CUDA when per-frame CPU time misses the frame budget: at 1280×720 and 30 fps the budget is 33 ms. A full-frame bilateral or median pass costs roughly 10–60 ms on one CPU core, and the same pass runs in 1–3 ms on a mid-range CUDA GPU.

Those CPU and GPU figures are planning values for common per-pixel filters — measure your own operations before you commit either way. The stream arithmetic you can compute exactly:

The 2.5 GB/s copy figure assumes pinned (page-locked) host memory on a PCIe 3.0 x4 link; expect less for unpinned copies. 442 Mbit/s sits well inside a GigE link's practical ceiling of about 940 Mbit/s, so the network is not your constraint. Uploading a frame costs 0.7 ms against a 33 ms budget, so the copy costs almost nothing; the win comes from running the heavy passes on the GPU instead of the CPU.

Decision rule: time each per-frame CPU pass, add the times, and compare the sum with 30 ms — 90 percent of the budget, leaving margin for capture and dispatch. Under 30 ms, stay on the prebuilt. Over it, build OpenCV with CUDA. The same budget at 60 fps halves to 16.7 ms, which is why higher frame rates tip projects into a GPU build.

What do you need before you start the build?

You need five things: a CUDA-capable GPU with its current vendor driver, the matching CUDA toolkit and C++ compiler, CMake, and the OpenCV and opencv_contrib sources at the same release, all from their official repositories.

Build on the machine class that will run the result: a workstation with a discrete GPU for development, and the production host for the final build. On the line, capture belongs on a vision controller. The MV-VC5525-256G70-NN exposes 5+ LAN ports with PoE camera ports, 32 GB of memory and 8 built-in light channels, so a GigE depth camera, trigger distribution and illumination run from one fanless box.

The depth source itself comes from the 3D sensors range — laser profilers, stereo and structured-light cameras, ToF and logistics cameras — and its resolution and frame rate are the two inputs to the budget above.

A trained network that ran on your bench GPU needs a deployment host at the line. The MV-VAC3504X-128G01 AI inference terminal carries 5+ LAN ports, 4 built-in light channels and a single inference card, which moves the finished model out of your workstation and onto the machine.

How do you build OpenCV with CUDA, step by step?

Configure CMake with WITH_CUDA=ON, add the opencv_contrib module directory, and restrict compilation to your GPU's compute capability. Confirm the configure summary reports CUDA support, then build and install into a prefix of its own.

1. Install the GPU driver and the CUDA toolkit that matches it, both from the GPU vendor's developer site, then run the toolkit's device query to confirm it sees your GPU.
2. Download the OpenCV and opencv_contrib sources at the same release tag into two separate directories. A version mismatch here is the classic cause of unresolved symbols after a long build.
3. Configure an empty build directory with CMake. Set WITH_CUDA=ON. Set OPENCV_DNN_CUDA=ON as well if the DNN module should execute networks on the GPU — install the vendor's deep-learning runtime first, because that back end depends on it.
4. Point CMake's extra-modules option at the modules directory inside your opencv_contrib checkout. Without this step you keep the core CUDA functions but lose the stereo and optical-flow ones.
5. Set the compute-capability list to your GPU's generation alone. Compiling for one capability instead of the full default spread roughly halves the build time.
6. Select the Release configuration and an install prefix outside every SDK's directory tree. Read the configure summary before building: it must list CUDA with your GPU's capability plus the contrib modules you expect.
7. Build and install, then check the installed tree for the CUDA libraries before you point any project at it.

How do you prove the CUDA modules are actually in the binary?

Call cv::cuda::getCudaEnabledDeviceCount(): it returns 0 when the binary has no CUDA module even on a perfectly working GPU machine, a positive count when build and driver agree, and a negative value when the driver is missing or mismatched.

int devices = cv::cuda::getCudaEnabledDeviceCount();
// devices == 0 : CUDA module absent from this binary (or no device visible)
// devices >  0 : one per usable GPU — build and driver agree
// devices <  0 : driver missing or version mismatch

The test that proves it: compile those three lines against the SDK's prebuilt OpenCV and the result is 0; link the same code against your build and it prints 1 or more. In Python, cv2.cuda.getCudaEnabledDeviceCount() on the same machine must show the same count.

A positive count is necessary but not sufficient. Push one frame through one operation and time both paths: thresholding a 1280×720 depth frame typically takes 1–3 ms on a CPU core and under 0.5 ms on the GPU. If your GPU time comes out worse, you are paying a host-to-device round trip between calls — keep the frame in GPU memory across the whole chain, the pattern behind slow GPU image processing: diagnose low frame rates.

How do you point your depth-camera project at the new build?

Prepend your install prefix to CMake's package search path and let find_package(OpenCV) resolve there. Then confirm which OpenCV the linker actually chose, because an SDK carrying its own bundled copy will otherwise win and hand you a CPU-only library.

Delete the old CMake cache before reconfiguring: a cached path keeps resolving to the previous library even after you change the search path. Fetch each frame into OpenCV's CPU image type without an extra copy using the SDK frame buffer to cv::Mat pattern, then upload once to the GPU.

If the SDK's samples or headers then refuse to cooperate, the two standard failure modes are covered step by step in CMake can't find OpenCV in SDK samples and camera SDK missing OpenCV headers. When capture and processing share one process and the SDK insists on its own OpenCV, rebuild the SDK from source against your build — the route most SDKs document for exactly this case. Forcing two copies into one process produces the duplicate-symbol and missing-symbol errors diagnosed in C++ camera SDK linking errors.

At runtime, keep exactly one OpenCV library on the loader path. On Windows the SDK's own bin directory is the usual shadow copy; on Linux, ldd on your binary shows which library actually loads.

Why does it still fail, and what fixes each cause?

Four causes cover nearly every failure: opencv_contrib left out (stereo and optical-flow functions unresolved), two OpenCV copies in one process (linker and loader conflicts), a compute-capability list without your GPU (kernels fail at launch), and the DNN CUDA back end built without its deep-learning runtime.

• Missing contrib functions: re-run CMake with the contrib modules added, confirm the summary lists them, and rebuild.
• Linker conflicts: one OpenCV per process — rebuild the SDK against your build, or split capture and processing into separate processes that exchange data.
• Kernels failing at launch: your GPU's compute capability is missing from the compile list; add it and rebuild.
• DNN refusing the GPU: install the deep-learning runtime, reconfigure with OPENCV_DNN_CUDA=ON, and rebuild. If the network then eats GPU memory, why small neural network models use large GPU memory explains the allocation behavior.

Two more appear on real rigs. Closing the camera while GPU buffers from its frames are still alive can invalidate the CUDA context in some SDK combinations — release GPU memory after you close the stream, the order shown in CUDA context invalid after camera close. And when you reproject depth to a point cloud on the GPU, channel-layout and stride mismatches look like image bugs but are memory bugs — GPU point cloud memory layout walks through the shapes and alignment.

When should you skip the CUDA build entirely?

Skip it when the CPU meets the frame budget with margin, when the machine has no CUDA-capable GPU, or when your operations are all transparent-API ones OpenCL already covers — OpenCL support ships in the standard prebuilt, so acceleration costs no compile time.

• Build OpenCV with CUDA: full-frame filtering, denoising, reprojection or DNN inference at 30 fps on 720p or larger frames, where the budget check above fails.
• Stay on the prebuilt: thresholds, ROI crops, or inspections a few times per second — the upload alone can exceed the compute you save.
• Use OpenCL instead: a CUDA-less GPU box, or an operation set the transparent API covers; measure before changing hardware.
• Spend the effort elsewhere: capture-heavy, processing-light lines run fine on a stock vision controller and the prebuilt, and lighting or lens choice moves accuracy more than a GPU build.

When the hardware side needs sizing, put the capture host together in the system configurator or start from the cameras finder, and send your camera, resolution and frame rate through the contact page — we will run the frame budget with you. More depth-camera guides are collected on the 3D topic page.