Your depth camera streams a fixed 30 fps and your small host is drowning: the SDK offers no 3 fps mode, the CPU sits near 100%, and results lag seconds behind the scene. This guide is for anyone running depth processing on a low-power CPU — a fanless box, a panel PC, an ARM board — who cannot change the camera's firmware. The fix is host-side decimation: grab every arriving frame, decode and process only every Nth, and give the CPU a workload it can finish in real time. You will measure what the CPU can actually process, compute the skip count, implement grab-and-retrieve in a few lines, and prove the delivered rate, the CPU load and the latency before deployment.
Why won't the camera accept a 3 fps setting?
Because streaming modes live in firmware: a camera line with no depth mode below 30 fps cannot be talked into one, and a support-only firmware never gains one — no driver update will make the 2-5 fps setting exist.
The rate is one of the firmware's streaming modes, not a register your code can nudge, and a support-only firmware keeps its mode list for good. Waiting for a driver that adds 2, 3 or 5 fps is a dead end. The practical question changes from 'how do you set the fps?' to 'how many frames per second can this CPU actually process?' — a number you can measure this afternoon.
One check first: some SDKs do expose a frame-rate node or a decimation node of their own. If yours does, set it directly and skip everything below except the measurement steps.
How many frames per second can your CPU actually process?
Time one full processing pass and divide: a pass that takes 60 ms caps one core at 1000 / 60 ≈ 16 fps, so a 30 fps stream outruns that CPU by nearly 2× and the excess frames either queue into lag or fall behind.
Worked example. The camera streams 640 × 480 depth at 16 bits, so each frame is 640 × 480 × 2 = 614,400 bytes ≈ 0.6 MB. The link carries 614,400 × 30 = 18,432,000 bytes/s ≈ 18.4 MB/s.
Your pipeline — decode, cloud conversion, matching — takes 60 ms per frame on the low-power CPU, so one core tops out at 1000 / 60 ≈ 16 fps. At the full 30 fps the pipeline demands 30 × 60 ms = 1.8 s of work per second: 180% of one core, which is exactly why frames queue and lag. Set the target at 5 fps for headroom. The skip count is 30 / 5 = 6; rounding the other way, N = 5 would deliver 6 fps at 6 × 60 ms = 36% CPU, and N = 6 delivers 5 fps at 30%.
How do you grab every Nth frame in code?
Call grab() for every arriving frame and retrieve() only every Nth: with the stream fixed at 30 fps and a 5 fps target, N = 30 / 5 = 6, so six grabs yield one decoded frame and the CPU does 5 fps of work instead of 30.
- Measure the per-frame time. Timer around the whole pipeline, 20 frames, average. In the example the result is 60 ms.
- Pick the target with headroom. Keep t_proc × fps_target near half a core: 60 ms × 5 fps = 300 ms of work per second, 30%, leaving the rest for the OS and the network stack.
- Compute the skip count. N = round(30 / 5) = 6.
- Grab every frame, retrieve every Nth. The calls below are OpenCV's pull API; every SDK with a pull interface has an equivalent.
- Run the test for 60 s. Expect grabbed ≈ 1800 (30 fps × 60 s), kept ≈ 300 (1800 / 6) and kept / elapsed ≈ 5.0 fps. If grabbed itself lands far below 1800, the delivered stream is the problem, not your loop — measure it first, as measuring the delivered frame rate on a stereo camera shows.
import cv2
import time
CAMERA_INDEX = 0 # int, first device the driver enumerates
CAMERA_FPS = 30 # int, the fixed firmware rate - read it from the stream, don't guess
TARGET_FPS = 5 # int, what the CPU sustains with headroom
SKIP = round(CAMERA_FPS / TARGET_FPS) # = 6
cap = cv2.VideoCapture(CAMERA_INDEX)
grabbed = 0
kept = 0
t0 = time.monotonic()
while time.monotonic() - t0 < 60.0: # 60 s test run
if not cap.grab(): # takes the frame off the driver, decodes nothing
break
grabbed += 1
if grabbed % SKIP != 0:
continue # skipped frame: no decode, no processing
ok, depth = cap.retrieve() # decode kept frames only
if not ok:
break
kept += 1
# ... your depth pipeline runs here, on kept frames only ...
print(grabbed, kept, round(kept / (time.monotonic() - t0), 1))
# healthy result: ~1800 grabbed, ~300 kept, ~5.0 fps
Where should the skip happen: in your loop or in the SDK callback?
Skip at the earliest point your API allows — grab-and-discard in a pull loop, or the first line of a callback — because the skip is only free while nothing has been decoded or copied yet.
In a pull API that is the pattern above. In a callback API, register one callback and drop at the first line, before any copy or conversion; an early return costs the driver almost nothing. If another consumer needs 'the newest frame', keep a single-slot buffer that the skip logic overwrites, so the consumer always reads the latest kept frame and never a stale one.
What does host-side skipping fix, and what does it leave unchanged?
It cuts host CPU — decode, cloud conversion and your algorithm now run at 5 fps — but the cable still carries every frame, so link bandwidth, sensor exposure and motion blur stay exactly as they were.
| Cost | 30 fps, no skipping | 5 fps kept, N = 6 |
|---|---|---|
| Frames decoded and processed | 30 per second | 5 per second |
| CPU for a 60 ms pipeline | 180% of one core — impossible, so frames lag | 30% of one core |
| Link traffic (640 × 480 × 16-bit) | ≈ 18.4 MB/s | ≈ 18.4 MB/s, unchanged |
| Time between kept frames | 33 ms (1000 / 30) | 200 ms (6 × 33 ms) |
| Exposure cap and motion blur | set by the 33 ms frame period | unchanged |
Two rows carry the limits. The link row does not move, so if a hub, a cable or a shared controller is the bottleneck, decimation cannot help — budget the link first, as the USB hub bandwidth walkthrough does. And if the CPU is still busy after decimation, the remaining cost is usually per-frame post-processing on the cloud, not the camera: the pointcloud filter that slows a depth camera's FPS is the next thing to trim.
Should you shrink the resolution or crop the ROI before skipping frames?
Yes — do it first: cutting each axis in half divides both the per-frame workload and the link rate by 4, turning the 0.6 MB frame into 0.15 MB, so a 320 × 240 stream at 30 fps costs 4.6 MB/s instead of 18.4 MB/s before you skip a single frame.
Pixels help twice, skipping helps once: a smaller frame lowers what the CPU pays per frame and what the link carries, while skipping only lowers the CPU side. Apply the largest crop that still covers the region you inspect, then decimate on top of it.
One trap: a decimation node can sit on the host side of the SDK, so the camera still ships full frames and the 'smaller' image saves decode time but no link traffic. Where the decimation actually runs decides this, and the depth image smaller than expected checklist shows how to find out. For the crop itself, the same logic governs 2D and 3D pipelines, as setting the ROI on 2D vs 3D cameras covers.
Which camera should you pick when the host CPU is the constraint?
For a new build, pick a camera that finishes the depth work on-board and hands the host a finished result: a ToF sensing camera or a structured-light 3D camera leaves stereo matching and cloud conversion inside the camera, and a fanless host only unpacks one small payload per frame.
If you are sizing a station rather than rescuing one, buy the compute where it does the most good. Check the SDK for a frame-rate node too: a camera that accepts a low setting directly deletes the decimation code. In the range, a depth camera that fits a weak host:
- Type
- Depth / ToF camera
If the task does not truly need depth, a low-resolution 2D camera is lighter still — 3D vs 2D for a machine vision system and choosing 2D or 3D for conveyor pick and place lay that decision out. When the host itself is the part worth replacing, a fanless box with real cores fixes it from the other side — the industrial mini PCs category is that hardware — and the camera finder narrows both depth and 2D options by type and interface. For the budget math on a small ARM board, the ARM SBC bandwidth and CPU walkthrough runs the same numbers.
How do you prove the throttled pipeline is healthy before deployment?
Log three numbers across a 60-second run: grabbed frames (about 1800 at a 30 fps stream), kept frames (about 300 at N = 6), and CPU percent (about 30% of one core) — if kept lags grabbed by more than a few frames, your loop is the bottleneck.
- Kept vs grabbed. kept / grabbed must sit at 1 / N = 16.7%; a falling ratio means your loop, not the skip logic, drops frames.
- Frame age. Timestamp each kept frame on arrival; the newest frame must stay under one keep period, 200 ms at 5 fps.
- CPU. Per-core usage in the task manager should sit near the 30% the arithmetic predicts; a higher number means skipped frames are still being decoded or a filter runs per frame.
- Honest fps readout. If the SDK or your loop prints 0 fps, count frames over time instead, as reading the frame rate reliably in OpenCV shows.
- Several cameras. Size for the summed kept rate: three cameras at 5 fps each is 15 processed frames per second, 15 × 60 ms = 0.9 s of work per second, 90% of one core — budget a core per camera or cut the per-frame cost. The three depth cameras on one host test gives the one-stream-at-a-time method.
What are the common mistakes with host-side frame skipping?
The three that cost the most: retrieving every frame and skipping afterwards (no decode saving), letting the SDK queue grow (latency), and choosing N without measuring t_proc first (a skip count set from hope, not from the 1000 / t ceiling).
Each has a quick test. Instrument the retrieve call — it should fire kept times, not grabbed times. Log the age of the frame you process — growth means lag. And recompute N whenever the pipeline changes: after you crop the ROI or trim a filter, the old skip count throws away frames the CPU could have used.
When kept rate, frame age and CPU hold for an hour of production-shaped traffic, the throttled pipeline is ready. Sizing a new station instead: every depth camera in the range sits in the 3D sensors category, the configurator below walks the choice in order, and if the fps target and the host budget still refuse to close, bring the measured per-frame time to the contact page and we will run the numbers with you.
The configurator matches a camera, lens, light, light controller and cables to each other, step by step, and lists only parts that fit what you already picked.
Questions
Can I skip frames in hardware instead of on the host?
Yes, when the camera exposes a trigger or timed-capture mode: firing the trigger at 5 Hz makes the sensor expose and transmit only five frames per second, which trims link traffic and the exposure cadence along with the CPU load — two things host-side decimation cannot touch. Trigger modes are common on area scan cameras and rarer on depth cameras, so check the SDK before counting on it; where none exists, grab-and-skip on the host remains the fix.
Does frame skipping change exposure or motion blur on the kept frames?
No. Each kept frame still exposes up to the 33 ms that a 30 fps cadence allows, so blur is identical whether you process 30 fps or 5 fps. The levers for blur are a shorter exposure (darker frames, so add light or gain) or a streaming mode with a shorter frame period, which raises the shutter-speed ceiling. Skipping changes which frames you compute; it never changes how any single frame was captured.
How many throttled cameras can one small host handle?
Budget cores, not cameras. Each camera kept at 5 fps with a 60 ms pipeline costs 30% of a core, so three cameras land near 90% of one core — one camera per core is the safe rule once the OS takes its share. Link traffic stays trivial: three 0.6 MB streams at 5 fps total 9 MB/s, well inside a USB3 host controller. The binding limit is per-frame CPU, so crop pixels or move depth on-board before adding a fourth camera.
What if the process needs more fps than the throttled pipeline delivers?
Decimation lowers CPU load; it never raises throughput. If the task needs 15 processed fps and the measured ceiling is 16 fps, skipping buys stability, not speed — the ceiling is the ceiling. To go faster, make each frame cheaper (crop or decimate the image), move the matching on-board with a structured-light or ToF camera, or move to a host with faster cores. A lower fps setting on a fixed-rate camera is not one of the levers.