Reduce stereo camera point cloud size before processing lags

A triggered stereo camera that hands your robot a million points per shot is easy to slow down: capture, transfer, matching and the robot's own move all queue behind data you mostly throw away. The fix belongs in the camera settings, not in your post-processing, because the point count is decided at the disparity map, before your code sees a single coordinate. This guide sizes the two levers — the disparity-map downsampling factor and the disparity-map area of interest — from your part and your cycle budget, walks the parameter-tree writes that avoid the errors most first attempts hit, and ends with the checks to run before the line starts. The arithmetic works for any structured-light stereo camera, including the ones in the 3D sensors range.

Why does a dense point cloud slow your robot loop?

Every stage after capture scales with point count: a 1280×1024 disparity map becomes 1,310,720 points, about 15.7 MB of XYZ per frame at 12 bytes per point (three 32-bit floats), and every filter, matcher and normal pass has to touch all of it.

The camera builds that cloud before your code runs. It correlates the image pair into a disparity map, then converts every disparity pixel into XYZ by deprojection, as the deprojection calculation walkthrough shows. The bytes then move: pushed to a GPU or across a 1 Gb/s link at a realistic 110 MB/s, 15,728,640 bytes cost about 143 ms — most of a 200 ms vision budget gone before matching starts. At 20,480 points the same move takes about 2 ms.

Matching-heavy steps behave worse than linearly. Neighbour search on N points costs roughly N·log N, so cutting N by 64 cuts that work by about 90×; the estimate assumes search dominates and ignores fixed overhead. Cropping in software does none of this: a post-capture XY crop runs after the full cloud already exists, so capture and transfer still pay full price. Time capture, transfer and match separately to find the stage that eats your budget, as in the processing-load diagnosis.

What does the disparity-map downsampling factor change?

It thins the disparity map before the point map is computed, one factor per axis: at the common default of 4 per axis the cloud holds one-sixteenth of the full point count, and at the usual maximum of 8 per axis one-sixty-fourth — 4× fewer again.

A map of W × H pixels at per-axis factors fx and fy yields (W / fx) × (H / fy) points. The values sit in the SDK's parameter tree, in a Downsample entry per axis under the disparity-map node; several viewer utilities never expose them, so you set them in the tree and the viewer then saves and displays the effect live.

The points are sampled, not averaged: a strided cloud shifts edges by up to half the new pitch, which the decimation distortion article quantifies. That shift is why the factor gets sized against your feature list rather than maximized.

This is the class the arithmetic tunes. If you are specifying hardware rather than tuning a rig, the projected structured-light cameras are the type to size:

How small can you downsample before the task stops working?

Let the finest feature the camera must resolve set the factor: pitch grows by the factor and a feature needs roughly three points across to survive segmentation — factor 8 on a 400 mm field holds a 40 mm bracket pose but not a 2 mm chamfer.

Run the same numbers for your field of view and feature list before touching a setting. When you are choosing a camera rather than tuning one, the same three-points-across rule sizes the sensor:

Software resampling is the alternative, and the per-axis resolution settings article keeps the geometry honest — but a software resample shrinks the file, not the capture and transfer bill.

How do you set an area of interest that shortens capture too?

Restrict the disparity map itself to the rectangle the part can occupy — written under DisparityMap/AreaOfInterest as two inclusive corner points, LeftTop and RightBottom — and enable UseDisparityMapAreaOfInterest so the rectangle applies during capture: the cloud and the capture-to-depth time both shrink.

The capture-time saving is real because stereo matching cost tracks the searched pixel area: with the switch on, correlation never runs outside the rectangle. With the switch off, the rectangle only trims the finished cloud, and capture still bills you for the full frame.

Size the rectangle from the part's presentation tolerance, not its nominal position. A ±10 mm tolerance is ±32 px at the example's 0.3125 mm map pitch, so give the nominal rectangle 64 px of margin per side. Coordinates are disparity-map pixels and both corners are inclusive: a 640×480 window starting at pixel 0, 0 has RightBottom at 639, 479, not 640, 480.

How do you write the settings without hitting parameter-tree errors?

Write one leaf at a time, parent before child: assigning a whole rectangle as a single dictionary fails with an item-type-not-compatible error, and writing a coordinate under an area-of-interest node that does not exist yet fails with an item-does-not-exist error.

1. Capture one frame at today's settings and record the point count plus the time of each stage: capture, transfer, match. The baseline proves what the new settings buy.
2. Set the Downsample entries under the disparity-map node, one value per axis; the common default is 4 and the usual maximum 8. On SDKs whose viewer hides them, the tree is where they live.
3. Compute the area-of-interest corners in disparity-map pixels, tolerance margin included, then write them as two corner points, one coordinate at a time: LeftTop first, RightBottom second, both inclusive.
4. Set UseDisparityMapAreaOfInterest to true so the rectangle applies at capture and not only afterwards.
5. Capture again and compare the reported point count with the arithmetic — (width / factor_x) × (height / factor_y) on the rectangle — within a few percent.
6. Copy the same writes into your camera start-up code so every session reapplies them, and read the values back after connecting; leftover state from a previous client makes tests flaky, as the known-state reset guide shows.

What do you check before locking the settings in production?

Check coverage at the part's worst position, the point count against the arithmetic, and each stage's time against your cycle budget, in that order — a rectangle that clips the part at one extreme position fails exactly when nobody is watching.

• Coverage: nominal rectangle plus tolerance margin stays inside the map; margin in px is tolerance in mm divided by the 0.3125 mm map pitch, rounded up.
• Count: measured points within a few percent of (width / factor_x) × (height / factor_y).
• Feature: smallest feature ÷ pitch ≥ 3, or the loosest ratio you accept for a pose-only task.
• Timing: capture never beats exposure plus readout — the floor the hardware-trigger frame rate article derives — and transfer and matching come from your own timestamps.
• Failure path: a part outside the rectangle disappears rather than measuring badly, so alarm on a low point count instead of matching the remainder.

When is downsampling the wrong fix?

Skip it when detail lives in fine edges, when the cloud is already sparse, or when capture rather than processing eats the cycle — the levers are then AOI only, a faster capture path, or hardware that emits density natively.

Fine edges and dense reference surfaces argue for factor 1 or 2 inside a tight rectangle: on the example map, a 640×480 area of interest is 23% of the pixels, a 4.3× cut before any downsampling, and template matching with deviation measurement spends the saved budget where the deviation is. A sparse cloud from small, dark or shiny parts gets worse, not better, when you thin it again; the small-parts matching article shows that failure mode.

If the host is the constraint, give the camera a dedicated machine: a vision controller pairs camera ports and built-in light channels with headroom for matching, for example:

For moving parts inspected one line at a time, a line-scan 2.5D system sidesteps the trade-off entirely — this one streams 16K-wide profiles at up to 120 kHz over a 600 mm emitting area:

For high-accuracy work on a static scene, a laser galvanometer stereo camera steers its projection to the region of interest instead of flooding the whole field:

Decide the factor from your smallest feature, the rectangle from the part's worst position, and the rest from measured stage times — in that order, and the cloud shrinks before it ever reaches your code. Filter the camera finder to stereo and structured-light types, work the full setup through the configurator, and browse the 3D topic page for the companion guides. When a live line needs a second pair of eyes, the contact page reaches the applications team.