Crop XY region of interest on a 3D depth camera point cloud

A depth camera gives you a depth range filter that cuts everything nearer or farther than z — and then it stops. There is no x-y field-of-view setting that shrinks the picture left-right and top-bottom, so phantom points from the fixture edge, the table or the wall behind the cell stay in the cloud and end up in your blob analysis, measurements or robot poses. You impose that lateral crop yourself. This guide gives you the decision — slice the depth image or filter the point cloud — the pixel and metric arithmetic for a worked 200 × 150 mm work area at 300 mm mounting distance, the filter code, and the tests that prove the crop landed where you meant.

Why is there no x-y field-of-view setting on a depth camera?

Depth camera drivers expose a depth range filter that cuts points by distance z, but almost none expose an x-y field-of-view crop, so you keep the full cone of data and remove lateral points yourself with a bounding-box filter.

The cone is fixed by the lens and the imager, and the depth pipeline — stereo matching or time of flight — runs on the full frame. The settings published for depth streams cover the z range, exposure or emitter power and output resolution; lateral bounds are rarely among them. A reduced scan region, where a camera offers one, changes the sampling density, not the cone.

The z filter does part of the job but not this part. It drops points by distance, and a fixture surface at the same distance as the part passes straight through. Lateral artifacts are a different axis: same z, wrong x and y. Output resolution is a related lever some pipelines expose at capture time — change 3D camera depth resolution and field of view at capture time covers what that does — and it still leaves the cone alone.

Should you crop the depth image or the point cloud?

Crop the depth image when the part always sits at one known distance and you want the processing saving of fewer pixels; crop the point cloud in metric x and y whenever the working distance can vary, because pixel bounds hold only at one depth.

A pixel crop is a fixed angular window: its physical width shrinks in proportion to distance, so the same columns cover half the width at half the depth. A metric crop on the points is a true box at any depth. If the part sits at a fixed distance and you deproject after the slice, the image crop is also the faster option, because deprojection then runs on a fraction of the pixels.

A practical set-up runs both: the pixel slice at capture to cut work, and the metric box as the final authority before matching or pose estimation. Each costs almost nothing per frame. The mistake to avoid is treating pixel bounds as physical bounds while the mounting distance drifts — recompute them whenever the camera moves, or the window quietly covers a different strip of the workbench.

What bounding box numbers should you use?

Start from the part's region plus a 10-20 mm guard band, and at 300 mm depth with an example 87° × 58° field of view that turns a 200 × 150 mm work area into a 450 × 325 pixel crop centered on a 1280 × 720 frame.

Work in that order: define the box in millimetres at the part, convert it to the cloud's unit for the point filter, and only then convert millimetres to pixels at the mounting distance if you also want the image slice. The pixel conversion comes from the pinhole relation between depth and covered width:

The frame, field of view, distance and part size here are example inputs — put your own in:

• Width at the part: 2 × 300 mm × tan(43.5°) = 600 × 0.9490 = 569 mm; height: 2 × 300 mm × tan(29°) = 600 × 0.5543 = 333 mm.
• Scale: 569 mm / 1280 px = 0.445 mm/px horizontally; 333 mm / 720 px = 0.462 mm/px vertically.
• Columns: 200 mm / 0.445 mm/px = 450 px, so (1280 − 450) / 2 = 415 → keep columns 415 to 865.
• Rows: 150 mm / 0.462 mm/px = 325 px, so (720 − 325) / 2 ≈ 197 → keep rows 197 to 522.
• The same box in metric form is x = ±100 mm and y = ±75 mm, which is what the point-cloud filter uses.
• Check: 450 px × 0.445 mm/px = 200 mm, so the pixel and metric bounds agree.

Distance and tilt define where that box sits on the bench, so settle the mounting geometry first — the 3D camera working distance and positioning guide walks through it.

How do you crop the point cloud in x and y, step by step?

Filter the deprojected points with one boolean mask that keeps x between −100 and +100 mm and y between −75 and +75 mm, then assert that no kept point sits outside those bounds; the whole crop is five lines of array code.

1. Capture and measure. Deproject one frame with the part in place and print the mean z over the part surface. That fixes the working distance (300 mm in the example) and shows you the unit the cloud is in.
2. Write the box in the cloud's unit. x_min = −100.0, x_max = +100.0, y_min = −75.0, y_max = +75.0 mm, alongside the z bounds you already trust from the depth range filter.
3. Mask the array with the bounds:

import numpy as np

pts = np.asarray(cloud, dtype=np.float64)  # your (N, 3) cloud in mm: columns x, y, z
x, y = pts[:, 0], pts[:, 1]
mask = (x >= -100.0) & (x <= 100.0) & (y >= -75.0) & (y <= 75.0)
cropped = pts[mask]

# test: no kept point sits outside the box
assert cropped[:, 0].min() >= -100.0 and cropped[:, 0].max() <= 100.0
assert cropped[:, 1].min() >=  -75.0 and cropped[:, 1].max() <=  75.0
print(cropped.shape[0], "of", pts.shape[0], "points kept")

4. Read the test. A failed assert usually means the unit, not the geometry: a factor of 10 or 1000 in the depth scale truncates or empties the box, which is the failure mode in depth scale mismatch: why your point cloud is cut off at close range.
5. Check the frame. Camera frames run x right, y down, z forward; robot frames often flip y. A sign flip puts your box above and behind the part, not on it — the 3D ROI coordinate frame guide shows how an inverted cloud looks and how to diagnose it.
6. Store the bounds with the calibration. A box is meaningless after a re-mount, so version it together with the mounting distance and extrinsics.

How do you crop the depth image instead?

Slice the depth array to the pixel bounds from the worked example — columns 415 to 865 and rows 197 to 522 — before deprojection, and shift the principal point of your intrinsics by the same offsets if you deproject the sliced image.

1. Convert millimetres to pixels at the mounting distance with the arithmetic in the section above.
2. Slice the depth array before deprojection. The example window keeps 450 × 325 = 146,250 of 921,600 pixels, about 16%, and deprojection then runs on that fraction.
3. If you deproject the sliced image rather than the full frame, shift the principal point, or every point lands displaced by the crop offset:

with x0 = 415 px and y0 = 197 px in the example. Deprojecting the full frame and cropping the cloud afterwards avoids the shift entirely. 4. Crop any registered color image with identical bounds, or the overlay drifts out of registration; the depth-to-color registration behind that overlay is explained in aligned depth point cloud intrinsics: which camera to deproject with. 5. Re-derive the pixel bounds whenever the mounting distance changes — the slice is valid at 300 mm, not at other depths.

How do you verify the crop landed where you intended?

Print the minimum and maximum of every kept coordinate — they should sit at the box edges, within a pixel or a fraction of a millimetre — and re-run the check after any re-mount, because a 10° tilt at 300 mm moves the scene sideways by about 53 mm.

• Kept extremes at −100.0 and +100.0 mm in x and −75.0 and +75.0 mm in y mean the box is the limiting factor, as intended. Extremes well inside mean the scene, not the box, is limiting and the crop is doing nothing.
• Render the cropped cloud in one color over the full cloud in another. The boundary should hug the work area, with the part edges a few pixels inside.
• Do a physical edge test: lay a ruler across the boundary at a known reading. A return at 95 mm inside the box survives; one 5 mm outside is gone.
• After any re-mount, re-run everything. The tilt number above is the reason: 300 mm × tan(10°) = 300 × 0.1763 = 53 mm, more than half of a 100 mm half-box.

Why do phantom points survive the crop?

Mirrors, shiny fixtures and multi-path returns can land inside your box at plausible depths, so a phantom surface survives a perfect crop; fix the scene — matte or re-angle the reflective surface — and tighten the depth range, not just the lateral bounds.

• Matte or re-angle reflective fixtures so the specular return leaves the cone instead of faking a surface near the part.
• Tighten the z range around the part depth. A multi-path return often arrives from a different distance even when its x and y land inside the box.
• Shroud the background with a matte panel; a physical occlusion beats any filter.

Check candidates on the full cloud, not a decimated one — decimation moves points, as point cloud downsampling distortion: why decimation shifts points quantifies. For the opposite artifact, holes where surface points should be, the causes and fixes are in missing points in a 3D camera point cloud: fix the depth holes.

When is cropping the wrong fix?

A crop changes which points you see, not how good each point is: density, noise and depth resolution are untouched, so it cannot sharpen a weak sensor, and a fixed box silently discards parts that travel outside it.

• Weak data, not stray data. If points per square centimetre or depth noise is the complaint, longer integration, more texture for stereo matching or a finer sensor moves the number; a crop cannot.
• Parts that move through a volume. Size the ROI around a detected part instead of around a fixed bench region; the trade-offs are laid out in 2D vs 3D camera ROI: where to set it and which one wins.
• Too many points, not wrong points. Voxel or grid downsampling is the tool for load — and it shifts points, per the decimation article above.

If the sensor is the limit, pick it for coverage and depth resolution at your mounting distance and keep the crop as hygiene. For a fixed desktop cell with a known scene, a projected structured-light camera such as the MV-DPS200P-02 provides the dense static cone that a software box crops cleanly.

For frame-rate robot guidance over the same fixed cone, a ToF sensing camera like the MV-DT01WMU runs the identical crop-and-filter pattern at capture rate.

For parcel tunnels and dimensioning stations, the box you filter to becomes the measure-zone outline itself; logistics-style 3D cameras in the 3D sensors category are built for exactly that volume-measurement duty.

When you size a new cell, start from mounting distance and coverage rather than from the crop: the 3D sensors category lists laser profilers, stereo and ToF models with their depth specifications, the camera finder filters them by type and series, and the configurator below steps through the rest of the chain.

More depth-image techniques — point clouds, ROI frames, intrinsics, unit mismatches — are collected on the 3D vision topic page.