Rectified texture image missing from a 3D camera grab

Disparity, point cloud and the raw camera images come back from every grab. Then you add the rectified texture to the same grab call and the buffer arrives empty — while the camera's own viewer shows the image without complaint. That combination is one of the most common dead-ends when integrating stereo and structured-light 3D cameras with a separate vision library, and the cause is almost never a broken camera. The node is simply never computed. You leave with the compute-then-re-grab fix in three steps, a table that separates the rarer causes, a validity check with the arithmetic shown, and the point at which a GenTL producer removes the problem for good.

Why does a 3D camera grab return an empty rectified texture image?

The grab call only copies image nodes that already hold data; naming the rectified texture in the data-items list does not run the computation that fills it, so the node is still empty when your grab request reaches it.

A data-items list is a fetch specification, not a processing recipe. The geometry side of a stereo capture — rectified camera pair, disparity, point cloud — is produced by the capture-and-match pipeline, so it exists by the time any grab reads it. Texture outputs sit outside that pipeline: the SDK computes them when asked, because computing them on every capture would spend processor time that applications which never display texture do not need. Your grab requests a node that no step has filled, and the buffer comes back with nothing in it. The camera is healthy; the call order is wrong.

Which images does the capture pipeline compute, and which wait for a compute command?

In most stereo SDKs the depth-side outputs — rectified camera pair, disparity, point cloud — come out of the standard capture-and-match pipeline, while texture outputs are computed on demand because they add processing time that not every capture needs.

The split is deliberate: a texture is a full extra image to compute, transfer and store on every frame, and plenty of measuring applications never look at one. Computing on demand keeps the trigger-to-point-cloud path short; the cost is that a texture node's state depends on your call sequence, not on the capture alone. That dependence is the whole trap.

You will meet this pattern on stereo and structured-light 3D sensors across the 3D sensors category:

On a ToF depth camera the analogous derived output is the amplitude image, so a grab that returns depth but no amplitude points at the same call-order cause; treat the two symptoms identically.

If you also feed the rectified camera pair into downstream geometry, confirm it is genuinely rectified first — the stereo rectification verification guide gives a quick check you can run on saved frames.

How do you fix it in three steps?

Remove the texture node from the data-items list, run the normal grab, call the SDK's texture-compute command, then grab again with only that node in the list; the image comes back valid.

1. Drop the texture node from the grab call's data-items list — typically a parameter such as grab_data_items — and run the grab exactly as before. Every depth-side output returns as it always did.
2. Call the texture-compute command for the capture you just took — ComputeTexture in the SDKs that expose this behavior — so the texture nodes fill with data.
3. Run a second grab whose data-items list contains only the texture node, and copy the result into your image container.

Two grabs per cycle feels clumsy, but it costs one extra image transfer and stays stable across SDK updates; on this class of camera it is the working pattern until you move interfaces.

Why does the camera viewer show the texture your code cannot get?

The vendor viewer computes derived images on demand for display, so a texture that appears there proves nothing about your grab path; it only proves the capture itself holds usable texture data.

That viewer behavior doubles as your fastest diagnostic. After a capture, open the SDK's image tree and look at the texture node itself. An empty node means no compute step has run, and the three-step fix applies as it stands. A filled node beside an empty grab buffer means the compute ran but your grab never fetched it — the same fix, plus a check of the node name in your data-items list, letter for letter. The tree also settles the replay case: if a saved capture loads with an empty texture node, it was stored before anyone computed texture, so compute after loading rather than after saving.

When should you move from the frame-grabber interface to a GenTL producer?

If the camera SDK generation you deploy ships a GenTL producer, plan the move at your next integration change: the producer is the camera vendor's own GenICam library, your vision software connects to it directly, and the bridge layer where this fetch-only behavior lives drops out of the stack.

GenTL is the GenICam transport-layer standard, and a producer is simply the camera SDK published in that form, so any GenICam-capable vision software can reach the camera's full feature set instead of a generic frame-grabber mapping. The older bridge pattern is what forces your code to spell out node fetches itself, which is why workarounds like the double grab exist at all; connecting through the producer retires that class of quirk instead of patching it case by case. For how such a producer exposes streams and parameters, see how the GenTL producer data path works for depth cameras. The decision: keep the double grab on a validated, running line, and retire it in the release that migrates the cell to GenTL.

What else produces an empty texture, and how do you tell the causes apart?

An empty texture buffer almost always means the missing compute step, but wrong node names, ordering mistakes and version mismatches produce identical symptoms; the tests below separate them in minutes.

Work the table top to bottom: the first two rows cover the overwhelming majority of cases. Version changes deserve one extra look, because an SDK update that regroups the image tree can quietly rename nodes; re-run the tree check after every update, and use the firmware and SDK alignment guide when symptoms first appear after one.

How do you prove the re-grabbed texture is valid?

Check three things before you trust the image: the buffer's byte count against the pixel format, its dimensions against the rectified image size, and its mean intensity against a mid-range expectation.

Intensity comes next: a normally lit scene at a correct exposure sits in the middle of the 8-bit range, so treat roughly 60 to 180 of 255 as healthy, while 0 or 255 sends you to the lighting row of the table above. Geometry comes last: texture pixels and depth pixels rarely share a grid, so coloring a point cloud from the texture uses each image's intrinsics — the intrinsic parameter conversion for depth and texture images guide covers the matrix work. One more trap sits after the grab itself: copying the SDK buffer by reference instead of by value turns a valid image into garbage in your container, which is the case the stereo frame buffer copy guide walks through.

How do you keep the fix working in production?

Wrap the whole sequence — depth grab, texture compute, texture-only re-grab, then the three validity checks — inside one acquisition function, so no code path can request the texture before its compute command has run.

• Give acquisition one owner: the depth grab, the compute call and the texture re-grab live in a single function, and everything else receives finished images.
• Assert on bytes, not on 'an image arrived': reject any texture whose buffer is not width × height × bytes per pixel, using your camera's real numbers in the formula above.
• Log the capture counter or timestamp with every image set, so a texture paired with the wrong frame shows up in the log instead of in your results.
• Re-run the tree check after every SDK or firmware update; a renamed or regrouped node stays invisible until the first empty grab on the line.
• If cycle time is tight, remember the re-grab adds roughly 21 ms of GigE transfer for a 2.3 MB-class Mono8 texture, plus the compute step itself — measure both once and put the total in the cycle budget.

The decision in one line: an empty rectified texture means nothing computed it, so compute then re-grab, prove the buffer with the byte and intensity checks, and move the connection to GenTL at the next integration change. To shortlist cameras to test this against, start at the 3D camera finder or the camera step of the configurator; more capture-side fixes live on the 3D vision topic page, and if the table still leaves two causes open, send the SDK version, camera model and a saved capture file through the contact page.