Multiple camera views on one GPU tensor buffer: slice per view

When multiple camera views land in one GPU tensor — a simulation packing all of its camera images into a single buffer, or your loader batching N physical cameras for training — the tensor is one flat row of elements, and each view sits at a fixed stride. Get the stride or the element count wrong and the failure is strangely selective: frames read correctly at one camera count and come back all -inf, or blank, at another. This guide gives you the buffer math behind it, the two layouts you will meet, a size check that catches the bug in one line, and the per-view slice that reads every camera.

What a packed multi-view tensor actually contains

A batching writer does not return a stack of images. It returns one 1-D tensor of exactly num_views × height × width × channels elements, laid out one of two ways.

In the view-interleaved layout, elements cycle view 0, view 1, up to view N−1 at every pixel, so element k belongs to view k mod N. In the view-major layout, view i occupies one contiguous block, from i × height × width × channels through (i + 1) × height × width × channels − 1.

Writers choose either because both are useful: interleaving gives every view a constant stride, view-major keeps each frame contiguous for fast copies. Which one your writer uses decides how you slice. Slicing either way is a metadata operation on the GPU — no copy, no host transfer — the same principle as feeding GPU tensors to a vision model without a CPU round trip.

Symptom: frames read at one view count, fill values at another

The signature that pins this bug class: correct images at 3 views, whole frames of -inf or zeros at 4. The scene did not change; the arithmetic did. A failure that flips with the view count is never the optics and never the model — it is the reader's stride or element count disagreeing with the writer's.

Read the failure pattern before you touch code:

Likely causes, most probable first

1. The reader's view count or stride disagrees with the writer's. A constant copied from an earlier configuration, or a variable captured before the writer replicated its cameras. Start here: it costs one line to test and the size check below settles it.

2. The allocation is longer than the written data. Allocators round buffer sizes up to a pool granularity or reserve for a maximum camera count. A buffer sized for four views but written with three leaves the fourth slot unwritten, and view 3's slice reads exactly that tail — a depth buffer's fill value makes every element -inf.

3. Channel or width mismatch. A 1-channel depth tensor read as 3 channels multiplies the stride by 3; a width read without the writer's line padding shifts every row. These produce stripes of other views rather than clean fill values.

4. dtype mismatch. Reading float32 elements as float16 doubles the element count you think you have, and the values come out denormal-small or as inf.

5. The read beats the write. The reader pulls the tensor before the render completes, so it sees the initial fill. Timing shows up as intermittent rather than tied to the view count.

Test in this order: causes 1 and 2 are one assert each.

Test each cause in minutes

1. Assert the element count. numel against num_views × height × width × channels, with the four numbers taken from the writer object, never from your configuration. This is the same first step as sizing any multi-channel image buffer — see Check multi channel image buffer size before processing.

2. Check the element size. Float32 elements are 4 bytes; an element size of 2 means you are holding float16 and every stride you computed is wrong by a factor of two.

3. Print one finite-fraction number per view. The share of finite elements in each view tells you instantly whether the failure is whole-frame (layout) or scattered pixels (real no-return data, section below).

4. Cross-check one view against the writer's own single-view read. Read view 0 through the writer's own accessor and through your slice, both copied to the host. Identical element for element means your layout is right; any difference means layout, not content.

5. Probe the parity. Run the scene once at N and once at N+1 views, before training. If the failure flips with the count, the cause is arithmetic — causes 1 and 2 — and no scene change will fix it.

6. Rule out timing. Synchronize the stream, then copy. A frame that turns finite after the synchronize was a read that beat the write, and queue handling matters more than layout — see Latest image only returns duplicate frames: drain the buffer queue and Camera streaming buffer count: why three frames beat one.

The fix: size check, then slice per view

Run the assert first, then slice. The stride slice flat[i::num_views] takes every num_views-th element starting at i, which is exactly view i in the interleaved layout, and it returns a view, not a copy — the data stays on the GPU.

import torch

# Example values - replace with the numbers your writer reports.
num_views: int = 4           # views packed by the writer
height: int = 480            # pixels per column
width: int = 640             # pixels per row
channels: int = 1            # 1 for depth, 3 for RGB
BYTES_PER_ELEMENT: int = 4   # float32

expected: int = num_views * height * width * channels   # 1,228,800

# Stand-in for the writer's tensor; in your program this is the object
# your simulation or grabber SDK hands you, never constructed here.
views_gpu = torch.zeros(expected, dtype=torch.float32)
for i in range(num_views):
    views_gpu[i::num_views] = float(i)   # interleaved writer fill

assert views_gpu.numel() == expected                 # catches causes 1 and 2
assert views_gpu.element_size() == BYTES_PER_ELEMENT # catches cause 4

view1 = views_gpu[1::num_views].reshape(height, width, channels)
assert bool((view1 == 1.0).all())                    # True: view 1 recovered exactly

On real data, replace the final assert with a finite-fraction check: torch.isfinite(view1).float().mean() should read about 1.0 on a fully rendered scene, and the reshape fails loudly if the element count is wrong. Set the threshold for your scene — a view that legitimately sees empty space carries real non-finite pixels, so a fully rendered test scene at 0.99-plus finite is the clean reference.

When -inf is the data, not the bug

Scattered non-finite pixels are normal depth behaviour. A pixel with no return — dark surface, out of range, grazing angle — carries the invalid value, inf or -inf by convention, and the fraction is information: 2 percent is a scene, 100 percent is layout. Mask them before the model, using the camera's own validity or confidence output where it exists; the decoding steps are in 3D camera confidence image values: decode bitflags and mask bad pixels. Keep the depth layout straight while you do it — which stream is aligned to which frame — as in Aligned depth streams: data layout and which intrinsics apply.

On a physical rig the simulated camera is replaced by a depth camera, and the same convention applies: depth in, per-pixel validity out, and the size-and-stride checks above unchanged. One safety note: a depth camera's emitter is a laser source even when the output looks like an image, so follow the vendor's eye-safety class documentation and never view an emitter at close range through magnifiers. The MV-DT01WMU is a ToF sensing camera in the 3D sensors range that fills that role in a physical cell.

Slot mapping is the physical-rig cousin of the stride bug: when one camera in a stereo pair shows no live image, the first suspect is which slot the reader indexes, not the camera.

Porting the batching to real cameras

When the views come from GigE or USB3 area scan cameras instead of a simulator, the arithmetic is identical and the hardware becomes the constraint: each stream needs a port, PoE saves a power drop per camera, and the batched tensor needs memory on the machine that feeds the GPU. A vision controller is built for exactly this — camera ports, light outputs and GPU-adjacent memory in one fanless box.

The MV-VC5525-256G70-NN pairs 5+ LAN ports and PoE camera ports with 32 GB of memory and 8 built-in light channels. That covers a multi-camera rig plus its illumination from one unit.

Compact rigs run on the smaller series:

The MV-VC2140-128G40-1T provides 3-4 LAN ports, 4 built-in light channels and 8 GB of memory with a 128 GB SSD plus 1 TB HDD. The MV-VC3301H-128G66 provides 5+ LAN ports, 4 built-in light channels, 8 GB of memory and includes the VM6200Pro license, and it has no PoE, so plan separate camera power.

When you are sizing that machine, memory headroom for the batched tensor is the number to check first, and a fanless industrial mini PC is the usual host when the vision controller's GPU budget does not cover training.

Prevention checklist before the first training run

• Assert numel against num_views × height × width × channels at every start-up. The assert costs microseconds and converts silent dataset poison into a stack trace.
• Read the four layout numbers from the writer, never from a constant in the training script. Configuration drift is the number-one cause.
• Log one finite-fraction number per view at start-up. A drift from about 1.0 flags a layout or render change immediately.
• Check the element size before any reinterpret or reshape. dtype drift between SDK versions is rare and expensive to find downstream.
• Probe N and N+1 views once. The parity probe catches count arithmetic before it reaches a dataset.
• Carry a timestamp with every frame on real cameras, via chunk data, so a late read is visible in the data — see Chunk data on camera frames: carry timestamp and exposure per image.
• On embedded boards, check the boot-time reserved memory. GPU and camera memory regions are carved at boot, and a reserved-memory setup sized for the GPU and camera that assumes fewer views leaves the batched tensor short of the written data.

The decision before any training run: the numel assert, the element-size check and one finite-fraction print per view have all passed at the exact view count you will train on. More integration-side fixes — buffers, GPU paths, timestamps — are collected on the systems topic page. When the pipeline moves into a real cell, walk camera, lens, light and controller together in the system configurator, or reach us through the contact page.