Feeding grayscale images to a 3-channel model input

You trained a detector, validation passed, and predictions ran clean in the training environment. Then the same weights in a new environment, or on a live mono camera feed, fail before the first layer executes — inside the predictor's own preprocessing, on a line that reverses channels and transposes axes. The traceback names a transpose; the cause is the shape of your grayscale frame. You will see the shapes mono cameras and grayscale files actually deliver, the three NumPy calls that turn a 2-D frame into the (1, 3, H, W) tensor the network wants, the memory cost of doing those calls in the wrong order, and the checks that separate a shape bug from a preprocessing mismatch.

What the failing line is telling you

The predictor preprocesses every frame with one combined expression before the network runs:

im = im[..., ::-1].transpose((0, 3, 1, 2))

The comment beside that line in the pipeline reads 'BGR to RGB, BHWC to BCHW': the slice reversal converts a BGR frame to RGB, and the transpose converts an (n, h, w, c) batch to (n, c, h, w). Both operations assume four axes — batch, height, width, channel.

A grayscale frame from a mono camera buffer, or a file read in grayscale mode, is (h, w): two axes. The reversal does not fail; on a 2-D array it silently reverses the width. The transpose then asks for axis 3 of an array that has only axes 0 and 1, and the process stops with an axes error. The crash lands one line downstream of the cause, which is why it looks like a framework defect and is not.

Treat this traceback as a shape-contract error. Print the array's shape and ndim at the last line before predict, before touching the weights or the framework version.

Why the network still wants three channels

In this failure the detector's first layer is a convolution declared with the arguments [3, 48, 3, 2]: three input channels, 48 output channels, a 3×3 kernel, stride 2. That summary prints at training time, and it holds even when every training image was grayscale. The training pipeline expanded each grayscale image to three identical channels before the model ever saw it, so 'trained on grayscale' still means a 3-channel tensor contract — the replication simply lived in the data loader, and your inference code has to repeat it.

Five causes, in the order to test them

1. A 2-D grayscale frame reached a 3-channel pipeline. The most common case, and the resolution of the failure traced here: an image-format mistake in preprocessing. Test: print frame.shape and frame.ndim on the exact object passed to predict. Two axes where the pipeline expects four is the whole answer.
2. The new environment reads images differently. Training read files in color mode, which converts a grayscale file to (H, W, 3) with three equal channels; the deployment code reads the same files in grayscale mode and gets (H, W). Test: load one saved training image in both environments and compare the shapes.
3. The batch axis is missing. A (H, W, 3) color frame still has three axes; the transpose wants four. Test: frame.ndim must read 4 at the boundary.
4. The values are wrong, not the shape. The model runs, but scores drop and detections vanish: the normalization range, channel order or resize differs from training. Test: push one labelled training image through the inference pipeline and compare the tensor with the training-time tensor; the first element that diverges names the step to fix.
5. An alpha or palette file. PNGs with transparency load as (H, W, 4); palette-indexed PNGs load as (H, W). Test: the same shape print as cause 1, then slice or convert before expanding.

The fix: expand, repeat, transpose

import numpy as np

# gray: np.ndarray, dtype uint8, shape (3000, 4000) - one Mono8 camera frame
gray = np.squeeze(gray)          # (H, W, 1) -> (H, W); no change if already 2-D
x = np.expand_dims(gray, -1)     # (H, W)    -> (H, W, 1)
x = np.repeat(x, 3, axis=-1)     # (H, W, 1) -> (H, W, 3)
x = x.transpose(2, 0, 1)[None]   # (H, W, 3) -> (1, 3, H, W)

assert x.shape == (1, 3, 3000, 4000)
assert x.dtype == np.uint8

The [None] index adds the batch axis. After the transpose, normalize exactly as the training pipeline did: divide by 255 if it divided by 255, subtract the same channel means if it subtracted them. One step is free for replicated channels — the BGR-to-RGB reversal reorders channels, and yours are identical, so it changes nothing. Keep the transpose; it changes memory layout, not meaning.

Worked example: a 4000 × 3000 mono frame at 30 fps

Inputs: one Mono8 frame of 4000 × 3000 pixels at 30 fps — the resolution and frame rate the LS-CMR-0166 prints. Read as a (3000, 4000) uint8 array, each pixel is one byte. The detector's first convolution takes three input channels, from the [3, 48, 3, 2] arguments in its training summary.

The rule in words: a frame's byte count is height times width times channels times bytes per value.

• Raw frame: 3000 × 4000 = 12,000,000 px; at 1 byte per pixel that is 12,000,000 bytes, about 12 MB.
• After the channel repeat: 12,000,000 px × 3 channels = 36,000,000 bytes, about 36 MB, still uint8.
• Converted to float32: 36,000,000 values × 4 bytes = 144,000,000 bytes, about 144 MB per tensor.
• At 30 fps: 144 MB × 30 = 4,320 MB per second of tensor data the pipeline allocates and copies.

The expanded float32 tensor is 12× the raw buffer (3 channels × 4 bytes), so operation order is the decision: resize and crop while the frame is single-channel uint8, then repeat channels and convert dtype as the last two steps. The resize then touches 12 MB instead of 144 MB. Below about 1 MP the order hardly matters — a 3-channel float32 copy of a 1 MP frame is 1,000,000 × 12 bytes = 12 MB — so the rule bites exactly where multi-megapixel area scan cameras feed the model.

Checks before you blame the weights

• Shape at the boundary. One print(frame.shape, frame.dtype) on the last line before predict settles causes 1, 3 and 5.
• Round-trip test. Push one labelled training image through the inference pipeline and compare its detections with training time; falling confidence on identical weights is OpenCV ONNX confidence dropping at inference, the same error class seen from the results side.
• Value range. uint8 spans 0–255; a pipeline trained on 0–1 floats needs the divide by 255. A 255× scale error passes every shape check and quietly sinks every score.
• Bit depth. Cameras that output 10- or 12-bit mono arrive packed or in a 16-bit container. Unpack 10-bit camera data to 16-bit planes before any channel work, and check why 12-bit values top out near 65520 in a 16-bit frame so your normalization divides by the right maximum.
• Stored files. A dataset saved after conversion can be grayscale on disk while training assumed color; the green channel extraction error on stored grayscale images is that trap seen from the code side.
• Buffer format. When the camera SDK hands you a raw buffer, its byte size must match the pixel format you assume; matching the pixel format to the image buffer covers the signs.

When the channel fix is the wrong fix

Read the first convolution's declared arguments from your own training summary. [3, 48, 3, 2] means three input channels and the expansion above is correct. A summary that starts [1, ...] means a true 1-channel rebuild: feed (1, 1, H, W) and skip the repeat, which would fail at the layer instead of fixing anything.

Two neighbours of this error die on the same line. A file path string, or a None from a failed read, reaches the reversal as a non-array and fails before shape is ever discussed — fix the read. And 16-bit mono frames take the same three calls, but the normalization maximum is the container's 65535 or the packed 10/12-bit maximum, not 255: dividing a 16-bit frame by 255 leaves every value 257 times too large, because 65535 / 255 = 257.

Prevention: lock the contract in one wrapper

• Wrap the predict call in one function that asserts ndim, channel count and dtype, and route every caller through it so the contract cannot drift between scripts.
• Pin the image library version between the training and deployment environments, so the loader's returned shape cannot change under you.
• Keep one loading path: the same reader call and the same grayscale or color mode in training and inference.
• Write the wrapper once on-device too. Mono smart cameras run models against a mono buffer inside the camera, and the same (1, 3, H, W) contract applies. The MV-SC5060XM-08M-WBN is a 6 MP mono smart camera with an 8 mm M12-mount lens; the LS-SMC-0017 is an 8.8 MP smart camera with a C-mount and a global shutter in color or mono.

More integration failures of this kind — buffer layouts, bit depth, preprocessing drift — sit under the systems topic. To pick the camera around the frames instead of the workaround, the mono camera finder filters by resolution and interface, and the vision system configurator walks camera, lens and light in one pass. If a deployment is stuck on a shape or preprocessing mismatch, contact us with the traceback and the frame's shape.