Image bytes vs pixel data: decode before processing

A byte array tells you how data is stored in memory, not what those bytes mean. An image returned by a file service might contain a complete compressed file, while a camera acquisition buffer might contain pixel samples, padding and metadata. Treating both as a flat grid produces incorrect colors, impossible dimensions or invalid reads. This explainer shows you how to identify the representation, decode when necessary, and establish a reliable contract before accessing individual pixels.

First decide which representation you have

Do not calculate pixel positions until you know what the producing API promises. A method called “get image data” does not, by itself, promise decoded pixels. Consult its documented return type and format description.

Three representations commonly appear in machine-vision applications:

A PNG or JPEG file is not an array of directly addressable RGB samples. Even an uncompressed image file can contain headers and offsets before its pixel content.

Conversely, a camera buffer is not necessarily a complete image file. Passing an arbitrary acquisition buffer to a file decoder is not a substitute for identifying its pixel format.

The distinction also matters when transferring binary objects through an API. Our explanation of binary data handling in GenICam file access covers a related boundary: preserving bytes during transport does not establish what those bytes represent.

Decode files instead of guessing their layout

For encoded input, use a supported decoder rather than implementing decompression or searching for a presumed pixel offset. Decoding can involve more than decompression: palette expansion, scanline reconstruction and color conversion may also be necessary.

A practical decoding sequence is:

1. Read the API documentation to establish whether the return value is a file or a pixel buffer.
2. Identify supported file formats using the decoder's format detection. Treat the filename extension as a hint, not proof.
3. Inspect dimensions and impose limits before allocating the decoded image, where the decoder permits this.
4. Decode the image and request a documented output representation, such as interleaved RGB with unsigned samples.
5. Retrieve the actual decoded dimensions, channel layout, sample type and row stride.
6. Reject unsupported or inconsistent output before calling the processing algorithm.

File signatures help identify a likely format, but they do not prove that the complete file is valid. Let the decoder validate its structure and report errors.

If your runtime cannot decode the required format, add a compatible, supported library or move decoding to a supported service. Reinterpreting the encoded bytes as grayscale does not recover the missing functionality.

Define the pixel descriptor before indexing

Once decoding is complete, a one-dimensional buffer can represent a two-dimensional image perfectly well. The missing information is its layout.

Record these properties alongside the buffer:

• Width and height: the decoded image dimensions, not dimensions guessed from file length.
• Sample type: unsigned integer, signed integer or floating point.
• Container size and meaningful bit depth: these may differ when a sample occupies a larger storage container.
• Channel meaning and order: grayscale, RGB, BGR, RGBA, Bayer samples or another documented representation.
• Organization: interleaved channels or separate planes.
• Row stride: the byte distance between successive stored rows.
• Image origin and orientation: which stored row corresponds to the displayed top edge.
• Valid buffer extent and ownership: how much memory is accessible and how long it remains valid.

Do not infer four channels merely because an image is described as color. A decoded color image might have three channels, four channels, or a palette that the decoder expands. A monochrome image can also carry an alpha channel.

For an acquisition buffer, obtain these properties from the SDK's frame descriptor. For a decoded file, obtain them from the decoder's output object.

Storage width deserves particular attention. A sample with limited meaningful precision can occupy a wider integer container or be packed across byte boundaries. The machine-vision camera bit-depth guide explains that distinction. Packed samples need unpacking according to their documented format before ordinary sample indexing works.

Calculate offsets using stride, not file length

For an interleaved image with byte-aligned, equally sized channel samples, the following equation gives a channel sample's starting byte offset. Coordinates and channel indices start at zero.

This equation does not apply unchanged to planar images, packed samples or subsampled color formats. It also assumes that increasing the row index follows the documented storage direction.

Consider a hypothetical decoded image with 640 columns, 480 rows and three unsigned channels, each occupying one byte. With no row padding:

That is 900 KiB of pixel storage, using 1 KiB = 1,024 bytes. The encoded file might be smaller or larger; its length is not a reliable substitute for this calculation.

Now change the width to 641 columns. Suppose the decoder aligns each stored row to a multiple of four bytes:

There is one padding byte after each active row. For pixel x = 10, y = 20, channel index 2, with the image starting at offset zero:

Using the active row width instead would produce 38,492 bytes. The error has accumulated to 20 bytes by that row.

Before reading, check that the complete sample lies within the accessible buffer. Use checked arithmetic when multiplying dimensions and strides. For positive stride, the final active row can end before the end of a fully padded allocation; use the API's reported extent rather than demanding one guessed allocation size.

Interpret channels, alpha and numeric values explicitly

Correct addressing is only half the job. You must also interpret the value at that address correctly.

For an unsigned, single-byte channel, a signed byte container can display values above its positive range as negative numbers. For example, the bit pattern representing unsigned 255 may appear as signed -1. Convert using an unsigned interpretation before arithmetic; do not clamp the negative value to zero.

For multibyte samples, establish byte order. The byte sequence 0x01 0x02 represents 513 when interpreted as a little-endian unsigned integer, but 258 when interpreted as big-endian. Prefer the decoder's typed accessors when available.

Channel names can also describe logical values rather than memory order. An integer described as ARGB does not automatically imply that its first addressable byte is alpha. Integer packing, byte order and the API's accessor rules all matter.

If red objects appear blue, first verify channel order with a known test image. The red and blue channel troubleshooting guide covers this failure separately from file decoding.

Alpha introduces another distinction:

• Straight alpha: stored color values are not multiplied by alpha.
• Premultiplied alpha: stored color values have already been multiplied by alpha.

Ignoring that distinction can darken colors near transparent edges. Decide whether the algorithm should ignore alpha, composite against a defined background, or work with another documented representation.

For quantitative intensity processing, also establish the color space and transfer function. Display-encoded channel values are not automatically proportional to scene light. Avoid comparing them with linear measurements unless the conversion is defined.

Verify the decoded image with controlled tests

Start with a small synthetic image rather than a complicated production scene. It makes indexing errors easier to locate and separates decoding mistakes from algorithm behavior.

Use a lossless test image containing distinct red, green, blue, black and white regions. Give its corners different colors so that rotations and vertical reversal are visible. Include a width that is not naturally aligned to the decoder's row boundary.

Check the following observations:

Use exact sample comparisons only where the format and conversion preserve those values. Lossy compression and color conversion can change channel values, so a JPEG round trip is unsuitable for an exact byte-equality test.

Also verify how the image object is wrapped by your processing library. Correct bytes paired with an incorrect matrix element type can produce distorted dimensions or repeated content. See matching matrix types to pixel formats.

Finally, confirm memory lifetime independently. A perfectly decoded view still fails if its backing buffer is released or reused. The camera frame memory lifetime guide explains why retaining a reference to an image wrapper may not retain the underlying frame.

Choose hardware without confusing transport and representation

This is primarily a software boundary problem. Replacing a camera will not make an encoded file directly indexable, and changing the transport interface does not remove the need for a pixel descriptor.

When selecting area scan cameras, document the required acquisition pixel formats and confirm their availability through the supported SDK. Use the camera finder to narrow hardware choices, then verify the software contract for the selected configuration.

The LS-CMR-0053 is a GigE area scan camera. Its listing provides a hardware starting point, not a guarantee about how an application exports image bytes.

The LS-CMR-0166 is a USB3 area scan camera. Apply the same representation checks when evaluating its acquisition path.

For processing hosted on vision controllers, verify runtime compatibility, decoder support and available memory for the complete pipeline. An input file, decoded image, converted image and algorithm workspace can coexist, so memory demand is greater than the compressed input size alone.

If your workflow saves decoded samples for later measurement, review saving high-bit-depth images without losing intensity data. Decoding correctly now does not prevent a later export operation from discarding precision.

Know when decoding is the wrong first step

Do not introduce a file codec when the acquisition API already provides usable pixel samples. Encoding a frame and immediately decoding it adds work; a lossy format can also change the measurements.

Likewise, do not assume that every buffer needs conversion to three-channel color. A monochrome algorithm may accept the native grayscale representation. A Bayer-processing algorithm may intentionally require the sensor sample arrangement rather than a demosaiced image.

The right boundary is the algorithm's input contract. Convert only what differs: sample packing, channel order, organization, orientation or numeric representation. Preserve a direct view when it is valid, and make an owned copy when downstream processing outlives the source buffer.

Before connecting the production algorithm, complete this checklist:

• Identify whether the input is an encoded file, decoded image or acquisition payload.
• Record dimensions, sample type, channel layout, stride and orientation.
• Apply decoded-size limits and validate all calculated offsets.
• Test known colors, row boundaries, transparency and intensity values.
• Confirm buffer ownership and release behavior.
• Document every conversion between acquisition, processing and storage.

For a new acquisition setup, start with the camera configurator. If the byte representation is still unclear, use technical support with the API return description, pixel-format name and image descriptor so the acquisition and decoding boundaries can be checked together.