CRC check on an image file: read the bytes, skip the pixels

Your area scan camera writes hundreds of BMP and JPEG frames per shift, and the quality department wants a cyclic redundancy check stored with every file. The job is simpler than it looks: a CRC consumes the file's raw bytes, so the image never gets decoded. This guide gives you the byte-level read in Python and C++, the AUTOSAR 8-bit parameters with their test values, the runtime cost on a real 6 MP frame, and where a pixel-level check is the right exception.

The steps at a glance

1. Pick the algorithm: CRC-8 SAE J1850 only when a receiving specification names it, otherwise CRC-32.
2. Verify your implementation against the catalogue check value for the ASCII string "123456789" before touching a real file.
3. Open the image file in binary mode and read it in 65,536-byte chunks, folding each chunk into the CRC register.
4. Write the finished CRC into a sidecar file beside the image at save time.
5. Recompute the file CRC on a schedule against the sidecars and treat any mismatch as a damaged file.

What data does a CRC read from an image file?

A CRC reads every stored byte of the file in file order: header fields, palette entries, compressed blocks and pad bytes, exactly as they sit on disk. It never touches decoded pixel values. So the file reaches it through a binary-mode read, never through an image decoder.

Binary mode matters on Windows. A file opened in text mode translates every carriage-return/line-feed pair (0x0D 0x0A) into a single 0x0A on the way in. The byte count drops, and every CRC after that point shifts. The 'rb' flag in Python and std::ios::binary in C++ both switch the translation off.

Two bytes at the start tell you what you are holding: 0x42 0x4D ('BM') for BMP, 0xFF 0xD8 for JPEG. Check them before trusting an extension, as in Load image files by content when the extension is wrong.

Why is imread() the wrong tool for this?

imread() returns decoded pixel values, not stored bytes. For a JPEG those values are reconstructed from compressed coefficients. For a BMP the rows come back top-down, unpadded and palette-expanded. A checksum over the returned matrix matches neither the file on disk nor any other tool's file-level result.

The JPEG pixels come from inverting a transform over stored coefficients, so there is no byte in the file you can point to and call "pixel 3,7". The BMP decode reverses the bottom-up row order and strips the 4-byte row padding, which is why 3072-pixel rows survive unchanged but odd widths do not. The decoder is doing its job correctly in both cases; it is answering a different question. The gap between what a file stores and what a decoder returns is the subject of Image bytes vs pixel data: decode before processing.

Which parameters define the AUTOSAR 8-bit CRC?

The 8-bit CRC used in AUTOSAR E2E Profile 1 is the SAE J1850 one: polynomial 0x1D, initial value 0xFF, final XOR 0xFF, no bit reflection. Fed the ASCII test string "123456789" byte by byte, a correct implementation returns 0x4C, and that single check catches every wrong constant.

Reflection means each byte enters the register least-significant bit first; CRC-8 SAE J1850 uses none, so the loop shifts left and the polynomial appears as 0x1D. If your specification names a different polynomial or initial value, change those two constants. The read-and-fold structure of the code below stays identical. Decision: CRC-8 SAE J1850 when a receiving specification demands the AUTOSAR variant, CRC-32 when the archive policy is yours to set.

How do you compute it in Python?

Open the path with open(path, "rb"), read it in 65,536-byte chunks, and fold each chunk into the CRC register before the next read. On the 6.29 MB example below that is 97 read calls instead of six million single-byte ones. Text mode is the one trap to avoid.

def crc8_sae_j1850(data: bytes, crc: int = 0xFF) -> int:
    for byte in data:
        crc ^= byte
        for _ in range(8):
            if crc & 0x80:
                crc = ((crc << 1) ^ 0x1D) & 0xFF
            else:
                crc = (crc << 1) & 0xFF
    return crc ^ 0xFF

# Self-test: the catalogue value for the ASCII string "123456789"
assert crc8_sae_j1850(b"123456789") == 0x4C

def crc8_of_file(path: str, chunk_size: int = 65536) -> int:
    crc = 0xFF
    with open(path, "rb") as fh:
        while True:
            block = fh.read(chunk_size)
            if not block:
                break
            crc = crc8_sae_j1850(block, crc)
    return crc ^ 0xFF

Related byte-level fixes around camera files are collected on the systems topic page.

How do you do the same in C++?

std::ifstream opened with std::ios::binary hands you the raw bytes through istreambuf_iterator. No std::hex formatting belongs anywhere in the path: the CRC consumes the bytes as numbers. Format only the finished 8-bit result, 0x4C-style, when you write the log line.

#include <cassert>
#include <cstdint>
#include <fstream>
#include <iterator>
#include <vector>

uint8_t crc8_sae_j1850(const std::vector<char>& data,
                       uint8_t crc = 0xFF) {
    for (char c : data) {
        crc ^= static_cast<uint8_t>(c);
        for (int bit = 0; bit < 8; ++bit) {
            crc = (crc & 0x80)
                ? static_cast<uint8_t>((crc << 1) ^ 0x1D)
                : static_cast<uint8_t>(crc << 1);
        }
    }
    return static_cast<uint8_t>(crc ^ 0xFF);
}

int main() {
    const std::vector<char> probe{'1','2','3','4','5','6','7','8','9'};
    assert(crc8_sae_j1850(probe) == 0x4C);   // catalogue value

    std::ifstream f("frame_000047.bmp", std::ios::binary);
    std::vector<char> bytes{std::istreambuf_iterator<char>(f),
                            std::istreambuf_iterator<char>()};
    const uint8_t crc = crc8_sae_j1850(bytes);
}

Run the assert on the nine ASCII digits before pointing the function at real frames; if it passes, the constants are right.

What does the CRC cost on a real 6 MP frame?

A mono8 BMP from a 6 MP camera carries 3072 × 2048 = 6,291,456 pixel bytes plus a 1,078-byte header and palette, so the CRC sees 6,292,534 bytes. A table-driven CRC-8 clears that in single-digit milliseconds; the bitwise loop shown above spends about 50.3 million iterations on it.

The 1,078 breaks down as a 14-byte file header, a 40-byte info header and 256 × 4 palette bytes. For the 3072 × 2048 frame: 1078 + 6,291,456 = 6,292,534 bytes, and ceil(6,292,534 / 65,536) = 97 read calls, of which the last carries 1,078 bytes. The bitwise loop multiplies the work by eight: 6,292,534 × 8 = 50,340,272 inner iterations, roughly 50-100 ms in C at 1-2 ns per iteration. A 256-entry table cuts it to one lookup per byte, about 1-2 ns, so 6-13 ms — reading the file from an SSD often costs more than the arithmetic. For widths not divisible by 4, add 4 − (width mod 4) bytes per row to the total.

When do you need a CRC over the pixels instead?

Only when the checksum must survive a change of container or compression, such as comparing an archived frame against a re-saved copy, do the pixels matter. Decode both to the same uncompressed layout, CRC the pixel buffer, and store that value separately; it never equals the file CRC.

Decode with the unchanged-pixels flag, confirm both dimensions and pixel types match, and checksum the contiguous buffer: tobytes() on the decoded array in Python, or data over total() × elemSize() bytes in C++ when isContinuous() is true.

For logging per-frame pixel values rather than checksums, see Read a sequence of images and log pixel values to file.

How does the check fit a production image archive?

Write the checksum beside each image at save time, recompute it on a schedule against the archive, and treat any mismatch as a damaged file. On the line this belongs on the vision controller that already holds the images, not on an office PC that copies them first.

Name each frame with a sidecar — frame_000047.bmp plus frame_000047.crc holding one hex line — and let the nightly pass diff the directory against the sidecars. Keep the originals untouched; annotated copies are a viewer's convenience, as in Save an image with drawn overlays as a file any PC can open.

The MV-VC3301H-128G66 pairs an Intel Core i3 with 8 GB of memory and a 128 GB SSD, and the nightly pass runs beside the inspection on it. The MV-VC2140-128G40-1T adds a 1 TB hard drive behind the same 8 GB and 128 GB SSD combination.

At 6.29 MB per mono8 frame, a 1 TB drive holds about 159,000 frames before file-system overhead. Size the controller by the archive you actually keep, not by the camera count.

Decision: CRC-8 SAE J1850 only where a specification demands it, CRC-32 for everything you own, sidecars written at save time, and a pixel CRC only for re-encode comparisons. If you are sizing the controller and the archive together, send the frame rate and retention period through the contact page and we will work the numbers with you.