JPEG decode hangs on raw pixel data: encode before decoding

You call the hardware JPEG decoder with a frame your camera library just grabbed, and the call never comes back. The serial prints stop and the loop stops; only a reset brings the board back. The frame itself is fine — you have watched the pixels scroll past on the serial link. This guide is for anyone bringing up a camera module on an MCU-class board with a hardware JPEG codec. It shows what the decoder is actually waiting for, gives you a first-bytes test that takes under a minute, and works through both fixes with the buffer arithmetic shown: feed the decode call real JPEG data, or drop the decoder and process the raw pixels directly.

The symptom: a ready decoder that never finishes

The setup follows the standard bring-up pattern. cam.begin(CAMERA_R320x240, 30) sets the camera to QVGA, 320 × 240, at 30 fps and cam.grab(fb) fills fb with one frame; then the decode goes in, in the shape of HAL_JPEG_Decode(&hjpeg, fb, (uint32_t)(320 * 240), fb2, (uint32_t)(320 * 240), HAL_MAX_DELAY) — and parks there forever.

Three observations narrow the fault to the input. First, hjpeg.State reads 1, the ready state, before the call, so initialization is fine. Second, replacing HAL_MAX_DELAY with a timeout of 0 makes the call return immediately through its own timeout check, which places the wait inside the decode call rather than in the grab. Third, writing fb out over serial shows a live grayscale image, so the camera delivers real pixels. A ready decoder is being fed; nothing it recognizes as a JPEG is arriving.

What the decoder is waiting for

A JPEG decoder does not read pixels; it parses a bitstream defined by ITU-T recommendation T.81. It expects the byte pair FF D8, start of image, then walks marker segments — quantization tables at FF DB, Huffman tables at FF C4, a frame header at FF C0, a scan header at FF DA — before the entropy-coded data, and finishes at FF D9, end of image. Only past the scan header does it know width, height and sampling; only during entropy decode do output pixels flow.

A raw grab buffer holds none of that. QVGA grayscale at 8 bits is 320 × 240 = 76,800 pixels, one byte each: 76,800 bytes of pixel values beginning with whatever the top-left of the scene is, around 0x80 mid-gray on a typical target. The codec scans for a header that is not there, never reaches its done condition, and the blocking call waits forever on a flag that never sets. The ready state you read before the call confirms the half people suspect first: the peripheral is fine; the input is wrong.

The buffer math, worked

For the QVGA grayscale frame in the symptom:

• Raw frame: 320 × 240 = 76,800 pixels at 1 B each = 76,800 bytes, and the grab buffer is full edge to edge.
• The same frame as a real JPEG: photographic content at typical encoder settings compresses roughly 5:1 to 20:1, so at 10:1 it is 76,800 / 10 ≈ 7,680 bytes ≈ 7.7 KB, from about 15 KB for a busy, noisy scene at 5:1 to about 3.8 KB for a flat scene at 20:1.
• Decoded sizes to plan for: grayscale output 76,800 B; a 4:2:0 color source decoded to planar YCbCr at 320 × 240 × 1.5 = 115,200 B; the same source to RGB888 at 320 × 240 × 3 = 230,400 B.

The first line carries the diagnosis on its own. A buffer filled exactly to width × height × 1 bytes is raw pixel data by definition; a compressed QVGA image would occupy a few kilobytes inside it, not all of it.

The first-bytes test

Read the head of the buffer before any decode:

/* fb is uint8_t fb[320 * 240], filled by cam.grab(fb) */
if (fb[0] == 0xFF && fb[1] == 0xD8) {
    /* JPEG bitstream: the decode call may proceed */
} else {
    /* raw pixels: no JPEG decoder will ever finish on this */
}

1. Print the first two bytes. FF D8 means a JPEG is present; anything else is raw pixels, and that is the hang.
2. Check the tail when the length is known. FF D9 closes a complete bitstream; its absence means truncation.
3. Decode one known-good JPEG from storage through the same call. Success clears the decoder, the buffers and the API sequence in one test, leaving the camera data as the only variable left to convict.

Keep the two-byte check in the grab path permanently. It turns a production hang into a logged, failed frame.

Likely causes, most probable first

1. The buffer holds raw pixels. Near-certain whenever the input is a camera-library grab and the capture example is built around raw byte capture. Test: first bytes are not FF D8 and the length is exactly 76,800 B. Fix: either fix below.
2. Input length is the array capacity, not the bitstream length. A real JPEG is smaller than the array holding it; passing the capacity sends the codec reading past end-of-image into adjacent memory. Test: print the encoder's output length or the file size and pass that. Fix: pass the true length.
3. Output buffer smaller than the decoded image. A color decode needs 115,200 B at 4:2:0 or 230,400 B at RGB888, against the 76,800 B the raw grab needed. Test: recompute with the formula above. Fix: size the output for the decoded layout, not the input.
4. Codec left mid-stream by an aborted decode. After any timeout, start from a clean slate: re-run the codec's initialization routine before the next decode (the checklist below carries the test).

Fix 1: feed the decoder a real JPEG

JPEG comes from two places. One is a camera that produces the bitstream itself — some compact modules and smart cameras carry an onboard encoder, and the module's feature list is where you check. The other is an encode step you add, and it belongs where cycles are abundant: software encoding costs far more than decoding, and on an MCU it competes directly with the control loop.

For a machine-vision build the encode therefore lives on the host. The cameras deliver raw frames and a controller does encode, decode and analysis: the MV-VC2140-128G40-1T carries a single-camera cell with 8 GB of memory, two USB 3 ports and 4 built-in light channels, and the MV-VC5525-256G70-NN steps up to multi-camera cells with 32 GB of memory, PoE camera ports and 8 built-in light channels.

With a real bitstream in hand the hardware codec earns its keep; the silicon vendor's application note on its JPEG codec peripheral walks the decode flow from header parse to output conversion. Encoding on the host has its own traps: if your encoder consumes float pixel data, see the fix for JPEG output that comes out black.

Fix 2: skip the decode and use the pixels

If the pipeline's purpose is measurement — dimensions, presence, surface defects, code reads — the JPEG step should not exist. Raw pixels are the working format of machine vision: exact sensor values, no block artifacts, no decode latency, and a buffer size known exactly, 76,800 B here. Decode-then-analyze adds a lossy round trip and a failure mode for no gain. The same rule holds one level up on the host: capture unencoded frames straight into the processing array, as in unencoded capture to a NumPy array.

The LS-BCM-0003 gives that architecture its front end: 120 fps at 3.1 MP over USB3, global shutter, board-mount.

Choosing the camera is its own decision — board-level camera selection for embedded vision machines covers the resolution, shutter and mount trade-offs, and the board camera evaluation checklist turns bring-up into a repeatable test. The host-side twin of this whole bug, feeding a decoder pixels where it wants encoded bytes, is the subject of image bytes versus pixel data.

Which fix your build needs

If any consumer outside the inspection code needs the images — an archive, an HMI thumbnail, a slow network link — take Fix 1 and encode at the source with CPU headroom. If the only consumer is the measurement code, take Fix 2 and delete the JPEG step. If the board already receives JPEG from somewhere, decode those bitstreams and manufacture no new ones on the MCU.

Checks before the next decode call

• Head and tail markers. FF D8 first, FF D9 last — two comparisons that catch this entire bug class at the boundary.
• True input length. The bitstream's own length, never the array capacity.
• Output sized by the decoded layout. 76,800 B grayscale, 115,200 B at 4:2:0, 230,400 B at RGB888 for QVGA.
• Bounded timeout. Five to ten times your measured worst decode; never HAL_MAX_DELAY in code that runs a cell.
• Re-init after a timeout. Re-run the codec's initialization routine, then read the state register and confirm it is back to the ready value before the next decode.
• Format assert in the grab path. A wrong-format frame fails fast and logs; it never parks a loop.

That assert habit scales past pixels: every raw stream is decodable only against its documented format, and the DBC-file walkthrough for decoding raw CAN frames applies the same input contract to bus data.

When the first bytes are FF D8 and it still hangs

• Truncated bitstream. An input length shorter than the real JPEG leaves the decoder waiting for entropy data that never completes; pass the true length.
• Progressive or arithmetic-coded files. Many hardware codecs handle baseline sequential JPEG only; re-encode the asset as baseline.
• MJPEG streams with padding. Extraneous bytes before the next start marker choke the marker walk — the corrupt JPEG data fix covers that case.
• Color layout mismatch. Decoders emit YCbCr-ordered data; conversion to the layout the renderer or algorithm expects is a separate, explicit step.

From here the decision is the one this bug forced all along: fix the format contract for every hop, then pick hardware that matches it. The camera finder filters the camera side, industrial mini PCs cover the host side where the encode step lands, and more pipeline pieces sit on the systems topic page. If a port budget or frame rate does not close, the contact page reaches the applications team.