JPEG camera to embedded display: decode before you store

You have a JPEG camera on a UART, a microcontroller, a microSD card and a serial LCD, and the plan is shoot, store, show. The open question is whether the display can open the JPEG file the camera produced. In almost every case it cannot. Serial display modules draw raw pixel data into a framebuffer; JPEG decoding is a CPU job their firmware does not do. This page fixes where the decode belongs, what storing decoded frames costs on the card, and which chain to build — plus the cases where the answer flips.

A JPEG camera hands you a file, not pixels

Set the resolution and quality registers, send the capture command, and the module returns a complete JPEG file: JFIF header, quantization tables, Huffman-coded 8 × 8 DCT blocks, framed by start and end markers on the UART. Those bytes are the compressed file. No pixel value exists in them until something runs the Huffman decode, dequantization, inverse DCT and colour conversion. Writing the bytes to the card unchanged gives a valid .jpg that a PC opens — and a display cannot.

The transport is the fragile part, not the format. These modules compress a VGA frame to a few tens of kilobytes at default quality; at 115.2 kbit/s with 8N1 framing the UART moves 11,520 bytes per second, so a 50 KB file takes about 4.4 s to arrive, and one dropped byte corrupts everything after it. The clean serial JPEG capture checklist for camera modules over UART covers the framing and checksum habits that keep files intact.

That file-versus-pixels split decides every choice below, and it is the same distinction behind image bytes vs pixel data: decode before processing.

Why the display module will not decode it

Serial LCD modules expose a command set: open a window, write pixels, blit a stored raw image, control backlight and touch. The pixel data they accept is already in framebuffer format — RGB565 at two bytes per pixel or RGB888 at three. Their firmware contains a blit, not an inverse DCT.

The decode arithmetic shows why it is a processor job. A 240 × 320 frame is 76,800 pixels, which is 1,200 luma blocks of 8 × 8 before any chroma is counted, and each block needs Huffman decoding, dequantization and an inverse DCT. A pixel-shovelling display controller does none of that, so JPEG bytes written to its card or framebuffer come back as noise.

The exception is a module whose command set includes an explicit draw-JPEG or draw-file call. Read the command list before you plan anything else; every decision below hangs on that one line.

Three chains that work

With 150 KiB or more of free RAM, decode the whole frame into a buffer and write it in one pass. Below that, stream: a block-based decoder writes each decoded line straight to the display or card and never holds a full frame. If the camera can output raw or YUV pixels instead of a JPEG, the third chain deletes the decoder entirely; camera module raw image capture: check before you code lists what to confirm on the module first.

Worked example: frame store for a 240 × 320 RGB565 panel

The case: a serial LCD of 240 × 320 pixels in RGB565, one snapshot on screen at a time, frames stored to a microSD card between shots. Panel size comes from the module's data; RGB565 is two bytes per pixel by definition.

The rule, in words: a raw stored frame costs width × height × bytes per pixel, and the JPEG file the camera produced keeps whatever size it arrived at, regardless of panel.

Result: each decoded snapshot stored for this panel costs about 150 KiB on the card, whatever the camera's JPEG weighed, and a 2 GB card holds about 13,000 of them (2,000,000,000 B ÷ 153,600 B ≈ 13,020) before file-system overhead.

The decision, with thresholds:

• Display takes raw pixels only, as usual → decode on the microcontroller before writing; budget one raw frame per snapshot on the card.
• 150 KiB or more of free RAM → decode into a full framebuffer, then write; simplest code path.
• Below about 160 KiB free → stream block by block to display or card, or move to a raw-output camera; do not buffer the frame in untested external memory.

Capturing raw instead: skip the decoder

If you choose the camera, raw output removes the decoder from the build. Check two things first. Bayer-format raw still needs debayering before a panel shows it — Bayer raw stream from an RGB camera: how to know you have it tells you how to confirm what the link actually carries. And raw frames are heavy: 153,600 B for this panel's worth of pixels, 614,400 B at VGA, so raw only suits links faster than a UART.

That is where machine-vision board cameras fit. They stream uncompressed frames over GigE or USB3 to a host, which converts once to RGB565 and stores or draws for the panel:

The MV-CB016-10GM-M-W keeps the link light at 1.6 MP mono behind an M12 mount with a global shutter. The LS-BCM-0003 is the faster choice: 3.1 MP at 120 fps over USB3, also global shutter, C-mount. The LS-BCM-0020 carries a 6.3 MP IMX178 sensor behind a CS-mount when the image needs more detail. The board-level cameras category holds the rest of the range.

Decoding on the microcontroller: what it costs

Microcontroller JPEG decoding is ordinary engineering now. Integer decoders built for MCUs — JPEGDEC is a common one — open the file on the card and emit decoded 8 × 8 blocks, so RAM need is a few kilobytes of working space plus one output row: 240 px × 2 B = 480 B per row for this panel, against a 150 KiB whole-frame buffer. The desktop-side failure that pushes people back toward raw capture appears in JPEG decode hangs on raw pixel data: encode before decoding; the embedded analogue is buffering a whole frame you do not have the RAM for.

Timing: a QVGA frame is 1,200 blocks, and an optimized integer decoder on a 100 MHz-class MCU clears it in tens to hundreds of milliseconds — a beat, not a pause. That serves snapshot-on-event display well and marginal video poorly; for anything resembling live view, use the display-side chain or raw capture.

Scaling: the camera shoots 640 × 480 while the panel shows 320 × 240 in landscape. Both are 4:3, so a 2:1 nearest-neighbour downscale fills the panel exactly; crop instead when the operator must read a detail.

When the usual answer is wrong

Three cases flip the decode-before-store rule.

The display side can decode. Modules built on OS-class hardware, and HMI and SCADA software, decode JPEG and MJPEG themselves — the MJPEG and JPEG still streaming setup for IP cameras in HMI software shows the pattern. If the display's command set has a draw-JPEG call, store the JPEG untouched and keep the card space.

The goal is a decision, not a picture. If the chain exists to check presence, gauge a part or read text, end it at a smart camera that inspects on the device and returns a verdict — no decoder, no panel:

The LS-SMC-0063 inspects at 1.6 MP and 60 fps with a 16 mm lens and built-in white light over GigE. The smart cameras category is the place to compare the range.

You control both ends and want live view. Skip JPEG altogether: capture raw or YUV, convert to RGB565 once, and store or stream pixels. The same skip-the-codec principle, argued for desktop pipelines, is in unencoded capture to a NumPy array: skip JPEG encode and decode.

Checks before you build

• Read the display module's command list end to end and mark any call that takes a file or JPEG. Pixel writes only means the decode is yours. Why it matters: every downstream choice hangs on this one line.
• Confirm the panel's pixel format and byte order — RGB565 versus RGB888, and which byte arrives first — before writing the converter. Why: a swapped byte order shows as false colours, a bug that looks like a broken camera.
• Set the camera's resolution and quality registers to the smallest JPEG that still answers the question. Why: halving the file halves the 4.4 s transport.
• Time the whole chain on the bench: capture to file, decode, card write. The transport and decode figures above tell you where the seconds go.
• Size the card from the raw arithmetic: 150 KiB per 240 × 320 snapshot, about 13,000 per 2 GB. Why: decoded storage is the silent cost of the first chain.

When the chain outgrows UART JPEG — live view, more resolution, measurement — move to uncompressed capture on GigE or USB3: the camera finder filters by interface and shutter, the camera configurator pairs the camera with lens and cable, and the systems guides cover the host-side storage and decode decisions around this one.