Lossless streaming image compression for machine vision streams

Your camera stream does not fit. The GigE port saturates at 3 fps on a 20 MP sensor, the recorder fills between runs, or the CPU falls behind and drops frames. You need compression that is lossless for measurement, runs pixel by pixel, holds memory to a line buffer and carries no dictionary. This page gives the algorithm family that meets those limits — delta prediction followed by Huffman-only entropy coding. It also gives the ratios to expect (1:1.5 to 1:2 on real-world 16-bit grayscale), the bandwidth arithmetic at 20 Mpix/s, and the rule for when coding saves a link and when only a wider interface will.

Which lossless coder works with only a one-line buffer?

Use a left-neighbour delta predictor feeding a Huffman-only entropy coder: it needs exactly one line buffer, carries no dictionary, and delivers 1:1.5 to 1:2 on real-world 16-bit grayscale and slightly above 2:1 on 8-bit.

Two stages do the work. The predictor subtracts the left neighbour from every pixel, so the residuals of a real scene pile up close to zero. The entropy stage writes those small residuals with short Huffman codes from a fixed table and appends the bits to the output stream. Because the table is fixed and nothing is learned from the data, memory is constant: one line of 2 KB at 1,024 pixels plus a few KB of code lengths.

Streaming falls out of that structure. Each residual is coded as soon as its pixels exist, so latency stays under one line plus the bits in flight, and the output feeds a socket or a disk as it forms. The decoder mirrors the encoder — the predictor only needs pixels it has already rebuilt — so decode runs in the same one-line budget. PNG's "sub" filter is exactly this left-neighbour delta, and Huffyuv-class codecs reach slightly above 2:1 at 8 bit with a median of the left, upper and upper-left pixels, still on one line buffer.

How much does delta plus Huffman coding actually compress?

Plan on 1:1.5 to 1:2 for padded 16-bit grayscale, about 2:1 for 8-bit, and roughly 1:2 for PNG's sub filter with Huffman-only zlib on typical photos; noise-dominated frames fall well below that.

These are reported results on photographic content, not worst cases. Several lossless RAW designs in camera firmware code the delta against the previous pixel of the same colour — a Huffman-coded length followed by that many raw bits — and they routinely land at 1:1.5 to 1:2, the same residual distribution a left-neighbour predictor produces. Lossless comparisons on 16-bit grayscale keep Huffman as the entropy coder of choice, and classic lossless JPEG, which is DPCM plus Huffman, sits at the bottom of the ratio table: the predictor matters more than the entropy stage.

Textured parts and backgrounds behave like the photographic content behind these numbers. Chequerboards and pure noise are the exceptions, and the noise case gets its own number further down.

How much bandwidth does it free at 20 Mpix/s?

At 20 Mpix/s and 16 bit padded the raw stream is 40 MB/s, which fits one GigE link with room to spare, and a 1:1.5 to 1:2 coder cuts it to 20-27 MB/s — enough headroom for a second camera on a dual-port NIC.

Worked through: 1,024-pixel lines at 20 Mpix/s is about 19,530 lines/s. The raw stream is 40 MB/s, or 320 Mbit/s. A 1 GbE link runs at 125 MB/s line rate; after packet framing, budget about 110 MB/s (an estimate — framing varies with packet size). The raw stream fits with roughly 2.7x headroom, so coding is optional on the link itself. Its value is the recorder: at 1:2 the 40 MB/s becomes 20 MB/s, and a 1 TB drive holds about 14 hours instead of 7.

How much frame rate one GigE link carries at a given resolution is its own line-by-line budget; the GigE Vision for a high speed camera: how much frame rate fits article walks it.

Any 20 MP camera on one GigE link sits at the edge of this arithmetic: a full-resolution 16-bit frame carries about 40 MB, so raw output tops out near 3 fps (125 / 40), and a 1:2 coder lifts that toward 6 fps without touching the camera. A 20 MP GigE camera in this stream class:

USB3 widens the budget to roughly 400 MB/s usable (500 MB/s line rate less protocol overhead). A 12 MP padded 16-bit frame is 24 MB, so 10 fps costs 240 MB/s and fits without coding; coding then serves storage, not the link. A 12 MP USB3 camera in the high-speed class:

Multi-camera hosts close the same arithmetic per port. Two coded GigE streams at 20-27 MB/s each fit one dual-port NIC with margin, and one camera per port keeps a reset on one stream from stalling the other; the assign cameras to ports reliably in a multi-camera system article covers the enumeration side.

Why not just run zlib, LZMA or a PNG encoder on the stream?

Use zlib's HUFFMAN_ONLY mode if you want a library instead of your own coder: it drops the dictionary, keeps memory in the kilobytes and reaches about 1:2 on typical photos, while default zlib and LZMA spend 32 KB windows to megabytes of dictionary and bursty latency.

Default deflate is LZ77 matching plus Huffman, and the matching pass is what costs you: a 32 KB sliding window, hash tables that grow with the data, and output that arrives in bursts as long matches land. The HUFFMAN_ONLY strategy switches the matcher off and leaves pure Huffman coding with kilobytes of state and a steady rate — the same class of coder as the hand-rolled one above. PNG hands you the predictor for free: its "sub" filter is the left-neighbour delta, so a PNG encoder set to "sub" plus HUFFMAN_ONLY measures your ratio in minutes.

For the production stream, drop the PNG container — line-filter bytes and chunk framing are pure overhead — and run predictor plus table directly; the file side, once you do keep files, is covered by how to save 16-bit grayscale camera images reliably.

The risk in this layer is pace, not ratio. An encoder slower than acquisition backs up the transport buffer, and newest-only handling turns the backlog into missing or duplicated frames — see latest image only returns duplicate frames: drain the buffer queue and acquisition faster than processing: fix the frame rate gap. Budget a spare core per 50 MB/s of coded stream on a modern CPU; past a few hundred MB/s per stream, the coder moves into the FPGA line pipeline or the link gets wider.

Can JPEG-LS or JPEG 2000 meet a pixel-by-pixel budget?

No: JPEG-LS needs context from several neighbouring lines and effectively works in passes, JPEG 2000 buffers tiles and 4-8 lines minimum per coder, and lossless JPEG posts the lowest ratios of the classic set on 16-bit grayscale — none of them meets a strict one-line, pixel-by-pixel budget.

JPEG-LS is the ratio leader of the classic lossless set, which is exactly why it tempts: its context model reads across several lines and its rate control re-passes the data rather than emitting it once. Wavelet coders buffer whole tiles and need 4 to 8 lines at minimum before they beat a median predictor. Old lossless JPEG streams in one line but compresses least on 16-bit grayscale; your left-neighbour design is essentially that coder with a better-chosen residual.

Decision: if a few lines of buffer are affordable and ratio is the binding constraint, JPEG-LS is the upgrade path. Under a one-line, pixel-by-pixel budget, predictor plus Huffman wins on structure, and its 1:1.5 to 1:2 sits close enough to the leaders that the simpler pipeline usually pays more than the missing ratio points.

What changes when pixels are 10 or 12 bit padded to 16?

Keep the pixels padded to 16-bit words and let the Huffman table absorb the wasted bits: the residual entropy lives in small symbol values, so padding costs container size, not ratio, and your stream stays byte-aligned at 40 MB/s for 20 Mpix/s.

The raw stream costs the container, not the true bit depth: 20 Mpix/s padded to 16 bits is 40 MB/s whether the sensor delivers 10, 12 or 14 significant bits. The ratio moves the other way — fewer significant bits concentrate deltas into fewer symbols, so 12-bit content compresses slightly better than full 16-bit. Keep padded words on the streaming path: byte alignment survives, and the table simply gives longer codes to the values padding wastes.

Packed formats change the arithmetic and add work. A packed 12-bit stream is 1.5 bytes per pixel, so 20 Mpix/s is 30 MB/s raw, but every pixel needs bit-shifting on ingest, and a mis-read packed buffer shows up as half the expected pixels — why your 16-bit image buffer has half the expected pixels is that failure, and unpack Mono12p to a 16-bit image from the raw buffer shows the unpack step. Check what a pixel-format change does to frame rate before you switch — changing pixel format halves your frame rate: check linked attributes. For archival, keep 12 bit as 12 bit end to end: record a 12-bit grayscale video file without dropping to 8 bits.

When does compression not save the link?

If the raw stream is more than about 2x over the link, no entropy coder saves it: change the interface, drop to ROI or binning, or lower the frame rate, because realistic lossless ratios top out near 1:2 on padded 16-bit data.

Work in bands. Up to about 1.5x over: code it, since real-world frames beat 1:1.5. Between 1.5x and 2x: code it and trim frame rate or ROI to close the rest. Over 2x: the required ratio is beyond what prediction plus Huffman reaches on padded 16-bit data, so the pipe changes — binning, a reduced ROI, or a faster interface.

Numbers decide the band. A 25 MP sensor at 25 fps is 625 Mpix/s, so the padded stream is 1,250 MB/s against about 1,150 MB/s usable on one 10GigE link (1,250 MB/s line rate less overhead): a 1:1.5 coder fits it at 833 MB/s. The same sensor at 60 fps is 3,000 MB/s, a 2.6x shortfall; the fix is hardware — a quad-link CXP-12 camera carries about 6,200 MB/s at line rate — or the frame rate comes down. Cameras built for this band ship 10GigE on board — a 25 MP example:

Two limits set the ceiling from there. Noise: on high-gain or dark frames the pixels are dominated by random noise, which prediction cannot decorrelate; a 16-bit container over roughly 10 bits of noise entropy caps the ideal ratio at 16 / 10 = 1.6:1, and real coders land near 1.2-1.4:1. Throughput: table-driven Huffman encode runs in the tens to low hundreds of MB/s per CPU core, so a multi-thousand-MB/s aggregate is not a software job. The host side for wide links: a dual-port 10GigE frame grabber:

Which checks settle it before you order?

Settle three numbers before you order: the measured compressed rate from 50 real frames, the usable link budget of about 110 MB/s per GigE port and roughly 400 MB/s per USB3 port, and one spare CPU core per 50 MB/s of stream.

Run the list in order; the first two items take an afternoon.

1. Encode 50 representative frames from the real scene as PNG with the "sub" filter and the HUFFMAN_ONLY strategy. The measured byte ratio goes into every calculation below — scene texture moves it, so your frames are the number.
2. Recompute the raw rate at your pixel rate and pixel width: pixel rate x 2 bytes if padded, x 1.5 bytes if packed 12-bit.
3. Budget each port: about 110 MB/s usable per GigE, about 400 MB/s per USB3, about 1,150 MB/s per 10GigE.
4. Reserve CPU: one core per 50 MB/s coded; past a few hundred MB/s per stream, plan the FPGA line pipeline instead.
5. Apply the band rule: up to 1.5x over, code it; up to 2x, code it and trim frame rate or ROI; over 2x, widen the link.
6. Prove losslessness once: hash a raw frame, pass it through encode and decode, compare the hashes — they must match bit for bit.

If the port budget closes at two coded GigE streams, the host hardware is one part. A dual-port PoE NIC:

From here: pick the camera in the area scan cameras range, match the frame grabbers to your port count, and size the chain in Build a system or the camera finder. The surrounding pipeline pieces live in more systems guides. If the arithmetic will not close, send the pixel rate and link budget through the contact page and we will run the numbers with you.