Bayer decode with a lookup table: fast LUT demosaic

Your acquisition library hands you a single-channel frame of red-green mosaic and offers no Bayer decoder. You can still get clean color from that frame with two lookup tables. A color sensor fixes each pixel's color by its grid position, so a four-entry position table plus three 256-entry value tables decode the frame with plain indexed reads: nearest-neighbor demosaic, no interpolation, no image-processing library. This how-to is for anyone integrating a color area scan camera over GigE, USB3 or a frame grabber. You will read the pattern from the pixel-format name, build and test the decoder in C, fold white balance into the tables, and compute what the decode costs at your frame rate.

Why position alone decides the color

A Bayer sensor puts a color filter over every pixel in a repeating 2 × 2 tile. In the most common layout, red sits at the top left, green at the two diagonal neighbors, blue at the bottom right. The tile repeats from the frame origin, so the color of pixel (x, y) depends only on x mod 2 and y mod 2 — four cases, not four million.

Demosaicing means filling all three channels at every pixel from these partial samples; the Demosaicing overview on Wikipedia sketches the algorithm families. Nearest neighbor is the zero-arithmetic end of that range: each pixel copies the value of the nearest same-color sample. On a Bayer grid that sample always sits inside the pixel's own 2 × 2 tile, so the whole decode is two table lookups per channel — one on position, one on value — and byte copies.

What you need

• A color camera streaming raw 8-bit Bayer: a pixel format named BayerRG8, BayerGR8, BayerGB8 or BayerBG8 in its format list. The 10- and 12-bit names share the same tile and change only the value width.
• Access to the raw frame buffer before any conversion, plus an output buffer of 3 × w × h bytes for interleaved RGB8.
• The pattern name from the camera. If it is missing or doubtful, a solid red and a solid blue target — colored paper, a bottle cap — settle it in two captures.
• 768 bytes of value tables (3 channels × 256 entries) for the folded white balance of the next sections.

If the viewer stays gray while a color format is selected, the pixel-format default is the usual culprit: GigE camera shows no color image: fix the pixel format default walks that fault down. If the camera refuses to enter a Bayer format at all, other settings often lock the list — Camera pixel format not changing: why it is fixed and how to set it explains why and how.

The decode runs on your host after acquisition, so any color camera that exposes a BayerXX8 format works. Two current picks across GigE:

Read the pattern from the pixel format name

The letters after Bayer list the colors of the first two pixels on the first line; the name follows the GenICam pixel-format naming convention. BayerRG8 starts red, green on line one and continues green, blue on line two. The other three names rotate the same tile. In the table, k 0 is pixel (0, 0), k 1 is (1, 0), k 2 is (0, 1), k 3 is (1, 1), in the order of the formula above.

Trust the name, then confirm it once against pixels; the red-target test below takes two minutes and catches cameras whose name and site map disagree. For 10- and 12-bit transports the tile is unchanged — only the values are wider or packed, and Packed Bayer 12-bit conversion to a 16-bit image covers unpacking those into a plain 16-bit buffer first.

Build the decode in four steps

1. Write the pattern table. Store P as channel indices in k order: RGGB is {0, 1, 1, 2} with 0 = red, 1 = green, 2 = blue. The table above gives the four entries for every format name.

2. Derive the offset tables. For each channel and each k, record the offset (dx, dy) from the current pixel to the nearest site of that channel. Inside one 2 × 2 RGGB tile, red pulls from its top-left site, green from the green at (1, 0), blue from the bottom-right site; break every tie toward that same tile partner, never across tiles. Twelve offsets result, tabulated in the code below.

3. Decode. Per pixel, compute k, then for each channel read src[(y + dy) × w + (x + dx)] and write it to that channel's byte. Clamp the offsets at the frame border. With the value tables of the next section, the write becomes tab[c][v] and the gains come along for free.

4. Test on synthetic frames. Assert before touching camera data; the test after the code proves the mapping.

#include <stdint.h>

/* Illustrative decoder; names are ours, not any vendor's API.
   src: raw 8-bit Bayer frame, w * h bytes.
   dst: interleaved RGB8, 3 * w * h bytes, preallocated.
   w and h must both be even; offsets clamp at the borders. */

/* Offsets (dx, dy) per channel, indexed by k = (y & 1) * 2 + (x & 1).
   Channels: 0 = red, 1 = green, 2 = blue. Tile layout: RGGB. */
static const int8_t DX[3][4] = {
    { 0, -1,  0, -1 },   /* red:   own site, left, up, diagonal    */
    { 1,  0,  0,  0 },   /* green: right at red sites, else own    */
    { 1,  0,  1,  0 } }; /* blue:  diagonal, down, right, own site */
static const int8_t DY[3][4] = {
    { 0,  0, -1, -1 },
    { 0,  0,  0, -1 },
    { 0,  1,  0,  0 } };

void bayer_rg8_to_rgb8(const uint8_t *src, uint8_t *dst, int w, int h)
{
    for (int y = 0; y < h; ++y)
    for (int x = 0; x < w; ++x)
    {
        int k = ((y & 1) << 1) | (x & 1);
        uint8_t *p = dst + 3 * ((size_t)y * w + x);
        for (int c = 0; c < 3; ++c)
        {
            int sx = x + DX[c][k]; if (sx < 0) sx = 0; if (sx >= w) sx = w - 1;
            int sy = y + DY[c][k]; if (sy < 0) sy = 0; if (sy >= h) sy = h - 1;
            p[c] = src[(size_t)sy * w + sx];
        }
    }
}

The test. Feed a 2 × 2 source {200, 100, 100, 50} — red 200 at (0, 0), green 100 beside and below it, blue 50 at (1, 1) — with unit gains. All four output pixels must come out (200, 100, 50), so assert all twelve output bytes. Then decode a uniform 4000 × 3000 gray frame of value 120: every output pixel must be (120, 120, 120), which exercises the border clamps on a realistic size in one pass.

For planar output — three w × h planes instead of interleaved RGB — keep the same tables and change only the destination address. Some downstream libraries prefer planes; the per-pixel cost is identical.

Fold white balance into the value tables

Red and blue rarely reach 255 on a neutral scene, so scale each channel by its white-balance gain before the first production frame. The channel at each pixel is fixed, so the gain hides in a value table: build once, apply per pixel at no cost.

With gains of 1.30 red, 1.00 green and 1.65 blue, a red sample of 100 becomes 130, and red samples above 196 clamp at 255 (196 × 1.30 ≈ 255). Three tables of 256 bytes each make 768 bytes in total — cache-resident, rebuilt per scene in microseconds, and no second 36 MB pass over the decoded image.

Prove the pattern before the first production run

Capture the raw frame under a solid red target. Only the red sites brighten, and their phase names the red k: bright at even-even is RGGB, at odd-even GRBG, at even-odd GBRG, at odd-odd BGGR. Repeat under blue and the pattern is pinned; green takes the two remaining positions. Compare with the camera's format name, and believe the pixels where the two disagree.

Debug packed Bayer acquisition before image processing covers the acquisition-side faults — broken lines, swapped bytes, wrong stride — that corrupt the raw frame before your decoder ever sees it.

What nearest-neighbor costs you

Count the samples. In every 2 × 2 tile, red and blue get one pixel each and green gets two. Replication writes each red value into a 2 × 2 block, so the red plane resolves detail only down to twice the pixel pitch in each axis; blue is the same, green sits at half that pitch. Color detail at full pixel pitch is copied, not measured.

Two artifacts follow. Slanted near-gray edges can show zipper, a one-pixel sawtooth, because adjacent columns alternate red and blue samples. And the tie rule shifts channel registration by a constant sub-pixel amount: red always samples up-left of true position, blue down-right. Geometry measured across channels inherits that constant; single-channel geometry does not.

For presence checks, geometry, defect contrast and most code reading, the cost is invisible. For color matching, print inspection or any measurement where a color edge carries the dimension, run bilinear or a gradient-corrected demosaic from a maintained library instead; the LUT decode is the fast path, not the reference.

The speed arithmetic: a worked example

The LS-CMR-0166 delivers 4000 × 3000 frames at 30 fps over USB3 — a realistic full load for one decode thread.

• Pixels per frame: 4000 × 3000 = 12,000,000.
• Raw input: 12,000,000 × 1 byte = 12 MB per frame, or 12 × 30 = 360 MB/s at 30 fps.
• RGB8 output: 12,000,000 × 3 = 36 MB per frame, or 36 × 30 = 1,080 MB/s.
• Total stream touched: 360 + 1,080 = 1,440 MB/s, about 1.4 GB/s.

Assume the decode thread sustains 2 GB/s of sequential stream traffic with tables cache-resident — a typical desktop figure; measure it once on your host. Per frame: (12 + 36) MB ÷ 2 GB/s = 0.024 s, so 24 ms. The frame period at 30 fps is 1,000 ÷ 30 = 33.3 ms, and 24 ÷ 33.3 ≈ 0.72, so the decode occupies 72% of one core. Capacity is 1,000 ÷ 24 ≈ 41.7 fps against the 30 fps needed.

Decision: one core carries this camera with headroom. Folding the gains into the value tables is what keeps it that way — a separate balance pass would add another 36 MB per frame of traffic and push a four-camera host toward its memory-bandwidth ceiling.

When to reach for a different decode

• Color is the measurement — color matching, print, food and ripeness scoring. Use bilinear or a gradient-corrected demosaic from a maintained library; 2 × 2 replication fails those.
• 12-bit scenes with real dynamic range. Decode with 4,096-entry value tables (sizes in the FAQ) or unpack to 16-bit first.
• The task is grayscale. Skip decode entirely: read the Bayer frame as mono. The alternating filters act as luminance, the sharpness loss at pixel pitch is modest, and you save the whole 3-bytes-per-pixel output stream.

When it is the camera side that needs choosing, the camera finder filters color models by interface, resolution and shutter, and the camera step of the configurator pairs sensor, lens and light in one pass. The cameras topic page collects the other pixel-format and decode guides.

Before you ship: the checklist

• Pattern proved by pixels, not by name — one red capture, one blue capture, phase matching the table. Why: a swapped pattern passes every visual check and corrupts every color downstream.
• Synthetic tests green — the 2 × 2 frame decodes to (200, 100, 50) everywhere; a uniform gray frame stays uniform. Why: index and border-clamp bugs surface in minutes, not in production.
• Gains folded into the value tables and centered on a neutral target under production lighting. Why: the decode stays single-pass.
• Decode time measured over 100 frames at production resolution, not assumed from the example. Why: the 2 GB/s figure is desktop-typical, and your host decides.
• Exit path kept — if the SDK later exposes its own Bayer decode, both paths hide behind one function call. Why: a vendor's optimized decode will beat byte copies when it exists.

If the arithmetic says your host is tight, request a quote with your pixel format and frame rate and we will size the decode host with you.