Grayscale image display on composite video: fix the RGB565 mismatch

The camera runs, the code compiles, and the composite video output shows anything but the grayscale picture you asked for: stripes, magenta-green noise, or the frame cut in half. Test for a pixel-format mismatch before changing the camera or cable: the display routine walks the frame buffer in a different format than the capture wrote, and the bytes still reach the screen, just interpreted wrong. This guide is for engineers wiring an embedded camera module to a composite or wireless video link on a prototype. You will read the failure from its on-screen signature, confirm it with a byte count, fix the capture format or convert the frame once, and scale the picture to fill the display.

Why does the grayscale picture come out wrong on the composite video output?

The display routine walks the frame buffer in a fixed pixel format, and the capture wrote a different one: an RGB565 frame carries two bytes per pixel against grayscale's one, so a QVGA gray frame's 76,800 bytes get consumed as 38,400 two-byte colour pixels.

The video-out path is a translator with a fixed input. The library takes the image object, reads the format it declares, and converts it into the framebuffer the encoder scans out to the monitor. Each format needs its own conversion branch: RGB565 expands into red, green and blue lookups, while grayscale bytes map one-to-one onto gray levels. A faulty conversion path can consume the buffer with the wrong rule; a library that rejects unsupported formats should instead return an error.

That is why the fix lives at the capture, not at the monitor. The sensor writes the buffer once, at snapshot time, in whatever format the pixel-format call set, and everything downstream inherits that choice: a QVGA gray frame is 320 × 240 = 76,800 bytes at one byte per pixel, the same frame in RGB565 is 153,600 bytes at two. The mismatch appears when the buffer's real format and the format the display path consumes disagree — because the capture stayed in RGB565, or because the build's grayscale branch is missing and the routine falls through to the colour reader.

What does the picture look like when the formats do not match?

The byte-count ratio gives a useful test, but the visible geometry also depends on row stride, bounds checks and the display converter. The table describes a simple tightly packed buffer walked at the wrong bytes per pixel, not a unique diagnosis from appearance alone.

The false colour follows from the bits. Take a flat mid-gray frame, so every byte is 0x80. Read two adjacent bytes as one RGB565 value and they form 0x8080, which is 1000000010000000 in binary. Split on the 5-6-5 boundaries and the pixel becomes red 16 of 31, green 4 of 63, blue 0 ? predominantly red, not gray. The extraction is red = (0x8080 >> 11) & 31, green = (0x8080 >> 5) & 63, blue = 0x8080 & 31. A colour cast supports checking the format, but does not by itself prove which conversion failed.

The same class of failure shows up on streamed previews, not just composite out; the guide on a camera preview that shows a gray image walks the host-side version of it.

How do you set the camera to hand the display a grayscale frame?

Set the sensor's pixel format to grayscale before the first capture: the image object inherits the format at snapshot time, and the display call then runs the grayscale path instead of reinterpreting colour bytes that were never there.

For firmware exposing the legacy module-level tv API, after importing sensor and tv, the capture sequence is:

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)   # was sensor.RGB565
sensor.set_framesize(sensor.QVGA)        # 320 x 240
tv.init()
tv.display(sensor.snapshot())

This setting fixes the capture side when the display supports grayscale because the format travels with the image: snapshot stamps the buffer with the pixel format that was active, and the display call reads that stamp. Nothing later in the loop needs to know. If the same module also feeds a colour stage, keep the RGB565 capture and convert explicitly just before the display call instead, and assert the converted object's format before you show it — the step the guide on verifying pixel format after conversion spells out.

Two mistakes account for most repeat failures. A format set after the first snapshot leaves the old buffer in circulation, because pixel formats are fixed at capture; that timing is exactly what the guide on pixel formats that refuse to change untangles. And a display library whose grayscale branch is missing in your build will garble even a correctly declared gray frame — the cure is a firmware update through the maker's own channel, not a workaround in your loop.

How do you scale the frame to fill the display without distortion?

Scale with the display object's own dimensions — divide display resolution by frame resolution, so a 320 × 240 frame on a 640 × 480 screen needs a scale of 2.0 on both axes — and pass a bilinear hint when the ratio is not a whole number.

The display object knows its own geometry, so never type the numbers: read width and height from it and derive the scale. For QVGA on a 640 × 480 screen that is 640 ÷ 320 = 2.0 horizontally and 480 ÷ 240 = 2.0 vertically, an even scale that nearest-neighbour handles cleanly. These are dimensionless multipliers, not output pixel counts. With sensor, tv and image imported and tv initialized, the legacy video-out call is:

frame = sensor.snapshot()
tv.display(frame, x_scale=tv.width() / frame.width(),
           y_scale=tv.height() / frame.height(), hint=image.BILINEAR)

The 640 ? 480 example is arithmetic, not the TV shield resolution. The current TVDisplay interface uses 352 ? 240 output; a 320 ? 240 frame therefore needs factors 1.1 and 1.0 to fill that buffer. Current firmware uses display.TVDisplay.write rather than the legacy tv.display call above; use the API matching the installed firmware. Filling both axes may change the displayed aspect ratio.

Fractional scales are where bilinear earns its cost. Scaling 320 lines onto a 480-line screen is a ratio of 1.5, and nearest-neighbour at that ratio duplicates or drops lines unevenly, which reads as shimmer on moving parts. Bilinear averages neighbouring source pixels and keeps edges stable, at the price of four source reads per output pixel.

Composite and wireless analog links add one wrinkle: their pixels are not square, so a picture scaled with equal factors can look slightly stretched on the monitor. Judge the aspect on the actual screen and trim y_scale by a few percent if it matters to the operator; the scale factors are your dials.

If the loop runs short of time, crop before you scale. Cutting the capture to the region that matters removes bytes at the source, the move the guide on ROI cropping to reduce bandwidth at the camera quantifies.

Should you convert the frame or recapture it in grayscale?

Recapture in grayscale whenever the sensor offers it and no stage in the loop needs the colour bytes; convert only when one capture has to feed both a colour consumer and the monochrome display, because conversion reads the source pixels and writes a new representation.

The same decision looks different on a machine-vision head rather than a microcontroller module: there it is sensor choice. A mono sensor has no colour filter array, so the sensor supplies monochrome samples; downstream code must still respect their format, bit depth and stride.

Recapture wins when grayscale is the only consumer: no conversion pass, no second format to assert, and a buffer half the size of the RGB565 frame. Convert only when one capture genuinely serves two masters, a colour code-reading stage and the monochrome display say, and then convert once at the boundary rather than inside the display loop.

The cost numbers keep the choice honest. Converting QVGA is a pass over 76,800 bytes; at 60 frames per second that is 76,800 × 60 = 4.6 MB per second of extra memory traffic, which a host PC shrugs off and a microcontroller feels. Scaling adds more on top, which is why capturing at display resolution beats capturing large and shrinking on small boards.

How do you verify the fix before you trust it?

Three checks settle it in minutes: the image object's reported format, a buffer length of exactly 76,800 bytes for a QVGA grayscale frame, and a flat mid-gray test frame that must show gray on the monitor with no colour tint.

1. Read the stamp. Print the image object's pixel-format property after the first snapshot and assert it equals the format the display path expects. This one line catches the late-set-format failure immediately.
2. Count the bytes. A QVGA grayscale buffer holds 320 × 240 × 1 = 76,800 bytes. For these two tightly packed formats, 153,600 bytes indicates RGB565; also inspect row padding and the actual declared bit depth before diagnosing other buffers.
3. Show flat gray. Fill a frame with 0x80 and display it. Clean gray on the monitor means the reader matches the buffer; a colour cast means the conversion path still needs investigation; the 0x8080 decode above predicts a reddish value for that specific reinterpretation.
4. Sweep a ramp. A left-to-right ramp from 0 to 255 must appear as a smooth ramp. Banding that repeats every two pixels is the byte-pair signature again.
5. Update through the vendor. If all four checks pass and the picture is still wrong, the build's grayscale display branch is the suspect. Update the firmware through the maker's own distribution channel rather than patching bytes in your loop.

How do you keep the mismatch from coming back?

Make the format explicit at every boundary: declare the capture format in exactly one place, take display dimensions from the display object rather than typed constants, and treat any code that indexes the buffer by length instead of width and height as a defect.

• One declaration. The pixel-format constant appears once, at init, in shared configuration; no call site types it a second time.
• Dimensions from the source. The scale factors come from the display object's own width and height, so a screen swap never leaves a stale constant behind.
• Buffer math in one place. Anything computing an offset or length derives it from declared width, height and bytes per pixel, never from a hardcoded 76,800.
• Asserted at every consumer. The display call, the logger and any conversion step each assert the format they require before touching the buffer.

Draw one boundary around all of this: every failure above happens in the digital framebuffer, before the composite encoder runs. If the checks pass and the monitor still shows rolling, torn or washed-out video, the fault is analog — signal level, termination or cable loss — and no pixel-format change will touch it. Driving composite video over longer runs is its own discipline; the write-up on receiving composite video over Cat5 with a variable-gain receiver covers that analog side. The format and pipeline fixes that live upstream are collected in the Learn guides hub.

Which camera and cabling get the grayscale path right from the start?

Choose a mono board-level camera for a pipeline whose only consumer is grayscale, run it on a data cable matched to its interface and length, and keep every format decision in software where a one-line assert can catch it.

When the prototype proves out and the build turns into a machine, the parts decide half the format questions for you. Mono board-level cameras remove the colour path entirely: the board-level cameras category filters by sensor, mount and interface, and the camera finder narrows it to a shortlist in a few clicks. The guide on board-level camera selection for embedded vision machines covers the mount, cable and bandwidth decisions that come next.

If you are wiring the capture-and-display module yourself, these two board-level heads cover a bare-board build and a housed miniature build.

Run the head on a data cable matched to its connector and run length; the camera data cables page is organised by exactly those two filters. To spec the whole chain in one pass, start the configurator at the camera step, and free expert support with every order means an applications engineer will check the format chain with you.