Embedded camera snapshot lockups: bypass the JPEG stall

An embedded camera that freezes mid-shift, with the debugger parked inside a JPEG compression routine and the last saved frame torn, is not a dying sensor. The snapshot pipeline stalled waiting for output buffer space: the snapshot call encodes JPEG, the encoder is paused because nobody is draining the debug stream, and the firmware then waits without a timeout. This guide is for engineers running snapshot-loop scripts on MCU-class vision boards — the sensor.snapshot(), GRAYSCALE-or-RGB565, find_edges kind of program. You will read the stall signature, confirm the cause with a 12-hour soak, size the buffers with real arithmetic, apply the raw-framebuffer fix, and decide when the job belongs on a smart camera or vision controller instead.

Why does a snapshot lock up inside the hardware JPEG compressor?

The snapshot call stops because the compressor's output buffer is full and nobody is draining it: the firmware parks the CPU in a wait-for-interrupt loop that checks a JPEG_output_paused flag, and with no timeout the call never returns.

The per-frame path is longer than the script suggests. sensor.snapshot() returns your frame and then refreshes the JPEG side of the framebuffer through framebuffer_update_jpeg_buffer(), which calls jpeg_compress(). That routine drives the hardware encoder, and whenever the output side is paused it sleeps in __WFI() until an interrupt says otherwise. The output side pauses precisely when its consumer — the WiFi debug stream or preview client — is not reading.

The encoder is not slow; it is orphaned. Your loop stops at the snapshot that precedes find_edges, because that call now includes an encode with an unbounded wait. The input-side flag, JPEG_input_paused, parks the same way when the sensor side stalls, which is why the debugger shows the CPU stepping between the two checks forever. Exposure is not part of the wait: exposure_us=2500 caps the sensor at 1 / 0.0025 s = 400 fps, far above anything the loop asks for.

What does the stall look like in the debugger and in the logs?

The signature is a stack parked in jpeg_compress, a frame counter frozen at one value for minutes, and a console that stops printing fps while ping, power and the link LED all stay healthy.

In the debugger, the backtrace ends inside jpeg_compress(), reached from framebuffer_update_jpeg_buffer() under the snapshot call, and single-stepping cycles the same three lines: the input-paused check, __WFI(), then the output-paused check. In your prints, clock.fps() simply stops. On the network, the board still answers ping with the link LED lit, so nobody hunting a network fault finds anything.

The image tells the third half of the story: the last buffer is torn, part new frame and part old. That is the corrupted framebuffer you end up posting with the bug report.

Why does the last framebuffer come out corrupted?

The last image tears because the pause breaks buffer ownership: the compressor still holds a frame while the sensor driver moves on, so a reset or a new snapshot hands out a buffer that is only half written.

A framebuffer is the RAM region that holds the current image, and on these boards one region serves both the raw frame and the JPEG output. While the encoder is paused, it still owns that memory. Re-running your script without restarting the runtime leaves the old stream session attached, so the next snapshot writes into a chain the encoder has not released, and the frame after that returns the half-written result as ordinary image data.

The corruption is a symptom of the pause, not a separate fault. One owner at a time, drained before reuse, is the same discipline as reusing image buffers instead of per-frame allocation, and feeding raw bytes to a JPEG decoder hangs the mirror-image way, which is why that path must encode before decoding. A torn last frame that survives a clean restart has its own wider checklist in the board-level image corruption guide.

How do you confirm the compressor is the cause?

Swap the pixel format to raw — GRAYSCALE or RGB565, so the snapshot call never touches the encoder — rerun the identical find_edges program, and if a 12-hour soak shows zero lockups while the JPEG build stalls within hours, the compressor path is confirmed.

1. Count the baseline: run the current JPEG build for a full 12-hour shift and note the lockups per 12 hours, for example two or three.
2. Change one thing only. Set the pixel format to sensor.GRAYSCALE or sensor.RGB565 so snapshot() no longer encodes, and keep framesize QQVGA and exposure_us=2500 identical.
3. Print a heartbeat: a frame counter every 1,000 snapshots, so a stall shows up as a missing print rather than a silent hang.
4. Soak both builds for 12 hours. A raw build that never parks next to a JPEG build that stalls within the first hours settles it.
5. If the raw build stalls too, the compressor is innocent and you are looking at a broader acquisition stall with its own isolation procedure.

The reason the fault looked random is the loop rate. find_edges with the Canny threshold at (50, 80) spends tens of milliseconds per cycle, and each cycle leaves a JPEG frame waiting in the output buffer for that whole time. With manual exposure there is no auto pacing to regulate the wait, so the probability of a park tracks the debug link, not the optics — which is why the lockups cluster after updates and during development sessions.

What is the fix: raw framebuffer first or JPEG on demand?

Take the compressor out of the per-snapshot path: capture raw GRAYSCALE or RGB565 every cycle, and encode JPEG only at the moment you actually serve an image, or restart the debug stream cleanly whenever you enter or leave it.

Do both, in that order. First, the loop:

sensor.reset()
sensor.set_pixformat(sensor.GRAYSCALE)  # raw path: the snapshot call never touches the encoder
sensor.set_framesize(sensor.QQVGA)      # 160 x 120
sensor.set_auto_exposure(False, exposure_us=2500)

img = sensor.snapshot()                 # returns raw bytes; it cannot park on a JPEG pause
img.find_edges(image.EDGE_CANNY, threshold=(50, 80))
# encode JPEG here, and only when a client is connected and reading

The calls are the loop's own; the comments are ours. Nothing in the algorithm changes — Canny still runs on the raw frame, as it always did.

Second, encode at serve time. When a dashboard or the debug stream wants an image, encode that frame then, and drop frames when the client falls behind. An encode that nobody consumes is the failure you just debugged, so make it conditional on a connected, reading client. Pipelines that snapshot and serve only what a consumer asked for are spelled out in the embedded JPEG snapshot guide, and the raw-sensor discipline behind GRAYSCALE is the same one that governs cameras with no ISP.

Third, close the state hole that started the incident: soft reset the runtime whenever the WiFi debug stream is attached or detached, and after re-running your program. A soft reset restarts the stream with known buffer ownership, so no orphaned session is left holding the output side paused. That clean restart is what cleared the confirmed lockups.

Guard the wait while you are there. Wrap snapshot() with a time budget — 100 ms is a workable bound for a QQVGA loop — log the pause flags when it trips, and restart the stream instead of hanging.

How much headroom does the raw path buy?

A QQVGA grayscale frame is 160 × 120 = 19,200 pixels at one byte each, so a raw loop moves 19,200 bytes per frame — 576,000 bytes per second at 30 fps — before any compression is involved.

Work the failing loop's numbers end to end. QQVGA is 160 × 120 = 19,200 pixels. GRAYSCALE stores 1 byte per pixel, so one raw frame is 19,200 bytes, about 18.75 KiB; RGB565 stores 2 bytes per pixel, 38,400 bytes, about 37.5 KiB. JPEG at quality 50 on grayscale frames compresses roughly 10:1 as a typical figure, so one encoded frame is 19,200 / 10 = 1,920 bytes. Assume the find_edges cycle runs at 30 fps, a 33 ms cycle, and the output buffer is 32 KiB = 32,768 bytes — tens of KiB is the usual scale on this class of board.

Budget the drain. With the client silent, the buffer holds 32,768 / 1,920 = 17 frames, and at 30 fps it fills in 17 / 30 ≈ 0.57 s. With a weak drain of 2,000 bytes per second, the net fill rate is 30 × 1,920 − 2,000 = 55,600 bytes per second, so the park starts after 32,768 / 55,600 ≈ 0.59 s. A lockup that seems to arrive randomly is following the debug link at exactly this timescale.

Units: B_buf and B_jpeg in bytes, R_drain in bytes per second, t_fill in seconds. The example evaluates to 32,768 / (30 × 1,920 − 2,000) ≈ 0.59 s.

The decision this buys: run the inspection loop at 30 fps raw regardless of the link, and cap the JPEG preview at the drain rate — 2,000 / 1,920 ≈ 1 frame per second — so the encoder never outruns its reader. If a client cannot keep its rate, drop frames; never let it pause the pipeline. Grayscale also halves per-frame memory against RGB565, 19,200 against 38,400 bytes, which often settles the pixel format on an MCU-class board.

When should you move the job to a smart camera or vision controller instead?

Move the job when the board must both run your algorithm and feed a live image: an on-device smart camera returns results, not frames, and a vision controller takes the raw stream on a fanless machine with dedicated camera ports.

Two signals say move. The first is duty: a development board that inspects during the day and streams previews to whoever asks is doing two production jobs, and the JPEG stall is the collision between them. The second is recovery: when the camera must run 24/7 and come back from a dropped link by itself, that behavior is firmware work on a dev-board class camera and a feature list on an industrial one, as the days-running network recovery guide shows. If the job stays on the board but gains a dashboard, take a transport path and its bandwidth math from the smart-camera data-logging guide rather than inventing a stream.

An on-device vision sensor keeps the whole loop on the camera and streams decisions, not frames.

The MV-SC3050XC-16M-ATN-H is a 5 MP color vision sensor with a global shutter, a 1/1.45" sensor format, an M12 lens mount and a Fast Ethernet interface. Browse the smart cameras category or filter the camera finder by resolution and interface.

Multi-camera cells that want the raw pipeline on a controller belong in the vision controllers category.

The MV-VC2140-128G40-1T vision controller provides 3-4 LAN ports, 2 USB 3 ports, four built-in light-output channels and 8 GB of memory for exactly that shape of cell.

How do you keep the lockup from coming back?

Prevention is three habits — raw pixel format by default, a soft reset of the runtime whenever the WiFi debug stream is toggled, and a watchdog that counts snapshots so a stalled pipeline trips in seconds, not days.

Then extend the list as the installation grows:

• Raw pixel format by default: GRAYSCALE or RGB565 in the snapshot loop, JPEG only at serve time.
• Soft reset discipline: restart the runtime whenever the WiFi debug stream connects, disconnects, or after re-running the program — never carry a stream session across a code reload.
• A frame heartbeat: print the snapshot counter every 1,000 frames and alarm when snapshot time exceeds budget, so a park trips in seconds.
• A drain budget: keep the reader above the encode rate with margin; in the worked example, a 2,000 bytes per second link supports about 1 preview frame per second, not 30.
• A 12-hour soak after every firmware or client update, comparing lockup counts before and after.
• Evidence before reset: when the torn last frame appears, dump the raw framebuffer before restarting — it is the only witness of the pause.

Run the drain arithmetic first; it decides whether you fix the board or move the job. If the numbers say move, spec the controller side with the system configurator. If a stall survives a clean raw-format soak, bring the debugger trace with you through the contact page and we will read the pause flags with you.