Python threading blocks the camera loop: use multiprocessing

Two Python loops in one script: one grabs frames from the camera, the other drives a plot or GUI window. Each loop runs fine on its own. Launch both with threading and one of them freezes; stop or crash the busy loop and the frozen one wakes up again. That pattern points at scheduling inside the Python process, not at the camera, the driver or the cable. The usual cause is the global interpreter lock (GIL): one process, one lock, and a loop that never puts it down. In the sections below you will confirm the cause in a minute, free the loops with releases you already have, and move acquisition into its own process with multiprocessing once the arithmetic shows threads cannot fit.

Symptom: stop one loop and the other wakes

Each loop on its own behaves. Both under threading, the display loop stalls while the plot keeps updating. Kill the plot loop and the camera feed returns on the very next frame. That revival is the diagnosis: the camera hardware was never stuck, and a thread inside your own process was waiting for a lock.

A camera loop also carries a hard deadline. At a 30 fps stream a frame arrives every 1000 ms / 30 fps = 33.3 ms; miss two or three in a row and windows freeze, buffers back up and the driver starts dropping frames. A plot loop is softer: with a 50 ms sleep between updates it owes 20 updates per second, and slipping to 10 costs nothing visible. Under one lock, the soft loop wins precisely because its missed deadlines are invisible.

Cause 1: the GIL makes interpreter time a single serial resource

CPython runs one thread's Python bytecode at a time inside a process; the lock that enforces this is the GIL. Two OS threads in one interpreter therefore take turns no matter how many cores the board has. When a second thread wants the lock, the interpreter forces a handoff within the default switch interval (sys.setswitchinterval, 5 ms). A handoff only helps when the running thread lets go, and a thread inside a long pure-Python stretch comes straight back for the next slice.

Three things do put the lock down: a blocking call such as time.sleep or a socket read, most heavy OpenCV and NumPy operations while their native code runs, and a wait on a queue or event. Loops that end each cycle in one of those share an interpreter cleanly. Loops that end in more Python starve their neighbors, and the starvation looks exactly like a hung camera.

Cause 2: one loop never releases the lock

The failing pattern is a cycle made of Python-level work with no blocking call at the end: building a list per event, updating a scatter plot point by point, redrawing, or processing pixels in a Python for loop. Each cycle is short, but at hundreds of cycles per second the loop holds the lock for most of every second, and the grab thread waits.

Quick test: add time.sleep(0.001) once per cycle inside the starved loop. If it recovers within seconds, lock contention is confirmed and nothing else has to change yet. Then measure: wrap each loop's body with start = time.perf_counter() at the top and elapsed = time.perf_counter() - start at the bottom, and print the average. Python-side work above half the cycle period predicts starvation, and the same numbers feed the budget in the worked example below.

Per-pixel processing in Python burns interpreter time regardless of locks; the per-pixel loop fix removes that load, and replacing busy waiting with a blocking read removes its twin on the acquisition side.

Cause 3: the GUI loop lives in the wrong thread

GUI toolkits require their event loop in the process's main thread. Build the QApplication inside a worker thread, as the failing pattern does, and window events stall or the process dies outright. On a fast office PC the timing margins can still hold; on a thermally capped embedded module they do not, which is why identical code ran on the desk and failed on the controller. More cores and faster clocks hide contention that a capped board exposes.

The structural fix costs nothing: keep the GUI loop in the main thread and move acquisition out of it, into a thread when the grab cycle ends in a blocking call, into a process when it does not. The same discipline shows up when a capture stalls the whole window: the fetch call belongs on a worker, never on the event loop. When a loop's behavior changes between runs because settings linger from the previous one, the known-state reset pattern is the companion check.

Worked example: a 30 fps camera loop and a 20 Hz plot loop, one interpreter

Inputs, with their sources:

• Camera stream: 30 fps, the data-sheet rate of a 12 MP USB3 camera such as the LS-CMR-0166 (4000 × 3000 resolution). Frame deadline: 1000 ms / 30 fps = 33.3 ms per frame.
• Plot cycle: time.sleep(0.05) between updates, from the failing two-loop pattern: one update attempt per 50 ms, so at most 1000 / 50 = 20 updates per second.
• GIL switch interval: 5 ms, CPython's default, the longest one runnable thread can delay another before a forced handoff.
• Work per cycle: 8 ms of Python-side work per frame in the grab loop and 40 ms per plot cycle inside the plotting and Qt calls. These two are illustrative; measure yours with the perf_counter timing shown in the previous section.

Rule in words: inside one process the GIL turns all pure-Python work into one serial stream. The process can execute at most 1000 ms of interpreter work per second, and each loop must fit its own work plus a worst-case 5 ms handoff inside its period.

Arithmetic:

• Grab loop load: 30 frames/s × 8 ms = 240 ms of interpreter work per second.
• Plot loop rate: 1000 ms / 50 ms = 20 cycles/s, so 20 × 40 ms = 800 ms per second.
• Combined: 240 + 800 = 1040 ms demanded per 1000 ms available. Over budget before any OS overhead.
• Per-cycle check for the camera: 8 ms work + 5 ms worst-case wait = 13 ms < 33.3 ms. The frame deadline fits; the per-second budget is what breaks.

Result and decision: threads fail once per-second interpreter demand crosses 1000 ms, so plan the split at half of that, 500 ms, and keep margin for handoffs and OS noise. At 1040 ms the plot loop moves to its own process. Each process then owns a full 1000 ms budget: the grab process idles at 24% load (240 of 1000 ms) and the plot process runs at 80% (800 of 1000 ms), side by side on separate cores of the same board.

Fix: one process per real-time loop

1. Keep the GUI loop in the main thread of the starting process.
2. Move the grab loop into its own process with multiprocessing.Process, and pass frames out through a multiprocessing.Queue with maxsize=1; when the queue is full, drop the stale frame and put the newest one. Keep the __main__ guard: on spawn-style platforms the child re-imports the file.
3. Prefer sending results over raw frames. A 4000 × 3000 8-bit mono frame is about 12 MB, and every queue put copies it; a detection list or a downsampled preview is kilobytes.
4. If threads must stay, end every cycle in a call that releases the GIL: time.sleep(0.001) or longer, a blocking read, or the heavy OpenCV call itself, and keep the measured Python work per cycle under half the period.
5. Prove the result: print the delivered rate for a full minute at startup and again after the GUI is loaded and interacting.

import cv2
import multiprocessing as mp
import time

def grab_loop(frame_queue: mp.Queue) -> None:
    cap = cv2.VideoCapture(0)        # open once, keep it open
    while True:
        ok, frame = cap.read()       # blocks inside the driver
        if not ok:
            time.sleep(0.001)        # never spin with the lock held
            continue
        if frame_queue.full():       # keep only the newest frame
            frame_queue.get_nowait()
        frame_queue.put(frame)

if __name__ == "__main__":
    frame_queue: mp.Queue = mp.Queue(maxsize=1)
    grabber = mp.Process(target=grab_loop, args=(frame_queue,), daemon=True)
    grabber.start()
    frames: int = 0
    t0: float = time.perf_counter()
    while frames < 300:              # 300 frames at 30 fps should take 10 s
        frame = frame_queue.get()    # waits; the newest frame wins
        frames += 1
    elapsed: float = time.perf_counter() - t0
    print(f"delivered {frames / elapsed:.1f} fps over {elapsed:.1f} s")

A healthy split prints a rate within a few percent of the camera's rate, near 30.0 fps for the example stream, held for the whole minute. A steady rate at half that points at the camera or link mode, not the scheduler.

Open the camera once inside the grab process and leave it open; reopening per frame pays set-up cost on every grab, and the same measured penalty shows up whenever a test opens and closes the capture in a loop. The same skeleton with a hardware trigger in place of the continuous read is in the hardware trigger grab loop.

Prevention: budget the interpreter like any other resource

• Keep the per-second sum of pure-Python work per process under 500 ms, half of the GIL's 1000 ms, so every loop keeps deadline margin.
• End every cycle in a release: a real sleep of 1 ms or more, a blocking read, or the heavy OpenCV or NumPy call itself.
• Keep bulk pixel work inside OpenCV and NumPy calls; a Python-level per-pixel loop burns the same budget at any camera speed.
• Send results, not raw frames, across process boundaries, and keep queues at depth one. Frame-sized messages in queues nobody drains grow until memory runs out, the failure described in out of memory in a video loop.
• Move a loop that stays CPU-bound in Python into compiled code, a native extension or a small C++ worker, or into its own process.
• Watch the gap between camera rate and processing rate; acquisition faster than processing shows how queue discipline keeps latency flat when the consumer falls behind.

When the board is the constraint, not the code

If the measured budget already sits under 500 ms per process and deadlines still slip, the board itself is short of CPU. Thermally capped embedded modules run out of headroom exactly when a second heavy process arrives. The fix then is hardware rather than cleverness: run the grab process and the GUI on a fanless vision controller with enough cores that each real-time loop owns one.

A vision controller also puts the camera ports, the light outputs and the timing in one box, so trigger and grab share one clock domain. The MV-VC2140-128G40-1T runs an Intel N-series N97 processor with 8 GB of RAM and a 128 GB SSD, and its card lists 3-4 LAN ports, 2 USB 3 ports and 4 built-in light channels: one port for the camera, one for the plot client, and the light channels on the same controller.

On the capture side, USB3 models in the area scan cameras range suit this two-process pattern, and the camera finder filters by interface, resolution and shutter. A 12 MP USB3 camera such as the LS-CMR-0166 delivers 30 fps at 4000 × 3000, the stream the worked example budgets.

Deploy checks before the line

Run these before the system goes on the line; each one catches a real fault:

• Delivered rate within 5% of the camera's rate for a full minute: 28.5 fps or better against a 30 fps stream. A lower steady rate is a camera or link mode problem, not a scheduler problem.
• top shows the grab process under one core of CPU with at least one core idle. If the board sits above 70% used, a heavier GUI will push a loop over budget later.
• Stop the plot process for ten seconds. The camera rate must not move; restart it, and the rate must still not move. That independence is what the split exists to give.
• Display latency under one frame period: at 30 fps the newest frame on screen is less than 33 ms old.

When the checks pass, the design is done. If the controller itself is the bottleneck, compare fanless industrial mini PCs sized for 24/7 duty, size the rest of the chain in the configurator, read the surrounding patterns on the systems topic page, or bring the timing question to us through the contact page.