Event camera frame generation crash: fix thread safety in the callback

Your event-camera pipeline runs for hours on a quiet desk, then dies the moment the scene gets busy: a segmentation fault inside the frame-accumulation code, or an abort reading double free or corruption (out). If you are building a calibration tool, a recorder or a viewer on an SDK whose on-demand frame generation crashes at high event rates, the cause is rarely memory you forgot to release. It is almost always two threads inside one buffer. Work through the diagnosis here: confirm it with the arithmetic of event rate and read time, restructure the pipeline so a single thread owns the buffer, and prove the fix with sanitizer runs and a checkerboard stress scene.

The symptom: a crash that tracks scene activity

Two failure modes, one defect. The first is a segmentation fault whose stack ends inside the frame-generation class — a method of OnDemandFrameGenerationAlgorithm, not a line you wrote — captured with a tracer such as backward-cpp. The second is an abort from the C allocator that prints double free or corruption (out). The difference is timing. When the reading thread dereferences memory the allocator has already released or moved, you get the segfault. When a concurrent write lands on allocator bookkeeping instead, the damage surfaces later, at the next release, as the abort.

The signature that identifies the race: crash probability scales with event rate. A still bench scene runs all day. A flashing checkerboard moved close to the lens — the fastest way to drive tens of millions of events per second — kills the process in seconds. The failure also reproduces in a minimal example, which matters: it takes the rest of your pipeline out of the suspect list and isolates the frame-generation path.

Likely causes, in order of probability

1. Two threads inside one frame generator. Event SDKs deliver packets on an internal thread, and the generator object — the class behind the OnDemandFrameGeneration mode — is typically reachable from that path as well as from your render or timer thread. Two threads append, iterate and reallocate one buffer without a lock: a textbook data race. Test: print the thread id (std::this_thread::get_id()) at every entry point you control and inside the events callback. Two distinct ids touching one accumulator is the diagnosis.
2. Event data kept beyond the callback. The delivered packet buffer belongs to the SDK, which reuses or releases it when the callback returns. Store a pointer into it and read it later, and you hold a use-after-free even with perfect frame timing. Test: an AddressSanitizer build names the exact read on the first run.
3. An unbounded queue behind the callback. At tens of millions of events per second, a queue that nothing drains grows until the allocator gives up, and that abort mimics the crash above. Test: watch process memory across a five-minute high-rate run. Growth that never plateaus is the tell, and the fix is the bounded queue described in our Python frame queue memory leak guide.
4. A defect in the SDK build. Credible only after the single-owner refactor below still aborts with a clean ThreadSanitizer report. Mixed C runtimes or a compiler mismatch then join the suspect list — see the last section.

Decision rule: a crash that tracks scene activity points to causes 1 or 2; unbounded memory growth points to cause 3; a crash that survives the refactor with clean sanitizers points to cause 4.

Hardware gets one minute, not one hour. Link saturation and cable faults drop packets, and dropped events show up as torn or shifted images, never as heap corruption inside your process. The guide on isolating shared-hardware image corruption and the one on what happens when events exceed the link cover those modes. If the image is intact until the abort, skip them and fix the threading.

Why low rates hide it: the numbers

R is the event rate in events/s, T the frame window in seconds, C the consumption rate of one core in events/s, B bytes per event. A packed event (two coordinates, polarity, timestamp) lands near 8 bytes. A tight per-event loop — index check, clamp, pixel update — consumes on the order of 100 million events per second on a desktop core (a few clock cycles per event at about 3 GHz). Treat both figures as example inputs; your loop differs, the method does not.

Stress scene at the sensor's practical ceiling: R = 50 Meps = 50,000,000 events/s, preview window T = 5 ms, a 200 fps generate-on-demand loop.

• Events per window: 50,000,000 × 0.005 s = 250,000 events
• Data rate: 50,000,000 events/s × 8 B = 400 MB/s
• Read time: 250,000 events / 100,000,000 events/s = 2.5 ms
• Race exposure: 2.5 ms / 5 ms = 50%

The delivery thread appends new events during half of every window, reallocating while the render loop dereferences. The first batch that forces the buffer to move ends the process: a segfault within seconds, on every run.

At R = 2 Meps the same loop sees 10,000 events per window (2,000,000 × 0.005 s), reads them in 0.1 ms (10,000 / 100,000,000), and sits at 2% exposure (0.1 / 5). The reader usually finishes between appends, and the allocator often reuses reserved capacity without moving anything, so mean time to crash stretches from seconds to hours. That is why the bug looks intermittent. It is deterministic; the event rate only decides whether you live long enough to see it.

The fix: one thread owns the buffer

The rule that removes the crash: from the moment events arrive until the finished frame leaves the accumulator, exactly one thread touches that memory, and ownership moves with the data. Two structures satisfy it.

Pattern A — generate inside the callback. The architecture this SDK family expects, and the one to reach for first.

1. Create the frame-generation object once, before streaming starts. After that, only the callback path uses it.
2. In the events callback, append the new packet to the accumulator. Same thread that received the packet, so there is nothing to lock.
3. Check elapsed time against the frame window (5 ms in the example above). When the window is due, generate the frame — still inside the callback.
4. Move the finished frame into a mutex-guarded hand-off slot, or swap it into a double buffer whose backing memory each side owns alone. Deep-copy anything another thread keeps, the way our stereo-frame guide copies Mat data out of an SDK buffer.
5. Return promptly. Display, saving, inference and the calibration math run elsewhere, on the copy; schedule them from the consumer side, not inside the callback.

Illustrative skeleton. The std names are real; Event, EventPacket, Frame, append and generateFrame are placeholders for your SDK's types, and onEvents keeps whatever callback signature the SDK documents.

std::vector<Event> acc;                     // owned by the callback thread
std::chrono::steady_clock::time_point lastEmit{};
std::mutex guiMutex;
std::deque<Frame> guiQueue;                 // hand-off queue, UI thread consumes

void onEvents(const EventPacket& batch) {   // runs on the delivery thread
    append(acc, batch);                     // same thread as delivery: no race
    auto now = std::chrono::steady_clock::now();
    if (now - lastEmit >= std::chrono::milliseconds(5)) {
        Frame f = generateFrame(acc);       // still the delivery thread
        lastEmit = now;
        std::lock_guard<std::mutex> lk(guiMutex);
        guiQueue.push_back(std::move(f));   // ownership moves with the frame
    }
}

Pattern B — one consumer thread. If the callback must return in microseconds, deep-copy each packet into a bounded, drop-oldest queue and let a single consumer thread accumulate and generate. The copy costs little per packet and buys a clean ownership line: the callback owns its local copy, the consumer owns everything after it. Copy the buffer before you queue it, keep the queue bounded as the memory-leak fix above does, and a stalled consumer degrades into dropped data instead of a memory abort.

Verify the fix before you trust it

1. Build a debug version with ThreadSanitizer and AddressSanitizer. TSan reports the race even when it does not crash; ASan names the exact heap-use-after-free access. Each costs roughly a factor of two in run time — a small cost for a proof.
2. Drive the scene to the sensor's ceiling: a flashing checkerboard on a monitor, moved toward the lens, reaches tens of millions of events per second without a motion stage.
3. Soak: ten minutes at ceiling rate for a first pass, a full multi-hour run for a release candidate. Zero sanitizer reports and zero allocator aborts is the pass mark.
4. Watch process memory across the soak. A flat line confirms the queue is bounded and the leak-family aborts are gone with it.
5. Re-run the calibration sequence you actually ship, checkerboard sweep included, because on-demand generation only exercises its heavy path on real data.

When the refactor does not fix it

A single-owner version that still aborts under a clean ThreadSanitizer report moves the suspicion out of your algorithm. Match the SDK's documented compiler and build configuration first: mixed C runtimes let two allocators release each other's memory, and an ABI mismatch corrupts calls at the boundary. If the crash survives that too, package the minimal reproducer, the stack trace and the SDK version, and ask the vendor's support one direct question: what threading contract does the documentation state for the frame-generation class? The report doubles as your record that the defect is not in your code.

Prevent it in the next integration

• Print the thread id of the events callback once, before designing anything; every structure you choose afterwards depends on the answer.
• One owner per buffer. Move ownership with the data, by copy or by swapping buffers each side allocated for itself.
• Copy at the boundary between the SDK and your code; the microseconds are cheaper than a heisenbug that appears only at the customer's scene.
• Bound every queue and write the overflow policy down. Drop-oldest is a decision; an unbounded push is an accident.
• Keep GUI, file and network calls off the delivery thread; hand them finished, copied data.

The capture side of the rig

The calibration program itself usually pairs the event sensor with a frame-based camera that watches the same checkerboard for ground truth. Our guide on multi-camera extrinsic calibration in one shared coordinate frame covers the geometry side. A 2.5 MP color GigE area scan camera such as the MV-CU020-90GC, global shutter on a 1/1.7-inch sensor, tracks a checkerboard cleanly. For a compact multi-sensor head, the 1.6 MP GigE board-level MV-CB016-10GM-M-W with an M12 mount and a global shutter sits beside each event sensor; the board-level cameras range holds the smaller variants and area scan cameras the larger ones.

Point the camera configurator at your resolution and interface for the remaining capture parts, or contact us with the sensor and the SDK version and we will help you specify the capture side. More frame-and-buffer guides sit under the cameras topic.