System designTroubleshooting

Capture video during a process delay: slice blocking waits

Why one blocking delay blanks the video, and how to slice it into 20 ms steps so you can capture video during the process and keep the display live.

LumaSpec Vision10 min read Share on LinkedIn

On this page 9 sections
  1. 01Why does the video freeze while the process runs?
  2. 02How long can a single wait be before frames drop?
  3. 03How do you slice a 200 ms delay so the video keeps displaying?
  4. 04What goes inside each slice?
  5. 05Does slicing stretch the process time?
  6. 06When should you use a second thread instead of slicing?
  7. 07How do you check the video stays live during the delay?
  8. 08How do you keep the freeze from coming back?
  9. 09Questions

A station program often has to do two jobs at once: run a sequential process — splicing fibers, dosing, fastening, a serial transaction — and keep a live video picture on screen while the process runs. Put both in one while loop with a single long delay and the picture goes blank for a fifth of a second on every cycle, even though the camera and the process each work perfectly on their own. The fix is not extra hardware. It is how the wait is written: slice the blocking delay into short steps, grab and draw one frame inside each step, and the video stays live on one thread.

Why does the video freeze while the process runs?

The video freezes because the delay call blocks the only thread in the loop: while that one wait runs, no fetch happens, no draw happens, and the display keeps showing the last frame until the wait returns.

One thread runs the loop in strict order: send the serial command, wait, read the reply, repeat. The grab and draw calls execute only between iterations, so the display repaints once per cycle at best. The camera, meanwhile, keeps exposing and delivering a frame every 33 ms at 30 fps, but nobody collects those frames while the wait holds the thread. New frames pile into the driver's queue or fall off its end, and the picture stays frozen at whatever it showed before the wait.

That is why each half works alone and the pair fails together: the wait is not the process's problem, it is the display's.

How long can a single wait be before frames drop?

Keep every single blocking wait at or below one frame interval — 33 ms for a 30 fps camera, 17 ms at 60 fps — because a wait longer than a few intervals outlasts the driver's frame queue and frames start to drop.

t_frame = 1000 ms / frame rate 30 fps: t_frame = 1000 / 30 ≈ 33 ms frames inside one 200 ms wait = 200 / 33 ≈ 6

Six frames arrive during one full block and the display shows none of them. Queue depth differs by SDK — a few frames on some drivers, more on others — but a queue only delays frames; it never displays them. Only a fetch followed by a draw puts a new picture on screen, so the wait has to end, or be cut, before the next frame matures.

A deep buffer absorbs a short burst, as in camera burst capture faster than link bandwidth: use the buffer, but a blocking wait on every cycle is not a burst — it is a schedule, and no buffer fixes a schedule. Grab at least once per frame interval and the display stays within one frame of the camera.

How do you slice a 200 ms delay so the video keeps displaying?

Split the delay into slices of one frame interval or less — 200 ms becomes ten 20 ms steps — and inside each step wait 20 ms, grab the newest frame, draw it, then continue the process work in the next step.

  1. Time the whole delay once with timestamps around the call, for example 200 ms between serial frames.
  2. Pick a slice time at or below one frame interval: 20 ms for a 30 fps camera.
  3. Divide: 200 / 20 = 10 slices.
  4. Rewrite the wait as a loop over the slices: wait until the slice deadline, grab the newest frame, draw it, then run one slice's worth of process work.
  5. Re-time the whole delay afterwards; it should still come out at about 200 ms.
delay_ms = 200                      # total process wait per cycle
slice_ms = 20                       # one frame interval or less
n_slices = delay_ms / slice_ms      # 200 / 20 = 10
t_start  = now()
for k = 1 to n_slices
    wait_until(t_start + k * slice_ms)   # deadline, not a plain sleep
    frame = grab_latest()                # newest frame in the queue
    draw(frame)                          # paint the display
    process_step()                       # this slice's serial work

The function names above are generic placeholders; substitute your SDK's own grab, draw and wait calls and keep the structure.

The camera for the worked example: the LS-CMR-0166 is a 12 MP USB3 camera that runs 30 fps at 4000 × 3000.

Area scan cameras12 MP USB3 area scan camera, IMX226, 1/1.7" rolling shutter, 30 fps, 4000 × 3000 resolution (LS-CMR-0166)LS-CMR-0166
Sensor format
1/1.7"
Interface
USB3
Resolution
12 MP
Shutter
Rolling
$560.00View partBuild a system with it

At 30 fps the frame interval is 33 ms, so 20 ms slices reach a grab before every frame matures. The old loop repainted the display 1000 / 200 = 5 times per second; the sliced loop repaints 1000 / 20 = 50 times per second, ten times as often, and every repaint shows a frame at most one interval old. The process sees the same delay either way.

What goes inside each slice?

Each slice carries three jobs — wait until the slice deadline, grab the newest frame, draw it — and grab plus draw should fit in about half the slice budget, roughly 10 ms of a 20 ms slice, leaving the process work the other half.

A grab from a filled queue returns as soon as the copy is made, so the fetch itself is rarely the expensive part. Set the fetch timeout to one slice or less so an empty queue cannot hang the slice; the habit of one wait call per frame is covered in frame didn't arrive within timeout: one wait call per frame. Draw cost rises with image and window size, so measure it once on the target host and hold it inside the half-slice budget.

Does slicing stretch the process time?

With deadline waits the total stays 200 ms: ten slices of 20 ms absorb the 6 ms of grab and draw inside each 20 ms slot, so the process still finishes in a fifth of a second, while plain sleeps stretch the same loop to about 260 ms.

Loop form One cycle takes Display during the wait
One 200 ms block 200 ms Blank, 1 repaint per cycle
10 × sleep(20) + 6 ms grab and draw 10 × 26 = 260 ms, +30% 50 updates/s
10 × deadline(20) + 6 ms grab and draw 200 ms, unchanged 50 updates/s

If grab plus draw overruns the slot — say it measures 25 ms — ten deadline slices take 10 × 25 = 250 ms, a 25 percent overrun. Widen the slices then: the full delay in 4 × 50 ms steps still gives 1000 / 50 = 20 updates per second, which reads as live motion, or move the grabs to a thread.

When should you use a second thread instead of slicing?

Slice when you own the delay code; move the grabs to a second thread when a call you cannot cut — a motion move that blocks until done, a serial read with no timeout — holds the loop for hundreds of milliseconds at a time.

Slicing keeps one thread: no lock around the shared image, no marshaling of draws to the UI thread, nothing to deadlock, and the loop still reads top to bottom. A thread buys independence from the blocking call at the cost of synchronization; the pattern is laid out step by step in GUI freezes during image capture: run fetch off the main thread. Default to slicing, and reach for a thread only when the block is not yours to cut.

The rule repeats across automation software. An HMI panel can hang on a blocking while loop launched at startup, as in this HMI boot loop and USB programming failure caused by a blocking while loop. In PLC structured text, a blocking delay inside a FOR loop stalls the scan the same way. On microcontrollers, a blocking while() loop around a sensor read starves everything else sharing that loop. Wherever the wait lives, the cure keeps the same shape: short slices, or a second thread.

How do you check the video stays live during the delay?

Instrument the loop and count renders: log a timestamp on every draw, then check that the gap between draws stays near 20 ms and never exceeds about 40 ms across the whole delay.

Three measurements settle it. Draw timestamps first: the gaps should sit at the slice time, and a spike back to the full cycle time means one blocking call survived somewhere — timestamp the segments inside the slice and the slow one names itself. Freshness second: stamp each frame at grab time and display its age; a healthy loop shows frames no older than one frame interval, 33 ms at 30 fps. The process third: the serial round trip should still complete in 200 ms plus small jitter, proving the slices did not tax the process. Run the check across a full process cycle, not one pass.

How do you keep the freeze from coming back?

Adopt three loop rules: no blocking wait longer than one frame interval (33 ms at 30 fps), one grab and draw per slice, and grab plus draw held under half the slice budget; every new blocking call gets timed against them before it joins the loop.

  • Time every new blocking call before it enters the loop; anything over one frame interval gets sliced or threaded.
  • Keep camera setting changes out of the streaming loop — apply them between cycles or during idle, as in apply camera settings while streaming without stalling capture.
  • Re-measure grab and draw whenever the image size, the display or the host changes; the 6 ms budget belongs to this host and this window.
  • Treat a repeated stall in any connected device as the same failure: a wait that cannot be cut is a call that needs a thread.

Frames can still drop after the timing is clean when two or more cameras share one port. Give each camera its own port instead: the LS-FGB-0004 is a PCIe x4 4-port PoE Gigabit card that turns one slot into four camera ports.

Frame grabbersPCIe x4 4-port PoE Gigabit Ethernet card, Intel i210AT (LS-FGB-0004)LS-FGB-0004
Interface
GigE
$359.00View partBuild a system with it

When the totals grow past GigE, the frame grabbers and host interface cards category covers 10GigE and CoaXPress cards with two to four ports each. To size the camera side of the station, start from the area scan cameras category or its camera finder, walk the whole chain in the system configurator, and bring any timing number that will not close to the contact page.

Questions

Why does the first picture after the delay show an older scene?

Frames kept arriving while the loop was blocked and sat in the driver queue, so the first grab after the wait returns the oldest queued frame, sometimes hundreds of milliseconds old. Most SDKs offer a latest-frame or snapshot-style fetch that discards the backlog; if yours does not, grab in a tight loop until the queue empties and draw only the last frame. Stamping frames on arrival and showing their age makes staleness visible during commissioning instead of surprising you in production.

Does a 60 fps camera force smaller slices?

Yes: the slice should stay at or below one frame interval, and 1000 / 60 works out to about 17 ms. Use 15-17 ms slices with one grab per slice, which gives roughly 60 updates per second. If grab plus draw cannot fit inside 17 ms on your host, draw a smaller image or move grabbing to a second thread. Keeping a 20 ms slice at 60 fps skips every other frame; it still looks live, but it shows half the frames.

Can a blocking serial read be sliced the same way?

Only when the read call offers a timeout you can set. Configure the timeout to one slice, call it repeatedly, and accumulate bytes until the full message arrives; the wait is then built from short reads and the grabs fit between them. A read with no timeout parameter blocks for an uncontrolled time and cannot be sliced, so that call belongs on a second thread.

Does drawing the image cost more than grabbing it?

Often, yes. A fetch from a filled queue returns in well under a millisecond, while decoding and painting a full-resolution image can take several milliseconds and grows with display size. Measure both once on the target host, then budget. If drawing dominates, paint a decimated copy: every second pixel and line costs about a quarter of the pixels and still reads as smooth motion on a live process. Keep grab plus draw under half the slice.