Frame differencing for motion detection: subtract two frames

Two consecutive frames of the same scene differ by almost nothing: pixels that did not change subtract to zero, and only the pixels an edge crossed keep a value. That residue is the simplest motion detector in machine vision, and it needs no model, no training and no background image. This article gives you the working numbers: the threshold that separates real motion from sensor noise, the minimum displacement the object owes you per frame, the exposure that keeps the moving edge sharp, and the save-order mistake that breaks most first builds. The worked example runs at 250 mm/s on a 200 mm field of view, and every part mentioned comes from the shop's area scan cameras and smart cameras.

What does subtracting two consecutive frames actually show?

Subtracting two consecutive frames leaves large values only where something changed between them. A pixel that sat at 120 grey levels in both frames lands at 0. An edge that moved across it shows the full step, typically 40-80 levels on a well-lit part.

Use the absolute difference. A signed subtraction hides half the event — wherever the new frame is darker, negative values clip to zero and the trailing edge vanishes. The absolute value shows two bands per moving object, one where it left and one where it arrived, each band as wide as the displacement in pixels. The rest of the frame is not exactly zero either: sensor noise leaves a few levels of spread under every pixel, which is why a threshold exists at all.

Holding frame n-1 while frame n arrives, then swapping the two buffers, is laid out step by step in Frame differencing with two image buffers for change detection. This page starts where that one stops: choosing T, choosing the camera, and getting the results saved without the usual ordering errors.

What threshold separates motion from noise?

Set the difference threshold at six times the noise standard deviation of a static-scene difference image. On a typical well-lit 8-bit set-up (σ of 3-5 levels) that lands at 20-30 grey levels; if you cannot measure σ yet, start at 25.

Measure instead of guessing. Run 20 frame pairs on the static, production-lit scene, subtract each pair, and read the standard deviation of the difference off a few flat patches — that number is σ. A clean, well-lit 8-bit line usually sits at 3-5 levels. Mains ripple or a loose mount pushes it up, and the 6σ rule follows the noise automatically.

The multiplier has a basis you can check. Difference noise is close to Gaussian, and the odds of noise passing 6σ are about 2 in a billion per pixel. A 2 megapixel frame therefore throws one false pixel only every few hundred frames. At 3σ the same frame lights thousands of noise pixels every cycle, which is why low-multiplier thresholds flicker on their own.

Add the area guard: ignore connected blobs under 50 pixels, about 0.5 mm² at the worked example's 0.104 mm per pixel — a 0.8 mm speck. If thresholding itself is new, Edge detection without a vision library: threshold basics covers polarity and the single-cut decision. The bit-depth version of these numbers is in Machine vision camera bit depth: choosing 8, 10 or 12 bits.

How much must the object move between frames?

Plan for the object to move at least 3 pixels between the two frames. Below that, the changed band at each edge is 1-2 pixels wide and competes directly with noise. Above it, every strong edge produces a clean band your threshold catches.

The floor is not arbitrary. The changed strip at each edge is exactly as wide as the displacement, so a 1-pixel step puts a single pixel column per edge into play — hard to separate from noise clusters unless σ is tiny. From about 5 pixels up, the bands are wide enough to measure a centroid, and 5-50 pixels per frame is the window worth aiming at with your frame-rate choice.

Worked example: a conveyor at 250 mm/s under a camera whose 1920-pixel width covers 200 mm of belt. Pixel scale: 200 / 1920 = 0.104 mm per pixel. At 30 fps the interval is 33.3 ms, so the part advances 250 × 0.0333 = 8.33 mm per frame. That is 8.33 / 0.104 ≈ 80 pixels — far past the 3-pixel floor, wide enough that both edge bands are unmistakable.

Exposure: 0.104 / 250 = 0.0004 s, so shoot at 0.4 ms or shorter and the moving edge blurs under 1 pixel; 0.8 ms tolerates 2. Cutting a 10 ms exposure to 0.4 ms costs a factor of 25, about 4.6 stops. Opening from f/4 to f/1.4 returns 8 of it, and the last 3× comes from gain or more light on the part. Add light before you lengthen the exposure: every doubling of exposure doubles the blur.

If the stream is too slow for the displacement you want, ROI vs binning vs subsampling: choosing faster acquisition lists the standard ways to buy frame rate. Deciding pixels runs the same arithmetic backwards — field of view and smallest event in, pixel count out:

Which camera and shutter type suit frame differencing?

Choose a monochrome, global-shutter camera. Mono gives you one clean grey-level channel with no Bayer interpolation to soften differences. A global shutter grabs the whole frame at one instant, so fast motion does not smear rows at different times.

A rolling-shutter sensor reads its rows in sequence, so when a part crosses many pixels in one frame, the bottom rows saw the scene a fraction of a millisecond later than the top. As a motion flag that still works. For anything measured off the band geometry, global shutter removes the variable. Mono removes the other one: no interpolation sits between the photon and your difference, so edges stay about a pixel sharper.

Two catalog picks that fit the brief directly:

The MV-CU020-90GC is the baseline motion gate: 2.5 MP, GigE, global shutter, C-mount. The MV-CH120-11UC carries the headroom when the follow-up is classifying what moved, not just flagging it: 12 MP over USB3, global shutter, high-speed version. Both are C-mount, so the lens follows the field of view you laid out rather than the camera.

Can the check run without a PC?

Yes when the check reduces to a comparison and an area count. Smart cameras run acquisition, image logic and the pass/fail output on the device, and a model such as the LS-SMC-0063 captures 1.6 MP frames at 60 fps, fast enough for most presence and motion checks.

A smart camera or vision sensor runs the whole loop in the device itself, over Fast Ethernet or GigE, so the hardware is one box and a lens. The trade is freedom: on-device pipelines are built around presence, position, count and code checks, and you give up the open-ended library a PC grants. For a motion gate, that is rarely a loss.

Two on-device routes from the catalog:

The LS-SMC-0063 arrives set up for exactly this kind of check: 1.6 MP at 60 fps, with its 16 mm lens and white light already fitted. The HK-TD032-12-WBN is the mono vision-sensor route: 1.6 MP, global shutter, Fast Ethernet, M12 mount. At the worked example's 0.104 mm pixel scale, 60 fps puts the 5-50 pixel displacement window over belt speeds from roughly 30 to 310 mm/s.

How do you stop light changes from faking motion?

Lock exposure and gain to fixed values before you trust a difference image. A half-stop auto-exposure shift multiplies light by 1.41, so a pixel at 100 grey levels moves about 40 levels between frames and reads as motion across the whole scene.

The usual culprits: auto exposure still enabled, re-balancing whenever the histogram drifts; mains-driven light at 50 or 60 Hz integrated unevenly by a long exposure; daylight through a door; a gain step. Each shifts the whole frame together, so the difference image lights up everywhere at once and the blob count explodes.

Fix in this order: switch exposure and gain to manual once the scene is lit; drive the light steadily or strobe it. When the exposure must stay long, hold it to a whole multiple of the mains period (20 ms at 50 Hz). Shield or baffle ambient daylight. If the auto loop keeps fighting you, Auto exposure and auto gain on color cameras: which pixels count explains what the camera is averaging, and Ambient light variation in pick and place vision: control it collects the shielding and timing fixes.

What are the steps to build the check?

The build is five steps: measure the noise floor, fix the exposure, grab frame pairs, subtract and threshold at six sigma, then decide on the connected area — with the arithmetic checked once against your real conveyor speed and field of view before the first production run.

You need a camera on a fixed mount with a stable frame clock, a lens covering the field of view, the light on a steady drive, and an environment that holds two frames — an SDK on a PC or a smart camera's on-device pipeline.

1. Fix the geometry and exposure. Compute the pixel scale s = FOV / image width in pixels, set the frame rate so displacement lands in the 5-50 pixel window, and set manual exposure to s / v or less — 0.4 ms at the worked example's scale and speed. Lock gain at the same time.
2. Measure the noise floor. Grab 20 frame pairs on the static, production-lit scene, difference them, and take σ from the flat patches. Set T = 6σ, floor 20 levels on 8-bit.
3. Hold frame n-1 in a buffer you own. Copy the pixels out as each frame arrives; the SDK reuses its own buffers, and a difference built on a reused buffer compares the frame with itself and reads zero.
4. Subtract, threshold, count. Absolute difference, threshold at T, label the connected blobs, drop anything under 50 pixels (about 0.5 mm² at the example scale). Call it motion when a surviving blob — or a total kept area you set, say 200 pixels — appears.
5. Verify before production. With the line stopped, 1,000 frames must give zero events. Hand-feed a part through the field and every pass must fire. Dim the light 10 percent and repeat the static run to confirm the cut still holds.

For a dry run on desktop hardware before any of this touches the line, the frame differencing webcam motion-tracking walkthrough runs the identical two-frame compare on ordinary webcam footage.

How do you save the difference frames without save errors?

Save each thresholded difference frame as its own image (PNG or TIFF) or stream them into a video file one frame at a time. Open the writer after the first frame exists — most save errors are an ordering problem, not a codec problem.

The classic ordering bug: the writer session opens before the first frame exists, so the first write call has nothing to write and the session errors out. Create the writer after the first difference frame is in hand, close it cleanly at shutdown, and give the acquisition call a timeout margin. A mid-run timeout usually means the loop demanded a frame the camera had not finished delivering, not a codec fault.

Copies matter as much as ordering. Write your own copy of every frame before the next grab. The failure mode and the fix are the same as in Stereo frames saved wrong: copy Mat data from the SDK buffer, where the saved file changes underneath its owner because the driver reused the memory.

On format: an 8-bit, 2 MP difference frame is about 2 MB raw, so saving every frame at 30 fps is roughly 60 MB/s of disk. Save only when the decision fires — one PNG or TIFF per event — and write video frame by frame only when you truly need the sequence. Save consecutive camera images without writing code covers capture-to-disk by configuration instead of code. On a 12-bit camera, Saving 12-bit camera images without losing intensity data is the checklist that keeps the levels you paid for.

When does frame differencing give the wrong answer?

Expect two blind spots: the interior of a uniformly coloured moving object cancels to zero, leaving only edge bands. A moving camera or drifting background makes everything appear to move — if the background itself moves, switch to a registered-subtraction or background-model approach.

The outline-only result is not a fault: uniform interiors cancel because every interior pixel maps onto itself, so presence detection works off the bands alone. Silhouette recovery needs accumulated difference frames or a background model. When displacement exceeds the part's own size, successive positions stop overlapping and the difference splits into two separate ghosts — count both, or lengthen the frame interval.

A moving background kills the method outright: a vibrating mount, a drifting belt below the part, a pan of the camera, and every pixel differs. Register the frames first or maintain a running background model — Object detection on a moving background with registered subtraction walks that variant. Slow changes hide too: a part warming under the light creeps a level or two per minute and never crosses a 6σ cut measured an hour ago. Re-measure σ at commissioning and after any lighting change.

The decision, plainly: two-frame differencing at T = 6σ (20-30 levels on 8-bit), a 50-pixel area guard, exposure under one pixel of motion, on a mono global-shutter camera running at the frame rate your speed math dictates. A smart camera replaces the PC when the decision is a count. A background model replaces two-frame differencing when the background itself moves. If you are speccing hardware now, start at the camera configurator or the cameras finder; the rest of the camera-side write-ups sit on the cameras topic page.