System designHow-to

Detect object motion direction with OpenCV frame differencing

Track a moving object's centroid across frames, compare positions over a fixed horizon and decide left or right motion direction with a threshold.

LumaSpec Vision10 min read Share on LinkedIn

On this page 9 sections
  1. 01The decision rule: the sign of the centroid displacement
  2. 02What you need
  3. 03Build the tracker, step by step
  4. 04Worked example: a person crossing a 640 x 480 frame at 30 fps
  5. 05Choose the horizon and the deadband for your scene
  6. 06When frame differencing is the wrong tool
  7. 07Verify it works, then trust it
  8. 08Where this leaves you
  9. 09Questions

You can already see motion in the frames; what you still need is which way it is going, so that a person crossing one way fires one action and crossing the other way fires a different one. OpenCV has no single direction function, and you do not need one. Threshold a difference or background-subtraction mask, take the largest contour's centroid, keep a timestamped history of those centroids, and compare the current x against the x from about half a second ago. The sign of that displacement is the direction. This walkthrough builds the loop in Python, proves the decision logic on numbers before any camera is involved, and marks the scenes where the method stops working.

The decision rule: the sign of the centroid displacement

Frame differencing tells you where the scene changed, not which way anything moved — the frame differencing walkthrough covers that half on its own. Direction comes from adding tracking: one position per frame, compared across a fixed time horizon.

That position is the centroid of the largest foreground contour. From the contour's moments, the center is x = m10 / m00 and y = m01 / m00 in pixels; the contour-to-coordinate-array walkthrough works the same math into NumPy arrays.

dx = x_now - x_then v_x = dx / dt direction = sign(dx) if abs(dx) > deadband, else none

Commit to the rule: dx above the deadband means motion to the right, because image x grows to the right; dx below minus the deadband means motion to the left; anything inside the band means no direction yet. The band exists because a centroid jitters by single-digit pixels even on a still scene, while a walking person covers hundreds of pixels across a 0.3-0.5 s horizon.

Comparing consecutive frames is the trap. At 30 fps a walking person shifts the centroid only about 25 px per frame, the same order as the jitter; a 0.5 s horizon turns the same walk into hundreds of pixels, an order of magnitude clear of the noise.

What you need

The pipeline needs three things and no GPU.

  • Python 3 with opencv-python and numpy (pip install opencv-python numpy).
  • A camera that does not move. A USB webcam at 640 x 480 px and 30 fps is enough to prototype; a motion-tracking webcam built on frame differencing shows the same pipeline end to end on plain hardware.
  • Even, reasonably stable light. The mask inherits every flicker, every shadow swing and every auto-exposure step.

For a station that runs all day, swap the webcam for an industrial camera and keep the code identical except for the frame source. The area scan cameras category filters by shutter, mount and interface, and the camera finder narrows the list by resolution. The MV-CU020-90GC is a 2.5 MP color GigE camera with a global shutter and a C-mount, and its SDK hands frames to the same grab loop a webcam uses.

Area scan camerasHikrobot MV-CU020-90GC 2.5 MP Area Scan Camera, GigE, GMAX4002, ColorMV-CU020-90GC
Sensor format
1/1.7"
Lens mount
C-mount
Interface
GigE
Resolution
Up to 3 MP
$309.00View partBuild a system with it

If the decision has to switch a PLC input and a PC is not welcome in the cell, the smart cameras category is the on-device route: inspection runs on the camera itself and the device drives its own outputs. The LS-SMC-0063 pairs a 1.6 MP sensor with a 16 mm lens and runs 60 fps over GigE.

Smart cameras & vision sensors1.6 MP smart camera, 60 fps, 16 mm lens, white light (polarized, transparent light) (LS-SMC-0063)LS-SMC-0063
Interface
GigE
Resolution
Up to 3 MP
$1,110.00View part

Build the tracker, step by step

  1. Capture and calm each frame. Read a frame, convert it to gray and blur it with a 5 x 5 Gaussian kernel; cv2.cvtColor and cv2.GaussianBlur do both, and the blur keeps sensor noise out of the mask.
  2. Build the foreground mask. cv2.createBackgroundSubtractorMOG2(history=300, varThreshold=25, detectShadows=True), applied once per frame, learns the empty scene and marks departures from it. MOG2 writes shadow pixels as 127 and true foreground as 255, so a threshold at 200 keeps only real foreground.
  3. Despeckle. cv2.morphologyEx with cv2.MORPH_OPEN and a 5 x 5 kernel erases isolated speckles before they become contours.
  4. Gate the largest contour by area. cv2.findContours with cv2.RETR_EXTERNAL and cv2.CHAIN_APPROX_SIMPLE returns the blobs; ignore anything under MIN_AREA_PX = 500 px^2, set to roughly a fifth of your object's area in the frame, so a flicker never becomes the object.
  5. Store timestamped centroids. Append (time.monotonic(), x) to a collections.deque. Timestamp every sample and never assume a fixed frame rate: cap.get(cv2.CAP_PROP_FPS) returns 0 on some streams, and dropped frames make frame counting lie.
  6. Decide. Find the latest sample at least HORIZON_S = 0.5 s old, form dx and apply DEADBAND_PX = 40.
import time
import cv2
import numpy as np
from collections import deque

HORIZON_S = 0.5      # s, look-back window for the direction test
DEADBAND_PX = 40.0   # px, |dx| inside this is "no direction"
MIN_AREA_PX = 500    # px^2, contours smaller than this are ignored

def decide(history, now_t):
    """history holds (t: float, x: float) samples; returns 'right', 'left' or None."""
    then_t = now_t - HORIZON_S
    older = [s for s in history if s[0] <= then_t]
    if not older:
        return None
    x_then = older[-1][1]
    x_now = history[-1][1]
    dx = x_now - x_then
    if dx > DEADBAND_PX:
        return "right"
    if dx < -DEADBAND_PX:
        return "left"
    return None

bgsub = cv2.createBackgroundSubtractorMOG2(history=300, varThreshold=25, detectShadows=True)
history = deque(maxlen=600)          # about 20 s of samples at 30 fps
cap = cv2.VideoCapture(0)            # index 0 is the default capture device

while True:
    ok, frame = cap.read()
    if not ok:
        break
    t = time.monotonic()             # timestamp each frame; do not count frames
    gray = cv2.GaussianBlur(cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY), (5, 5), 0)
    fg = bgsub.apply(gray)
    _, fg = cv2.threshold(fg, 200, 255, cv2.THRESH_BINARY)   # drop shadow pixels
    fg = cv2.morphologyEx(fg, cv2.MORPH_OPEN, np.ones((5, 5), np.uint8))
    cnts, _ = cv2.findContours(fg, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
    if cnts:
        c = max(cnts, key=cv2.contourArea)
        if cv2.contourArea(c) >= MIN_AREA_PX:
            m = cv2.moments(c)
            history.append((t, m["m10"] / m["m00"]))
    state = decide(history, t)
    if state == "right":
        pass   # left-to-right crossing: fire your action here
    elif state == "left":
        pass   # right-to-left crossing: fire your other action here

Prove the decision function on numbers before pointing a camera at anything:

from collections import deque

h = deque([(0.00, 200.0), (0.50, 600.0)])
assert decide(h, 0.50) == "right"    # dx = 600 - 200 = +400 px

h = deque([(0.00, 600.0), (0.50, 200.0)])
assert decide(h, 0.50) == "left"     # dx = 200 - 600 = -400 px

h = deque([(0.00, 500.0), (0.50, 530.0)])
assert decide(h, 0.50) is None       # dx = +30 px, inside the 40 px deadband

Worked example: a person crossing a 640 x 480 frame at 30 fps

The inputs: a webcam stream of 640 x 480 px at 30 fps; the tracker logs a centroid at x = 200 px at t = 0.00 s and at x = 600 px at t = 0.50 s; the horizon is 0.50 s and the deadband is 40 px.

The rule in words: the direction is the sign of dx between the current sample and the latest sample at least 0.50 s old, provided abs(dx) clears the deadband.

The arithmetic: dx = 600 - 200 = +400 px. Speed v = 400 px / 0.50 s = 800 px/s, which at 30 fps is 800 / 30, about 27 px per frame. abs(400 px) is ten times the 40 px deadband, so the decision is right: the person walked left to right and the welcome event fires. The mirror case, x falling from 600 px to 200 px, gives dx = -400 px and fires the departure event. A centroid drifting from 500 px to 530 px gives dx = +30 px, inside the band, so nothing fires — a person standing near the door stays silent.

What moves the result: halve the walking speed and dx halves to 200 px, still five times the deadband, so the decision survives; run the camera at 15 fps and the 0.50 s horizon becomes an 8-frame lookback at the same walking speed. Push the deadband above 200 px and a half-speed crossing stops announcing itself.

Choose the horizon and the deadband for your scene

Two settings carry the design. Pick the horizon as the longest delay your response allows, because the event lands about one horizon after entry. Pick the deadband from measured jitter: log centroids for a minute on a still scene and take the largest swing; a well-lit blob of a few thousand pixels stays within single-digit pixels. Keep the deadband at least three to five times that jitter, and check that a real crossing displaces the centroid at least five times the deadband.

Scene Horizon Deadband Fires above
Person at a door, webcam at 30 fps 0.50 s (15 frames) 40 px 80 px/s
Box on a conveyor at 30 fps 0.30 s (9 frames) 25 px 83 px/s
Small fast part at 60 fps 0.10 s (6 frames) 15 px 150 px/s

"Fires above" is deadband divided by horizon — the slowest speed the scene still calls directional. Two people crossing together merge into one contour and still give a sane direction when both move the same way; if they can cross in opposite directions, match centroids between frames by nearest distance instead of taking the largest blob.

When frame differencing is the wrong tool

Verify it works, then trust it

Run these five checks before the loop drives anything real:

  1. Log every (t, x) sample to a CSV and plot one crossing: the trace should be one clean ramp of a few hundred px, not noise.
  2. Cross the frame left to right, then right to left, at normal speed and at half speed; all four runs must fire with the right sign.
  3. Stand still in frame for 30 s: nothing may fire.
  4. Time an event against the crossing: it lands about one horizon, 0.50 s here, after entry. If that is too slow, cut the horizon and the deadband together, never the deadband alone.
  5. Retest after a lighting change — a door opened, afternoon sun. If jitter has grown past a third of the deadband, raise the deadband or fix the light.

The mistakes, in the order they usually happen: comparing consecutive frames only, so dx sits at noise level; skipping the area gate, so a 20 px flicker becomes the object when the person steps out of frame; trusting a fixed frame rate instead of timestamps; letting shadow pixels into the mask; and nudging the camera after tuning, which invalidates the geometry the deadband was sized against.

Where this leaves you

Commit the rule as stated: direction is the sign of the centroid displacement over a 0.5 s horizon, with a 40 px deadband for a doorway webcam, scaled by the table above for faster scenes. Prototype on the webcam, prove the decision function on the asserts, then move the frame source to an industrial camera — the camera configurator builds one from sensor, mount and interface. The systems topic page collects the surrounding techniques, from subtraction to registration and tracking, and the smart camera route above takes the decision onto the device when no PC belongs in the cell.

Questions

Does OpenCV have a built-in function that returns the direction of motion?

No. cv2.absdiff and the MOG2 background subtractor return a change mask, and the direction logic is yours to write on top: centroid displacement over a horizon, or optical-flow vectors from cv2.calcOpticalFlowPyrLK or cv2.calcOpticalFlowFarneback when you need per-point motion. For a left-or-right call the centroid route is fewer lines and much easier to debug.

How far apart in time should the two centroid samples be?

Work in time, not frame counts: 0.3-0.5 s for people, 0.1-0.3 s for conveyor parts, sized so a real crossing displaces the centroid at least five times the deadband. At 30 fps a 0.5 s horizon is a 15-frame lookback. Timestamp each sample, because streams that drop frames make fixed frame counts unreliable.

Why does my direction logic flip or go quiet when the person stops moving?

The background model absorbs a stationary person, the contour disappears and the history goes stale. Hold the last decision until fresh foreground appears, and require the contour to survive three consecutive frames before re-arming. Fast lighting changes blank the mask the same way, so shade windows and lock the exposure before tuning.

Will a rolling shutter camera give the wrong direction?

Not for a left-right call at walking speeds. Rolling shutter shears fast-moving shapes, which shifts a contour centroid a little, but the sign of a 400 px displacement is safe. Specify a global shutter model when parts cross the frame in a handful of frames or when the same camera also measures size or position.

How do I fire two different PLC signals for the two directions?

From the PC loop, publish one event per crossing to the PLC over the fieldbus or I/O interface you already run, latched so each crossing sends a single pulse. If the decision must live on the machine, a smart camera or vision sensor drives its discrete outputs straight from the inspection result, with no PC in the cell.