Detecting camera vibration with OpenCV frame tracking

A camera that moves between frames poisons everything downstream: measurements drift, calibration goes stale and defects seem to wander. The video stream itself is the cheapest vibration sensor you own. By tracking image features from frame to frame in Python, you can measure how far the camera shifts, in pixels and in millimetres, log it over a shift, and separate camera shake from real part motion. This guide gives the full method, the arithmetic, and the mounting fixes that stop the motion at its source.

What you need

• A mounted camera streaming live images (GigE or USB3, grabbed with your usual SDK or OpenCV's VideoCapture).
• Python 3 with opencv-python and numpy installed.
• A roughly static scene in view: a fixed target, the belt surface between parts, or the machine frame.
• Your field of view in millimetres at the working distance, so you can convert pixels to physical units.
• A stable mount. A rigid camera stand with a heavy base makes the measurement meaningful; a camera taped to a bracket measures the tape.

If you have not yet characterised the field of view, work it out first with the lens focal length calculator; the pixel-to-millimetre conversion below depends on it.

The idea in one paragraph

The camera cannot see itself move, but it sees the world move the other way. If the scene is fixed and the image shifts by 4 pixels, the camera shifted by 4 pixels' worth of scene. OpenCV's feature tracker finds distinctive points in frame N, follows them in frame N+1, and the median shift of those points is your camera motion estimate. The median matters: parts passing through view, specular highlights and flicker create outliers, and the median ignores them.

Step 1: grab two frames and track features

The core measurement needs about thirty lines. Run it on a live stream or a saved video; the logic is the same.

import cv2
import numpy as np

cap = cv2.VideoCapture(0)  # or your SDK's frame source

ret, prev = cap.read()
prev_gray = cv2.cvtColor(prev, cv2.COLOR_BGR2GRAY)

# Good features to track: corners with strong gradients
p0 = cv2.goodFeaturesToTrack(prev_gray, maxCorners=200,
                             qualityLevel=0.3, minDistance=7)

while True:
    ret, frame = cap.read()
    if not ret:
        break
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    p1, st, err = cv2.calcOpticalFlowPyrLK(
        prev_gray, gray, p0, None,
        winSize=(21, 21), maxLevel=3,
        criteria=(cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT,
                  30, 0.01))
    good = st.ravel() == 1
    if good.sum() >= 10:
        dx = np.median(p1[good, 0, 0] - p0[good, 0, 0])
        dy = np.median(p1[good, 1, 0] - p0[good, 1, 0])
        print(f"dx={dx:+.2f} px  dy={dy:+.2f} px  points={good.sum()}")
    prev_gray = gray
    p0 = p1[good].reshape(-1, 1, 2)
    if len(p0) < 50:  # re-seed features when too few survive
        p0 = cv2.goodFeaturesToTrack(gray, 200, 0.3, 7)

Three details decide whether the numbers are trustworthy:

1. Use the median, not the mean. One moving part can drag a mean by tens of pixels; the median shrugs it off.
2. Watch the surviving-point count. If fewer than about ten tracks survive, the shift estimate is noise. Re-seed features as shown.
3. Lighting must be steady. Flickering mains-powered lights move feature intensities and create false shifts. If you see large oscillation at mains frequency, fix the light before blaming the mount.

Step 2: convert pixels to millimetres

Take your field of view at the working distance. Say the camera covers 96 mm across a 1920-pixel-wide image:

scale = 96 mm / 1920 px = 0.05 mm/px

A measured frame-to-frame shift of 6 pixels is therefore 0.3 mm of camera motion. Multiply by the frame rate for velocity: at 30 fps, 6 px per frame is 180 px/s, or 9 mm/s of shake. These two numbers, amplitude and velocity, tell you most of what you need for diagnosis: large amplitude at low velocity suggests a loose joint; small amplitude at high velocity suggests resonance or an unbalanced motor nearby.

Step 3: separate vibration from real part motion

If parts or the belt are visible, the tracker will follow them, not the camera. Three clean ways out:

1. Capture between parts. Trigger or sample frames when the field of view is empty. Even a 100 ms window per part gap is enough for two frames.
2. Aim at the structure. Point the test camera at the machine frame or a stuck-on printed target during the test run, then put it back.
3. Compare runs. Measure with the line stopped (pure camera motion) and with it running (camera plus parts). The difference isolates the belt contribution.

Global motion is not the only artifact. Rolling shutter cameras shear the image when the camera or scene moves during readout, which inflates apparent dx on one side of the frame. Compare the median shift in the left and right halves of the image: if they disagree strongly, you are seeing rolling shutter, and a global-shutter camera or shorter exposure will make the measurement honest. For a general camera at this level of work, a compact global-shutter model such as the MV-CU020-90GC removes the artifact entirely.

Step 4: read the pattern in the log

Plot displacement magnitude against time and classify what you see:

A common trap: auto-exposure and auto-gain make the image itself move slightly, and no mount fix will help. Lock both features before you measure, and read how in the guide to automatic iris, exposure and gain control. Likewise, if the shift appears only after a reconnection, the cause may be software rather than mechanical, as described in the article on auto exposure ROI instability from detection feedback.

Step 5: fix the mechanics

Once the log shows real camera motion, fix it in this order, cheapest first:

1. Torque every joint. Loose knobs on stands and arms are the number one cause. Mark the settings with paint pens so a change is visible.
2. Decouple from the conveyor. Never clamp the camera stand to the conveyor frame. Belt motors and product impacts travel straight through. Give the stand its own footing.
3. Shorten and stiffen the lever. Every centimetre of overhang multiplies motion at the sensor. Keep the camera close to the stand column.
4. Add mass and damping. A heavier base lowers the amplitude; a rubber or Sorbothane pad under the base reduces high-frequency transmission. Test after each change with the same script: the log is your acceptance test.
5. Shorten the exposure. If motion blur, not shift between frames, is the problem, more light and a shorter exposure freeze the image even when the mount is imperfect.

For a permanent cell, a slide-table style stand such as the LS-MNT-0040 gives you a heavy base and a locked-off position once alignment is done, which is exactly what a vibration log wants to see: flat lines. If the shake changes after any camera or robot work, re-verify the alignment with the procedure in the article on vision calibration recovery after camera or robot changes, because a mount that once moved can shift the calibration too.

Checklist before you trust the numbers

• Exposure and gain locked, auto modes off.
• Scene static during capture, or capture timed between parts.
• Median shift used, with at least ten surviving feature tracks.
• Scale (mm/px) computed from a measured field of view, not assumed.
• Log at least one minute; classify the pattern before touching hardware.
• Re-run the same script after each mechanical fix to confirm the improvement.

When to stop measuring and just build it stiffer

If the log shows persistent oscillation near the line's operating frequency, tuning the mount is a job of weeks. Sometimes the faster answer is to replace a tall flexible arm with a short rigid column on its own base, or to mount the camera closer to the action. When you are planning the cell layout from scratch, the camera stand finder and the system configurator let you pick a stand by base size and travel before anything is bolted together. And if vibration analysis becomes a routine part of your commissioning, the same feature-tracking code doubles as a drift monitor in production: log one shift value per minute and alert when it exceeds your pixel budget.

For a next step, browse the camera stands and area scan cameras categories, or contact our applications team through the contact page with your log attached; the pattern usually points straight at the fix.

More reading in the systems section of the knowledge hub: the article on conveyor tracking camera placement relative to the robot covers the mechanical side of mounting near moving machinery, and machine vision autofocus and image stabilization explained handles related focus drift, another consequence of a camera that ends up not quite where you mounted it.