An mp4 recording of the line is the cheapest training data you will collect: one afternoon of footage holds more variety than a day of staged photo sessions. The trap is extraction. Dumping every frame to disk turns a 2 min clip into 3,600 images, most of them the same scene twice over, and labeling hours burn on duplicates and motion-blurred stills. This guide gives you the interval rule, a short OpenCV script that saves every Nth frame as a JPG, and two filters — blur and near-duplicate — that run before annotation. If you can capture stills straight off a live camera instead, Extract video frames for a training dataset locally covers that path.
Set the interval from frame rate, not from feel
Frame rate sets the budget. A clip recorded at 30 fps holds 30 frames for every second of footage, so the interval N decides everything downstream:
Every 10th frame of a 30 fps clip yields 3 stills per second. That is the starting point this guide uses: enough stills to cover a part crossing the view from several positions, few enough that consecutive keeps are rarely the same scene. For a fixed camera watching a part that needs about 2 s to cross the field, five stills per crossing is plenty, and the arithmetic sets the interval directly:
N = fps × crossing time / stills per crossing = 30 × 2 / 5 = 12
Faster scenes shrink N; a line that idles between parts grows it. Re-derive the interval per clip instead of copying a number between lines.
What you need
- The mp4 files, in whatever codec your recorder produced. OpenCV decodes common H.264 and MJPEG output through its bundled backend; if a file will not open, the codec is the first suspect, not the script.
- Python 3 with two packages:
pip install opencv-python numpy. - Disk space by arithmetic, not guesswork: 360 stills at 1080p occupy roughly 108 MB at about 0.3 MB per high-quality JPG, so scale by your resolution and clip count before a batch run.
- If the recording still has to be made on the line, Write MP4 video from a camera on an embedded board covers the capture side.
One broken file costs an afternoon. A recording that stopped mid-write loses its index, and the tail frames will not decode until the file is repaired — recovering a truncated MP4 recording is its own procedure, so check the last second of every clip before batching.
Extract every Nth frame with a short OpenCV script
-
Read the frame rate first. The container usually reports it, but odd files report 0.0; Video FPS returns zero in OpenCV: read frame rate reliably shows how to get a number you can trust. Every later calculation uses fps.
import cv2 cap = cv2.VideoCapture("clip01.mp4") fps = cap.get(cv2.CAP_PROP_FPS) # float, e.g. 30.0 print(fps) # 0.0 means fix the container question first -
Set N from the interval rule above — 10 for the example used throughout this guide.
-
Loop sequentially and keep frame i when
i % N == 0. Counting costs the same decode time as seeking and keeps frame indices exact. On hour-long recordings, watch memory: the failure mode in Out of memory in a video loop: fix the per-frame image leak starts when frames accumulate by accident. -
Print totals at the end — that print is the test proving the run. On the worked example below it prints
3600 360 30.0.import cv2 N = 10 # interval: keep every 10th frame clip = "clip01.mp4" # input video prefix = "out/clip01" # output folder and name stem cap = cv2.VideoCapture(clip) fps = cap.get(cv2.CAP_PROP_FPS) # float, e.g. 30.0 i = 0 # frames read kept = 0 # frames written ok, frame = cap.read() # frame: BGR image array while ok: if i % N == 0: cv2.imwrite(f"{prefix}_{i:06d}.jpg", frame) kept += 1 i += 1 ok, frame = cap.read() cap.release() print(i, kept, fps) # proof: 3600 360 30.0 -
Spot-check ten outputs at full size, oldest and newest included. Look for exposure or white-balance drift the player hid, and for zero-byte files — the mark of a decode hiccup mid-run.
-
Run each clip into its own folder and filter per clip. Merging first mixes blur and duplicate statistics across scenes; filter per clip, then merge the survivors.
Drop blurred frames before they reach the labels
Motion blur is frozen into the recording: exposure happened at capture time, and no extraction setting undoes it. Score sharpness and cut in the gap. The standard quick metric is the variance of the Laplacian — sharp frames carry many strong edges, so the variance is high:
import cv2
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # frame: one candidate image
score = cv2.Laplacian(gray, cv2.CV_64F).var() # float sharpness score
Run the score over all candidates of a clip and sort it. The distribution splits in practice: a sharp cluster, and a low tail of blurred frames roughly an order of magnitude beneath it. Set the cut in the gap between the clusters, then open the twenty frames nearest the cut and confirm by eye which side they belong on. If the two clusters do not separate, the whole clip was shot at the edge of exposure — re-record with a shorter exposure rather than tuning a threshold.
Drop duplicate frames with a difference test
Consecutive keeps from an idle scene differ only by sensor noise and compression artifacts, and the mean absolute difference tells them apart:
import cv2
import numpy as np
# prev_gray: the previous kept candidate's gray image
diff = float(np.mean(cv2.absdiff(gray, prev_gray))) # mean |difference| in 8-bit levels
Set the floor at the noise, not at a round number. 8-bit recordings carry a few gray levels of temporal noise and compression adds about as much again, so a mean difference of 3 levels or less means nothing moved: keep the earlier frame and drop the later one. A part entering the view moves the mean by tens of levels in one step, which is exactly the change worth keeping.
This filter protects inference too: overlapping frames of the same part cause duplicate detections at test time, the failure covered in Conveyor vision duplicate detection with overlapping images. Fewer near-identical training images, fewer double boxes at deployment.
Worked example: sizing the interval for a 2 min, 30 fps recording
Inputs. A conveyor recording saved as clip01.mp4: 120 s of footage (the capture length), 30 fps read from the container and confirmed by the printed totals, interval N = 10 — the starting interval for 30 fps footage, worth 3 stills per second — a duplicate floor of 3 gray levels (8-bit noise plus compression), and a blur cut read off the clip's own sharpness histogram.
Rule. Candidates = fps × seconds / N, and the filters keep only frames that are both sharp and changed.
Arithmetic. Frames in the clip: 30 fps × 120 s = 3,600. Candidates: 3,600 / 10 = 360 stills, arriving at 30 / 10 = 3 per second. Disk: 360 stills × roughly 0.3 MB ≈ 108 MB ≈ 0.1 GB at 1080p.
Decision. 360 candidates is a set you can filter and label in one sitting, so N = 10 stands. Watch the dedupe counter: if it keeps fewer than half the candidates, the camera idles between parts and every 10th frame was always going to double up — widen to N = 15, which drops the yield to 30 / 15 = 2 stills per second (240 candidates from the same clip) at zero cost, since the frames you lose were duplicates. If the blur cut removes more than a third of survivors, the problem is upstream exposure: fix the capture, not the threshold.
Match the stills to the camera that will run the model
A dataset is only as good as its resemblance to deployment. The model sees the deployed sensor's resolution, shutter behavior, noise and optics — not the recording's. Two adjustments close most of the gap:
- Resize or crop candidates to the deployment pixel count before labeling. Training on 3840 × 2160 frames and deploying a 1.6 MP camera wastes the detail and mis-scales object sizes.
- Match the shutter. A part moving fast through the view smears under a rolling shutter; favor global-shutter hardware for capture and deployment alike, or the model learns motion distortion as a feature.
Deployment hardware spans a wide range: a 1.6 MP smart camera with a fixed 16 mm lens running 60 fps behaves differently from a 12 MP USB3 area scan camera delivering 4000 × 3000 frames at 30 fps. Sample at the pixel count you will deploy, and if the camera is not chosen yet, choose it now — area scan cameras and smart cameras cover both ends.
- Sensor format
- 1/1.7"
- Lens mount
- C-mount
- Interface
- GigE
- Resolution
- Up to 3 MP
- Sensor format
- 1/1.7"
- Interface
- USB3
Check the set before you annotate
Three checks take ten minutes and save days of labeling:
- Counts per clip. The script prints kept and read per clip; a clip that kept under a quarter of its candidates was recorded while the line idled — re-record it or leave it out.
- Spread. Stills should come from every clip and every lighting state on the line; sort by filename and check the source histogram. Variety matters more than count — the per-class targets in How many images do you need to train a custom YOLO dataset give the numbers to aim at.
- A pilot label of 30 stills. If annotation stalls on ambiguous classes or unclear edges, extraction is fine and imaging is not: fix focus, exposure or camera placement before labeling the rest.
When saving every frame is the right call
Three cases reverse the rule. Clips under about 10 s hold 300 frames or fewer at 30 fps — extract all of them; the filters can still trim, but an interval saves nothing worth having. Temporal models that consume frame sequences need consecutive frames, so sampling destroys the signal. And a quick visual review of a long recording in a browser is a playback task: Extract all frames from a video in JavaScript: seek every frame covers that side. For labeled training sets from line recordings, the interval plus the two filters wins — fewer images, sharper images, and labeling hours spent on frames that teach the model something.
Before you annotate: the checklist
Run down this list after each batch:
- Frame rate read and trusted — a 0.0 or a guess here invalidates every interval calculation.
- Printed totals match the arithmetic: 30 fps × 120 s = 3,600 read, 360 kept at N = 10.
- Sharpness histogram shows two clusters with a gap; the cut sits in the gap and the twenty nearest frames were reviewed by eye.
- Duplicate floor set at 3 gray levels of measured noise, and each clip kept at least a quarter of its candidates.
- Stills resized or cropped to the deployment pixel count, shutter behavior matched.
- Every clip and lighting state contributes stills; no class leans on one recording.
- The mp4 originals are archived until the model ships.
Next step: pin down the camera the model will run on, because that choice sets the pixel count you sample at. The camera finder filters by resolution, shutter and interface; the configurator starts at the camera step:
The configurator matches a camera, lens, light, light controller and cables to each other, step by step, and lists only parts that fit what you already picked.
The rest of the systems guides cover the capture and pipeline sides, and contact us if a set-up question survives this guide — we answer application questions with numbers.
Questions
Should extracted dataset frames be saved as JPG or PNG?
Save JPGs at quality 90 or higher for bounding-box and classification labeling: compression artifacts sit far below labeling tolerance and files stay around 0.3 MB at 1080p. Save PNG when the stills themselves will be measured — pixel-accurate segmentation masks, low-light noise studies, dimensional checks. In all cases keep the mp4 originals until the model ships, so you can re-extract at a different interval without re-recording.
Can I extract the frames with FFmpeg instead of Python?
Yes, one command keeps every 10th frame: ffmpeg -i clip.mp4 -vf "select=not(mod(n\,10))" -fps_mode vfr out_%06d.jpg. The select filter tests the frame number n; -fps_mode vfr prevents duplicated output frames. On FFmpeg builds older than version 5, use -vsync vfr instead. For blur and duplicate filtering at the same time, the Python loop is easier to extend than a filter chain.
Why do extracted frames look flat or soft compared with the video player?
Usually color range. Players expand limited-range video, stored with levels 16-235, to fill the screen, while saved frames keep the stored values and look washed out. Record with full range where the option exists, or expand the levels once during extraction. The softness is the player upscaling; extract at native resolution and never enlarge a frame before labeling.
Is keyframe extraction good enough for a training set?
No — use it for previews. Keyframes sit wherever the encoder saw a scene change, so their spacing is uneven and their timing biased toward cuts; a dataset built from them under-covers long steady stretches of the cycle. Decode keyframes only for a fast contact sheet of a long recording. Run the interval loop for the set you will label, so coverage is uniform by construction.
How many frames should one clip contribute to a class?
Work from a per-class target, not from the clip length: a few hundred sharp, varied stills is a workable start for one distinct class, and classes that resemble each other need several times more. Variety — lighting states, positions, part lots — matters more than raw count. Let validation performance, not folder size, close the decision on whether to extract more.


