Extract video frames for a training dataset locally

A clip you already have — a phone video of the station, a DVR export, a recording from the camera that will run the job — is the most direct source of training frames that match your deployment pixels. The decision is the sampling step. Keep 1 frame in 30 from a 30 fps clip and every still covers one second of footage, so 10 minutes of video yields 600 candidates. A short Python loop with OpenCV extracts those frames locally, saves them as JPEG and drops the stills that repeat the last kept one. It is the file-based counterpart to seeking every frame in JavaScript, and it leaves you a folder of stills whose count you can check against arithmetic.

What you need

• Python 3 with two packages: pip install opencv-python numpy
• The clip or clips and their frame rate; most players show it, and the loop reads it too
• Disk: a quality-95 1080p JPEG lands around 0.5-1 MB, so plan for roughly 1 GB per 1,000 stills
• One output folder per clip, named after the clip

Everything below runs offline — nothing leaves the machine.

Choose the step from the frame rate

The step is the only real decision. One still per second is the default for inspection footage: 30 fps / 1 = step 30. For a fast-changing scene drop the step to 10-15, which is 2-3 stills per second; for a fixed camera over a mostly static station raise it to 60-90, a still every 2-3 seconds.

Worked example: a 2:00 clip at 30 fps carries 30 frames/s × 120 s = 3,600 decoded frames. With step 30 the clip yields 3,600 / 30 = 120 stills. Ten clips of that length give 1,200 stills, which lands near the budget many detection projects plan around — the how-many-images article works that number out for YOLO training.

Read fps with cv2.CAP_PROP_FPS before you size the step; containers sometimes report 0, and the fps-returns-zero repair fixes that read first. FRAME_COUNT is only metadata — the loop's decoded counter is the number your stills math has to match.

Run the loop

1. Install the packages: pip install opencv-python numpy.
2. Make one output folder per clip and set step from the section above.
3. Run this listing. It decodes the file once, keeps every step-th frame and skips a still that repeats the last kept one:

import os
import cv2
import numpy as np

video_path = 'clips/line1_station3.mp4'   # input clip
frames_dir = 'frames/line1_station3'      # one folder per clip
step = 30                                 # keep 1 frame in 30: 30 fps -> 1 still per second
diff_cut = 2.0                            # duplicate cut, mean abs diff on a 0-255 scale

os.makedirs(frames_dir, exist_ok=True)
cap = cv2.VideoCapture(video_path)        # documented API: opens the container
fps = cap.get(cv2.CAP_PROP_FPS)           # documented API: float, frames per second

kept, seen = 0, 0
prev_thumb = None                         # 64x36 gray thumbnail of the last kept still
while True:
    ok, frame = cap.read()                # documented API: uint8 array, shape (H, W, 3)
    if not ok:
        break
    if seen % step == 0:
        thumb = cv2.resize(frame, (64, 36)).mean(axis=2)
        duplicate = prev_thumb is not None and np.abs(thumb - prev_thumb).mean() < diff_cut
        prev_thumb = thumb
        if not duplicate:
            name = os.path.join(frames_dir, 'line1_station3_%06d.jpg' % seen)
            cv2.imwrite(name, frame, [int(cv2.IMWRITE_JPEG_QUALITY), 95])
            kept += 1
    seen += 1
cap.release()
print('decoded %d frames, kept %d, container reported %.1f fps' % (seen, kept, fps))

4. Read the last line. For the 2:00 example clip you expect decoded 3600 frames, kept 120 when nothing is filtered; a smaller kept count means the duplicate cut fired, which is the filter doing its job.

Decoding is sequential inside one file, and the listing does no per-pixel Python work beyond a 64 × 36 thumbnail — keep it that way, because per-pixel loops are what kill frame rate in OpenCV. The OpenCV documentation lists the VideoCapture flags used above. To use every core, run several clips at once as separate processes.

How the duplicate filter chooses

Two stills one second apart on an idle line are the same picture. The filter resizes each candidate to 64 × 36 gray, takes the mean absolute pixel difference against the last kept still and rejects anything below diff_cut. On the 0-255 scale, static stretches sit near 0-3 and real scene changes push above 10, so 2.0 keeps genuine variety and drops idle repeats. Print the values for one clip first; if your bands sit elsewhere, move the cut into the gap between them.

Verify before you annotate

Three checks take half a minute:

import glob

# continues from the extraction script
saved = sorted(glob.glob(os.path.join(frames_dir, '*.jpg')))
assert len(saved) == kept, '%d files on disk vs %d kept' % (len(saved), kept)
sample = cv2.imread(saved[len(saved) // 2])
assert sample is not None and sample.shape[0] > 0

• The count matches the arithmetic: 3,600 decoded / step 30 = 120 expected for the worked clip, minus each still the cut dropped.
• The middle file opens with real pixels at the clip's resolution; a file that refuses to open is usually mislabeled rather than corrupt, and the load-by-content guide sorts those out.
• Coverage is the loop's, not luck's: zero-padded names keep the first, middle and last second of the clip in order in the folder.

Common mistakes

• Sizing the step from FRAME_COUNT when the decoded count differs — you come up short. Size from fps, check against the decoded counter.
• Stepping through idle hours. The filter catches the repeats, but record short clips around the events worth labeling instead.
• Saving every frame in case. The 2:00 example carries 3,600 frames, about 3,600 × 0.8 MB ≈ 2.9 GB — step first, then save.
• Recompressing. Extract at quality 95 once; re-saving a saved JPEG at lower quality bakes a second loss into the pixels you will annotate.
• Mixed color orders. cv2 reads and writes BGR; frames arriving as RGB from another pipeline need one conversion or reds and blues swap — the recorded-frames color guide covers it.
• One flat folder for all clips. Per-clip folders with the clip name inside each filename keep splitting, labeling and debugging possible.

When the clip is not the best source

If the station exists, the best frames come from the camera that will run the job: a hardware-triggered grab loop saves full-quality frames with no compression generation at all. If your recording is a depth camera's bag file rather than a video, extract the color frames from the bag instead of re-recording a playback window. Recorded footage also beats rendered imagery for deployment match — the synthetic-data photorealism question marks where renders still earn their place.

If the station is new, record the clips with the camera you will deploy. The LS-CMR-0166 delivers 30 fps at 4000 × 3000 resolution over USB3, and the camera SDK's recording function writes those frames to a video file the loop above reads unchanged.

Once the model is trained, inspection can run where the pixels come from. The MV-SC5060XM-08M-WBN carries a 6 MP mono sensor and inspects on the device over GigE; the smart cameras range covers on-device inspection and the area scan cameras the PC-connected route.

When a station still needs its full chain picked, the camera finder filters by resolution, shutter and interface, and the configurator walks lens, light and controller next. The systems guides hold the companion pieces — grab loops, fps reads, coordinate handling — for the deployment side.