Optimal image downsample factor from the Fourier spectrum

A 3840 × 2160 frame can look sharp and still carry no more real detail than a 960 × 540 one, because the extra pixels hold interpolation artifacts instead of content. Storage, network load and inference time all scale with the pixel count, so empty pixels cost bandwidth and cycle time on every frame. An SSIM sweep against the original only tells you that you lost something after you have already lost it, and its threshold moves from scene to scene. The frame reports its own usable resolution in its Fourier spectrum: real detail occupies small radii, and content stops where the spectrum merges into noise. The steps below measure that edge, turn it into a downsample factor with one formula, and verify the result.

Why bandwidth decides the downsample factor

Detail is spatial frequency. The finest structure an n-pixel-wide image can carry sits at the Nyquist limit of 0.5 cycles per pixel, and a spectrum bin at radius r of an n-point transform corresponds to f = r / n cycles per pixel. Downsampling by a factor D keeps only the frequencies below 0.5 / D; everything above it is removed by the antialias low-pass or, without one, folds back into the picture as aliasing. The largest safe factor therefore follows from one measurement: the radius where the image's spectral energy dies.

Past that radius a frame holds noise, interpolation residue and compression junk. A frame upscaled from 1920 × 1080 to 3840 × 2160 shows a cliff at bin n / 4, because interpolation cannot create frequencies above the source grid; badly deinterlaced video shows a second shoulder at half height. An SSIM sweep answers the wrong question in both directions: it weights all structure by local contrast, so it is dominated by the strongest low-frequency content, and a threshold that passes a smooth scene fails a textured one. The spectrum locates the content before you cut, region by region.

What you need

• Python 3 with NumPy; every transform used below is in the NumPy FFT reference.
• One mono8 frame, or a grayscale conversion of a color frame, straight from the camera rather than a compressed copy.
• A square crop, 512 to 1024 pixels on a side, centered on the finest detail you must keep.
• The two functions and the test below.

Any modern area scan camera supplies suitable frames; the camera finder filters the range by resolution and interface if you are sizing a new one.

Measure the spectral edge in five steps

1. Crop a square region of interest. Take 1024 × 1024 float pixels around the finest structure you need — a bar code, a scratch zone, a texture patch — and convert to float.
2. Window and transform. Apply a separable Hann window, take the 2-D FFT with DC shifted to the center, and take log10 of the magnitude. Normalizing the peak to 0 dB makes frames comparable.
3. Set the display window. Map the normalized log values from 10^-6 to 10^0 onto your display range for the classic spectrum view: a bright core, texture rings, and the ring where it dies. That range is for your eyes; the decision uses the floor in the next step.
4. Find the edge radius. Average the spectrum over integer radii, estimate the noise floor as the mean over the outer 5 % annulus, and mark the edge where four consecutive bins fall below the higher of the floor and -40 dB relative to peak. The factor 10^(-40/20) = 0.01 means 1 % of peak amplitude, about 2.5 gray levels on an 8-bit frame, which sits under typical sensor noise. Requiring four consecutive bins ignores single-bin dips and narrow spectral valleys.
5. Convert to the factor. Round D_max down to an integer and decimate with a low-pass before you resize; the filter rules are in the companion guide on antialiasing when you downsample or warp images for vision.

import numpy as np

def band_radius(img, floor_db=-40.0):   # img: (n, n) float; floor_db: dB below peak
    n = img.shape[0]
    win = np.hanning(n)[:, None] * np.hanning(n)          # separable Hann window
    spec = np.log10(np.abs(np.fft.fftshift(np.fft.fft2(img * win))) + 1e-12)
    spec -= spec.max()                                    # peak normalized to 0
    yy, xx = np.mgrid[0:n, 0:n]
    r = np.hypot(yy - n // 2, xx - n // 2).astype(int)    # integer radius map
    floor = float(spec[r >= int(0.95 * (n // 2))].mean()) # outer 5 % annulus
    prof = np.array([spec[r == k].mean() for k in range(n // 2)])
    thr = max(floor, floor_db / 20.0)                     # 1 % of peak, min. floor
    below = prof <= thr
    run = np.convolve(below.astype(int), np.ones(4, dtype=int), 'valid')
    if not (run == 4).any():
        return n // 2                                     # band fills to Nyquist
    return max(1, int(np.argmax(run == 4)))               # first 4-bin run below

def decimation_factor(n, r_edge):       # largest safe downsample factor, int
    return max(1, int(0.5 / (r_edge / n)))

n = 1024                                # test: band edge known exactly, bin 120
yy, xx = np.mgrid[0:n, 0:n]
disk = (np.hypot(yy - n // 2, xx - n // 2) <= 120).astype(float)
img = np.fft.ifft2(np.fft.ifftshift(disk)).real        # band-limited synthesis
edge = band_radius(img)
assert 118 <= edge <= 128, edge                        # window skirt ~ 4 bins
assert decimation_factor(n, edge) == 4                 # 1024 / (2 * 120) = 4.27 -> 4
print(edge, decimation_factor(n, edge))                # 125 4

The test synthesizes an image whose band edge is known exactly, a filled disk of radius 120 bins in a 1024-point spectrum, and the estimator must return the factor that the exact bound 1024 / (2 * 120) = 4.27 predicts, rounded down to 4. The estimator reads a few bins past the true edge because of the window's spectral skirt, which makes the decision conservative: you lose at most a sliver of in-band detail, never take aliased content.

Worked example: a 4000 × 3000 frame

This is the frame size where the method pays for itself. The LS-CMR-0166 captures 4000 × 3000 frames over USB3 at 30 fps:

Crop A, over a machined surface: the azimuthal mean dies at r_edge = 256 bins of n = 1024, so f_max = 256 / 1024 = 0.25 cycles per pixel and D_max = int(1024 / (2 * 256)) = 2. The output is 4000 / 2 × 3000 / 2 = 2000 × 1500. On mono8 the original frame is 12 MB, so the stream runs 12 MB × 30 = 360 MB/s; decimated, 3 MB × 30 = 90 MB/s — one quarter of the bytes with the inspected content intact.

Crop B, over a fine-pitch connector: the edge sits at r_edge = 470 bins, f_max = 470 / 1024 = 0.459, D_max = int(0.5 / 0.459) = 1. That region carries content almost to Nyquist, so keep it at full resolution and crop it in the acquisition code; the guide on setting the camera ROI in Python shows the settings. The frame-level factor is the minimum across crops: 2 for the archive and display streams, full resolution for that inspection ROI.

Reading the spectrum picture

Shapes tell you where the content went:

• Energy flat to the outer ring: the frame fills the sampled band; the estimator returns n / 2 and the factor 1.
• A plateau ending in a cliff: an upscaled source; the cliff marks the source grid, at n / 4 or n / 8 for factor-2 and factor-4 upscales.
• A second shoulder at half or quarter height: interlaced or stitched readout residue; the true content limit is the lower shoulder, not the first edge.
• Periodic spikes along the axes: a repeating grid — sensor pattern, or the 8 × 8 block lattice of a compressed copy.
• A smooth ring falloff with no cliff: natural texture; the -40 dB floor decides, and the factor is stable unless the edge sits near a boundary.

On color sensors the demosaiced luminance carries less real detail than the pixel count suggests; the guide on color camera spatial resolution explains why.

When the right factor is 1

If every crop reads D_max = 1, the frame fills the band and decimation buys nothing. The lever moves to the camera side. Cropping to the inspected region cuts pixels before transfer, and binning changes what the sensor reads out; the guide on binning versus sensor readout and frame rate covers that side. If the spectrum is full yet edges still look soft, lens resolution rather than pixel count is the limit — the guide on small pixel camera selection works that trade-off, and the camera resolution calculator runs the task-side pixel budget from field of view and smallest feature.

High-resolution sensors raise the stakes. The MV-CH250-25TC-F-NF delivers 25 MP frames over 10GigE:

A 25 MP feed decimated by 2 drops to 6.25 MP, one quarter of the pixels per inference call. Whether that is safe is exactly what the spectrum answers, and the smallest measured feature sets the other bound: a feature spanning p pixels keeps three or more of them while D ≤ p / 3. Below about 2 px per feature, quantization noise starts to dominate any edge or centroid fit.

Verify the factor, skip the traps

Verification per frame:

1. Run three crops from different regions; the factors must agree within one integer step, and the frame factor is the minimum.
2. Decimate, then rerun the estimator on the result: its edge must now sit within about 5 % of the new Nyquist, because the kept band is fully used.
3. Subtract the rescaled decimated image from the original; the difference must carry no energy below r_edge of the original. Anything left there is filter leakage, and at -40 dB it is invisible.

Traps, in the order they bite:

• Compressed copies. Measure the camera frame, never a JPEG or a re-encode; the block lattice adds spikes that fake a too-high or too-low edge.
• Bayer input. One color per pixel on a grid; measure mono or demosaiced luminance.
• Rectangular crops. The circular average and f = r / n hold on the square axis; pad to square or split the region.
• One crop decides all. A flat background reads a short edge radius and a connector pad a long one; the minimum is the rule.
• A blank frame reads a huge factor. The arithmetic is correct and the result is pointless; the one-to-one zoom check on a real feature catches it.

The decision for each case

Run the two functions on three crops of your worst frame, then commit:

• D_max ≥ 2: decimate the stream by D_max behind an antialias low-pass before storage, transmission and inference. Pixel count and bytes fall by the factor squared — 4× at D = 2, 16× at D = 4 — and nothing inside the kept band is lost.
• D_max = 1 on inspection crops only: keep full resolution where you inspect, decimate the rest for archive and display, and let camera-side ROI do the cropping.
• D_max = 1 everywhere with soft edges: the optics set the limit, not the pixels — re-plan the camera and lens, starting from the area scan cameras or the camera configurator.

The neighboring material lives on the systems topic page, and the team answers setup questions through the contact page.