Matching lens resolution to pixel size: the Nyquist budget method

A 20 MP camera behind a 5 MP-class lens produces 20 MP of soft image, and no software restores contrast that never reached the sensor. The sensor's pixel pitch fixes the highest spatial frequency it can record; the lens decides how much of that frequency survives. This guide gives you the budget method: convert pixel pitch into a line-pairs-per-millimetre target, set the contrast the lens must hold for your application, then convert the result into object space so you can state the smallest feature the system resolves. Two worked examples follow: a measurement set-up on 20 MP-class pixels, and a presence check where the matched-pixel answer costs less.

The sensor sets the target: Nyquist from pixel pitch

One line pair is one dark bar plus one bright bar, and sampling theory needs two pixels to record it. The Nyquist frequency of a sensor is therefore one divided by twice the pixel pitch; patterns finer than that alias into false detail rather than disappearing. That frequency, not the megapixel count, is what your lens budget has to reach.

with f_N in line pairs per millimetre (lp/mm) and p the pixel pitch in mm. A 3.45 µm pixel is 0.00345 mm, so f_N = 1 / (2 × 0.00345 mm) = 145 lp/mm. The same arithmetic gives 208 lp/mm at 2.4 µm, 182 lp/mm at 2.74 µm and 85 lp/mm at 5.86 µm.

The sensor side of that trade is covered in Nyquist limit and camera resolution: match sensor to lens. Here the sensor is given and the lens is the variable.

What the lens must hold at that frequency

A lens MTF curve plots contrast against spatial frequency in lp/mm, and two working targets cover most vision work. For defect detection and presence checks, use the rule of thumb that the lens holds at least 30% contrast at two thirds of Nyquist: 139 lp/mm for the finest row of the table (2/3 × 208). For measurement and colour inspection, hold at least 20% contrast at Nyquist itself, because subpixel edge fitting and colour-channel registration both degrade once contrast there falls toward 10%.

The megapixel rating printed on a lens is shorthand for exactly this. A 5 MP rating promises usable contrast near 145 lp/mm on the named format; a 20 MP rating targets the 182-208 lp/mm range. The rating transfers only between sensors of similar pitch, and the image circle still has to cover the format. Read the curve at your production f-number before trusting either number: how to read lens performance curves marks the points that matter.

Convert to object space: the feature size you can state

Here m is magnification, image size over object size. The 3-pixel rule is the standard detection margin argued in camera resolution for defect detection, and object-space pixel size follows from magnification step by step in calculate object space pixel size.

Worked example 1: a 0.03 mm defect on a 50 × 35 mm field

1. Three pixels across a 0.03 mm defect need an object pixel of 0.03 / 3 = 0.010 mm.
2. Width needs 50 / 0.010 = 5,000 pixels and height 35 / 0.010 = 3,500 pixels, so 17.5 MP; the 20 MP class is the nearest fit.

3. Take a 20 MP, 1" sensor of about 13.2 × 8.8 mm at the finest pitch in the table. At m = 0.25 the field covers 13.2 / 0.25 = 52.8 mm by 8.8 / 0.25 = 35.2 mm, framing the 50 × 35 mm part with margin. The object pixel is 2.4 / 0.25 = 9.6 µm, so the smallest defect the rig shows reliably is 3 × 9.6 µm = 28.8 µm, inside the 30 µm requirement.
4. The pitch demands f_N = 1 / (2 × 0.0024 mm) = 208 lp/mm, and measurement duty sets the 20% contrast floor there. At this station's f/2.8 the Airy diameter is 2.44 × 0.55 µm × 2.8 = 3.8 µm, inside the 4.8 µm two-pixel budget, so a wide-open aperture serves the pitch rather than fighting diffraction.
5. The paraxial estimate WD ≈ FOV × f / sensor width gives 52.8 mm × 16 / 13.2 = 64 mm of working distance at a 16 mm focal length.
6. Distortion carries a measurement cost: 0.3% across a 52.8 mm field is 0.158 mm of edge error, so either buy a low-distortion rating or calibrate it out with the workflow in lens distortion correction for pixel-to-mm measurements.

A lens rated 16-25 MP on a 1.1" circle, opening at F2.8-22 with under 0.3% distortion, covers every line of that budget:

For the sensor, a mono 20 MP part on 1" with C-mount and USB3 keeps the chain simple:

Worked example 2: a presence check where the matched-pixel answer wins

Task: confirm a connector sits in its nest and its 0.5 mm orientation mark is present, inside an 80 mm field. The 3-pixel rule needs an object pixel of 0.5 / 3 = 0.167 mm, and 80 / 0.167 = 480 pixels across the field. Every camera in the area scan cameras range clears that bar; buying 20 MP here adds cost, diffraction pressure and depth-of-field loss without adding a detectable feature.

A 5 MP, 2/3" sensor with 3.45 µm pixels carries 2,448 pixels across. At the 80 mm field the object pixel is 80 / 2448 = 32.7 µm, so the smallest confident defect is 3 × 32.7 µm = 98 µm, five times finer than the mark requires. Detection duty only asks 30% contrast at 2/3 Nyquist: 2/3 × 145 = 97 lp/mm. Per-pixel charge scales with pitch squared, so this pixel holds (3.45 / 2.4)² = 2.1 times the charge of the fine-pixel sensor at equal exposure, and the lighting budget shrinks with it.

The matched part is a 6 MP-class 2/3" lens from the machine vision lenses range, with 35 mm of focal length framing the field at a practical working distance:

If exposure is still tight, stepping further down to the 5.86 µm class (2 MP on 1/1.2") drops Nyquist to 85 lp/mm and multiplies charge by another (5.86 / 3.45)² = 2.9, provided the lens circle covers the larger format.

Match format and circle, not just the megapixel number

Two ratings must agree: the lp/mm demand set by pitch, and the image circle set by format. A 20 MP-rated lens on a 1" circle covers a 5 MP 2/3" sensor; a 2/3"-circle 5 MP lens over a 1" sensor vignettes the corners, whatever the labels say. When the same lens moves to another sensor the field changes too, which same lens on a different sensor quantifies, and the format-to-circle map is in lens sensor format explained.

Three C-mount ratings cover the common classes: LS-FFL-0602 rated 10 MP on 1", LS-FFL-0679 rated 6 MP on 2/3" and LS-FFL-0567 rated 20 MP on 1.1":

All 3 share: Mount C-mount. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

The diffraction ceiling: set the f-number before you read MTF

Even a perfect lens blurs: diffraction spreads a point into an Airy pattern of diameter 2.44 λ N, with λ the wavelength and N the f-number. Keep that diameter under two pixels or physics, not glass, sets your resolution. Solving 2.44 λ N = 2p gives the widest useful stop:

For the finest class in white light, λ = 0.55 µm: N_max = 4.8 / (2.44 × 0.55) = 3.6, so work near f/2.8-f/3.5 and treat f/5.6 onward as soft. At 850 nm the same pixels cap at N_max = 4.8 / (2.44 × 0.85) = 2.3, which is why small-pixel NIR rigs run wide open. For 3.45 µm pixels in white light the ceiling is 6.9 / 1.342 = 5.1, so f/5.6 remains usable. A lower f-number also buys exposure headroom, the same effect shown at fixed focal length in M12 lens light gathering.

Depth of field shrinks as pixels shrink

The permissible blur circle that defines depth of field can usefully be set to the pixel itself. The first-order estimate DOF ≈ 2 × N × p × (1 + m) / m², with the blur circle equal to pixel pitch p, gives at m = 0.25 and f/2.8: for the fine-pitch sensor, 2 × 2.8 × 0.0024 × 1.25 / 0.0625 = 0.27 mm of focus latitude; for the 5.86 µm pixel, 2 × 2.8 × 0.00586 × 1.25 / 0.0625 = 0.66 mm. High resolution buys 2.4 times shallower depth at the same stop.

The same inputs feed the depth of field calculator:

For measurement duty, 0.27 mm of latitude is small. Hold the part to a plane fixture, stop toward the diffraction ceiling above, or tilt the focal plane as Scheimpflug optics explained for tilted surface imaging sets out.

Which class per application

For gauging edges, magnification constancy matters as much as raw lp/mm, which is the argument in entocentric vs telecentric lenses. At 1x the pixel pitch maps straight onto the part: a 3.45 µm pitch reads 3.45 µm edges, and the lens must still hold contrast at 145 lp/mm. One matched part for a 2/3" gauging station from the telecentric lenses range:

Checks before you order

• Compute Nyquist from the camera's actual pixel pitch, not from the megapixel label; pitch sits on the sensor line of every camera page.
• Read the lens MTF at your production f-number and at 0.7 image height; ratings assume wide open and the best field. How datasheet figures are printed is compared in geometric vs TV distortion.
• Check image circle against sensor diagonal first; the MP label second.
• Run the diffraction check N_max = 2p / (2.44 × λ) for your lighting spectrum.
• For measurement, convert distortion to millimetres at your field: 0.3% of 52.8 mm = 158 µm, then decide between a low-distortion rating and calibration.
• Bench-test the pair with a USAF 1951 target at working distance and production stop; the finest group still crisp should reach group 7 (128-228 lp/mm) in image space for the 145 lp/mm classes.

The decision, case by case

• 0.03 mm defects across 50 × 35 mm: a 20 MP, 1" mono camera with its fine pixels, a lens rated 16-25 MP on at least a 1" circle, working near f/2.8-f/3.5. That is worked example 1, with 28.8 µm of detectable defect size against a 30 µm requirement.
• 0.5 mm presence marks in 80 mm: a 5 MP, 2/3" camera at 3.45 µm with a 6 MP-class 2/3" lens; spend the difference on light, not pixels.
• Edge gauging on 2/3": a 1x telecentric with contrast at 145 lp/mm; pitch lands 1:1 on the part.
• Colour inspection at 3.45 µm: the 6-12 MP class with low chromatic aberration, checked at R, G and B separately.

Everything here is lens-side; the surrounding guides sit on the lenses topic page. To turn your own field, defect size and pitch into a parts list, start at the lens finder or the lens step of the configurator; if the budget will not close, the contact page reaches the applications team.