MTF curves for lens selection: contrast at Nyquist frequency

A lens can look superb through the middle of its curve and still deliver soft, gray edges on the parts you inspect. The fix is to read MTF curves as two numbers instead of as pictures: contrast at your sensor's Nyquist frequency, and contrast at two-thirds of it. For a 5 MP 2/3-inch camera, those two points are 145 lp/mm and 97 lp/mm. This guide gives you the formula, the contrast targets, the aperture ceiling that caps them, and a worked lens selection with the arithmetic shown.

What does an MTF curve tell you about a lens?

An MTF curve states, as a percentage, how much object contrast a lens delivers to each image-plane spatial frequency in line pairs per millimetre. It is only valid at the working distance, f-number, sensor format and wavelength printed alongside it.

A value of 65% at 100 lp/mm means a black-and-white grating of 100 line pairs per millimetre arrives at the sensor with 65% of its contrast. That percentage becomes gray-level difference between neighbouring pixels. At 10%, a 128-level edge survives as roughly a dozen levels, and edge algorithms have little left to interpolate.

Every machine vision lens datasheet draws one curve per field height — on-axis, 70% of the image circle and the corner — split into solid sagittal and dashed tangential lines, with the ideal diffraction limit plotted above them all. Contrast falls with frequency on every line; what separates lenses is how slowly. For curve anatomy in detail, see how to read lens performance curves.

Which frequency on the curve matches your camera?

Read the curve at your sensor's Nyquist frequency, one over twice the pixel pitch, and at two-thirds of it. For the pitch in the table below: 1 ÷ (2 × 0.00345 mm) = 145 lp/mm, and 0.67 × 145 ≈ 97 lp/mm as the second point.

with p the pixel pitch in millimetres. A sensor can place at most one dark and one bright line per pixel pair, so anything finer than f_N aliases into false detail.

The Nyquist limit guide for camera resolution works this out from the camera side. The same sampling bound governs systems sampled in time; the Nyquist frequency calculation for sampled control systems applies it to control loops.

How much contrast is enough at that frequency?

Demand about 30% contrast at two-thirds of Nyquist from the tangential curve at the field point where your part sits. Treat 20% at any frequency you actually need as the floor below which edges smear and decoders start failing.

Below roughly 20%, lens loss plus sensor noise leaves an edge two or three usable gray levels at best. Sub-pixel edge fitting, barcode decoding and fine defect detection then degrade together. The 30% target at two-thirds Nyquist leaves headroom for what the curve omits: Bayer interpolation, a few tens of micrometres of defocus from mounting tolerance, and unit-to-unit spread. The floor also explains failures that look mysterious. A lens can cover the format and focus crisply at centre while a corner measurement returns just 8.8% contrast at Nyquist — under the floor — and the symptom is unreadable fine detail, not a dark image.

Why do tangential and sagittal curves differ, and which one counts?

The tangential and sagittal curves measure contrast in two perpendicular directions at each field point. Aberrations such as astigmatism pull them apart away from the centre, and you should design to the lower of the two.

On-axis the two coincide. Off-axis, astigmatism and field curvature focus the two directions at slightly different depths, so a crack, pin or printed mark aligned with the weaker direction loses contrast first. Curves where T and S track each other across the whole field mark a well-corrected design; a wide gap warns that results depend on part orientation. Mark the radius where the part sits — a centred part lives at the 70% field line, not the corner. Read both curves there, and design to the lower one.

How does the f-number change the contrast you get?

Diffraction sets a hard ceiling of one over wavelength times f-number. At 0.55 µm the cutoff is 455 lp/mm at f/4 and 227 lp/mm at f/8, which caps even a perfect lens near 25% contrast at 145 lp/mm once you stop down to f/8.

At λ = 0.55 µm, the mid-visible wavelength most curves assume, the cutoffs are 455 lp/mm at f/4, 325 lp/mm at f/5.6 and 227 lp/mm at f/8. The diffraction-limited contrast at 145 lp/mm works out near 60% at f/4, 45% at f/5.6 and 25% at f/8 — a ceiling no lens can beat, computed from the ideal circular-aperture MTF.

Aberrations push the other way: wide open, measured curves sit far below the ceiling; stopped down, the ceiling itself sinks. At a 145 lp/mm Nyquist, the working aperture that preserves full contrast is f/4 to f/5.6. If depth of field forces f/8, judge the lens at 97 lp/mm instead. At magnification the working f-number becomes N × (1 + m), as the working f-number guide derives, and stopping down also cleans up aberrations, per the aberration guide.

What does a full selection look like with these numbers?

The 5 MP, 2/3-inch camera from the table, a 35 mm lens at f/4 and a 300 mm working distance gives m = f ÷ (WD - f) = 35 ÷ 265 = 0.132 and a field of view of 8.8 mm ÷ 0.132 ≈ 67 mm.

Object side: pixel pitch ÷ magnification = 3.45 ÷ 0.132 ≈ 26 µm per pixel. A 0.12 mm feature spans about 4.6 pixels and sits near 4 lp/mm on the part — far below Nyquist, where any FA lens is comfortable. The frequencies that decide this selection are the fine ones.

Restated as a spec: at least 30% contrast at 97 lp/mm and at least 20% at 145 lp/mm, tangential, at the 70% field radius, at f/4 and a 300 mm working distance.

The matching lens for this camera is a 35 mm focal length rated for 2/3-inch sensors at the 6 MP resolution class:

The class matches the pixel pitch; the printed curve at your working distance and aperture is what confirms the two readings.

Push the pitch to 2.5 µm and the two-thirds point climbs to 133 lp/mm, past what ordinary FA lenses hold at the field edge. From roughly 3 µm pixels down, shortlist lenses measured for 16-25 MP sensors. One such design is a 16 mm lens for 1.1-inch sensors with <0.3% distortion:

As pixel counts climb, the sensor format and lens coverage guide maps image circle to format.

Dimensional measurement changes the arithmetic again. At 1x magnification the sensor's 145 lp/mm maps straight to the part — pixel-sized features. At 0.166x the same sensor reaches only 24 lp/mm, a ~21 µm smallest feature. Magnification, not lens contrast, sets the small-feature limit there. That is why gauging stations run telecentric lenses at the magnification the tolerance needs; the entocentric versus telecentric comparison and the object-space pixel size calculation cover that trade. For a 2/3-inch camera at 1x, the C-mount shortlist starts:

When is a Nyquist-contrast check the wrong test?

Skip the Nyquist reading when your features are coarse. A 0.5 mm print spanning 19 pixels at 26 µm per pixel lives near 1 lp/mm, where nearly any FA lens holds 80-90% contrast. There, lens choice should follow distortion, spectral response and mounting instead.

Two more cases move the decision to lower frequencies. Sub-pixel measurement: gradient and interpolation algorithms read edges from contrast spread over several pixels, so accuracy tracks contrast at one-quarter to one-half of Nyquist — 36-72 lp/mm here — more than at 145 lp/mm. Large pixels: at 5.86 µm pitch (85 lp/mm Nyquist) most machine vision lenses clear the 20% floor, and distortion, relative illumination and telecentricity become the real differentiators. The general lens selection guide walks those criteria; the Nyquist reading earns its keep once pixels drop below about 4.5 µm.

What do you check before ordering?

Reproduce your set-up in the curve's own variables — working distance, f-number, sensor format, wavelength. Read the lower of the two curves at 97 and 145 lp/mm for this camera, keeping about 10 contrast points of margin for unit-to-unit spread.

• Match the curve's stated conditions to your set-up; a curve at f/2.8 says little about your f/4.5 working aperture.
• Read at the field radius where the part sits: centred parts at the 70% line, full-frame parts at the corner.
• Take the lower of tangential and sagittal at that radius.
• Check both frequencies: 30% at two-thirds Nyquist, 20% at Nyquist itself.
• Confirm the image circle covers the sensor diagonal — 11 mm for a 2/3-inch format.
• Compute the aperture ceiling 1 ÷ (λ × N) and place it above your Nyquist frequency.

When the numbers clear, the rest is mechanical: the machine vision lens finder filters by sensor format and resolution class, the lens configurator steps through sensor, mount and working distance, and contact takes a curve question with your camera model and working distance attached.