Lens resolution vs pixel size: matching MTF to your sensor

Pick a lens from the sensor outward, not from the megapixel sticker. Two numbers decide whether the pairing works: the spatial frequency, in line pairs per millimetre, that the lens must pass for your pixel pitch, and the image circle that has to cover the sensor diagonal. This guide turns both into formulas you can compute in a minute, works them through a 5 MP 2/3" camera, names the MTF value to look for on the datasheet, and shows the diffraction ceiling your f-number sets.

The two numbers that decide the match

A lens and sensor work together only when two conditions hold at once.

Resolution: the lens must deliver usable contrast at frequencies up to 500 divided by the pixel pitch in microns — the sensor's Nyquist frequency — times a margin of 1.2 for inspection or 1.4 for measurement. Coverage: the image circle must be at least the sensor diagonal. Miss either and no software recovers the loss; an under-resolved lens blurs the detail before the pixel samples it, and an undersized circle darkens the corners of the frame. Focal length, mount and iris then frame and expose the image, but these two numbers decide whether it carries usable detail at all.

What lp/mm and MTF actually tell you

A line pair is one black bar plus one white bar. Resolution in line pairs per millimetre (lp/mm) counts how many such pairs fit across one millimetre at the sensor plane; higher frequency means finer detail. The modulation transfer function, MTF, is the honest half of the spec: how much contrast survives at each frequency, from image centre to corner.

"Resolves 150 lp/mm" on its own is half a statement. A lens can pass a frequency while keeping 3% contrast, which no algorithm can work with. The lens contrast and resolution guide walks through the full curves; for matching, you need one value from them: the contrast at your Nyquist frequency.

From pixel size to the lp/mm you need

The constant 500 falls out of sampling: a line pair needs two pixels, one for the black bar and one for the white bar, and a millimetre holds 1,000 µm. A 3.45 µm pixel therefore samples up to 500 / 3.45 ≈ 145 lp/mm; a 7 µm pixel only 71 lp/mm. The Nyquist limit and camera resolution guide gives the sampling argument in full.

The margins are not gold-plating. A datasheet MTF applies at its quoted f-number, wavelength and field point, and a real station adds focus error, cover glass and alignment tolerance on top. Twenty percent covers inspection; forty percent is the habit for gauging, where edge repeatability feeds tolerance decisions.

Contrast target: ask for at least 30% lens MTF at Nyquist for detection and 50% or better for gauging. The basis is the pixel itself. A square photodiode passes about 64% contrast at its own Nyquist frequency — the sinc response of its aperture — so a 30% lens leaves roughly 0.64 × 0.30 ≈ 19% system contrast, enough for repeatable sub-pixel edges. Below that, lighting drift and focus error consume the rest.

Worked example: a 5 MP 2/3" camera

Take a common camera class as the example inputs — a 5 MP sensor in 2/3" format with 2448 × 2048 pixels; your datasheet's values replace these. Pixel pitch is sensor width divided by pixels across:

Here: 8.45 mm / 2448 = 3.45 µm. The area route lands in the same place: 59.7 mm² / 5.01 million pixels ≈ 11.9 µm² per pixel, and √11.9 ≈ 3.45 µm — the formula needs pitch, not area.

• Nyquist: 500 / 3.45 µm ≈ 145 lp/mm
• Inspection target: 145 × 1.2 ≈ 174 lp/mm
• Gauging target: 145 × 1.4 ≈ 203 lp/mm

Coverage: diagonal = √(8.45² + 7.07²) = √(71.4 + 50.0) ≈ 11.0 mm, so any lens marked for 2/3" format — the 8-11 mm image circle class — covers the frame.

On this camera the right grade is a 6 MP-class 2/3" lens such as the LS-FFL-0679, a 35 mm C-mount design.

Sharpness and framing are separate decisions. At a 200 mm working distance, a 35 mm lens frames roughly 8.45 × 200 / 35 ≈ 48 mm across; if the part needs a different field, change focal length or distance, not the resolution grade.

Match the image circle to the sensor diagonal

Coverage fails more quietly than resolution. A lens projects a disc of usable light; beyond it the image falls to black, and just inside it, brightness rolls off toward the edge even on a good design. The rule is one comparison — image circle ≥ sensor diagonal:

Overshooting wastes money and barrel length: a 16-22 mm circle on a 2/3" sensor illuminates pixels that do not exist. Undershooting shows up as dark corners, and on measurement optics as magnification error at the edges, which is why telecentric lenses are specified with their circles so strictly. The lens sensor format guide decodes the inch names, and same lens on a different sensor: how HFOV changes covers moving one lens between two cameras.

Pixel pitch sets the lp/mm requirement; the format sets the circle; grade labels track the pitch across the catalogue:

All 3 share: Mount C-mount. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

The pitch class, and with it the 145 lp/mm requirement, is the same for all three. What changes is the circle: the LS-FFL-0679 for 2/3" (8-11 mm), the LS-FFL-0602 for 1" (11-16 mm), the LS-FFL-0567 for 1.1" (16-22 mm). One optics problem, three sensor sizes.

The diffraction ceiling: f-number sets the limit

One more boundary comes from physics rather than any catalogue. Stop a lens down to f-number N and diffraction at the iris blocks everything above roughly 1800/N lp/mm at visible wavelengths — the cutoff 1/(λN), with λ ≈ 550 nm. Even a perfect lens cannot beat it; real lenses stop short.

Scored against the 174 lp/mm target from the worked example:

The contrast column is the ideal circular-aperture figure — a ceiling, not a promise. For 3.45 µm pixels, f/5.6 is the practical stop-down limit and f/4 the comfortable setting; at f/8 the perfect-lens contrast has already collapsed to about 12% before aberration. A depth-of-field demand that pushes past f/8 argues for larger pixels, not a higher resolution grade; the small pixel camera selection guide works that trade.

Where the 500/p rule misleads

Oversampled sensors. A 1.6 µm pixel demands 500 / 1.6 ≈ 313 lp/mm before any margin. Past roughly 200 lp/mm, aberration and diffraction — not the pixel — set the limit: the sensor samples finer than a practical lens delivers, so the extra pixels interpolate rather than resolve. Buy those cameras for oversampling and noise smoothing, not for the pixel count on the label.

Large pixels. A 7 µm pixel needs 71 lp/mm, about 86 with margin. Nearly every current factory-automation lens clears that with contrast to spare, so the specs that matter become circle, distortion, working distance and mount. A 25 MP-grade optic on such a sensor buys contrast the pixels cannot represent; spend the difference on lighting or sensor size.

The megapixel ticket is a proxy. Grades such as 5 MP or 20 MP bundle pitch, format and a reference field point into one label. Where corner sharpness decides pass or fail, read the MTF curve at half to four-fifths of the field height and at your working f-number, then confirm on a resolution target under the production light before committing the line. The geometric vs TV distortion guide covers the other datasheet trap.

Measurement systems. For dimensional gauging, MTF is not even the first spec: magnification accuracy, telecentricity and distortion set the error budget before contrast enters. A 1x telecentric such as the LS-TCL-2118 — a 2/3" C-mount design with 108 mm working distance — holds magnification constant across part-depth variation. Once magnification is fixed, the object space pixel size guide converts it into microns per pixel on the part.

Two telecentrics that fit small-part gauging at small pixel pitches:

On shiny or specular parts, the LS-TCL-2194 has the coaxial light port built in (0.5x, 1", 110 mm working distance), so illumination travels down the optical axis rather than through a beam splitter hung in front of the lens.

Checks before you order

Run the list against the two datasheets, in order:

• Pixel pitch, from the camera datasheet (or area ÷ pixel count, square-rooted).
• The target, written down: Nyquist = 500 / pitch, then × 1.2 or × 1.4 — "must hold ≥ 174 lp/mm near the field edge" for the worked example.
• Circle: image circle ≥ sensor diagonal, from the table above.
• Iris: 900 / target gives the widest f-number that keeps the target at half the diffraction cutoff — 900 / 174 ≈ f/5.2 here, so run f/4 and treat f/5.6 as the edge.
• MTF at your frequency and field height, not the centre-field headline.
• A validation image: resolution target first, then the real part under the production light, before the purchase order.

The decision, per case

• 5 MP 2/3" inspection camera, 3.45 µm pixels: about 174 lp/mm across an 8-11 mm circle, running f/4-f/5.6. The LS-FFL-0679 is that ticket.
• 1.1" gauging camera, 3.45 µm pixels: raise the target to about 203 lp/mm and insist on corner MTF. Buy inside the 16-25 MP grade; the LS-FFL-0567 sits in it.
• Dimensional gauging on small parts: a 1x telecentric such as the LS-TCL-2118 so magnification stops moving with part depth, with pitch and circle checked as above.
• 7 µm-pixel sensor: the resolution grade stops mattering at 71-86 lp/mm; buy on circle, distortion and lighting fit instead.

Set the sensor side first, then filter the lens finder by format and resolution grade, or step through the lens configurator with the camera and working distance at hand. The machine vision lenses category holds the full range, the lenses topic page gathers the guides behind every rule above, and machine vision lens selection for reliable inspection is the broadest of them. If the numbers will not close, bring the camera datasheet to the contact page and we will run the match with you.