Two cameras can both be labelled 12 megapixels and still resolve different amounts of detail. The number that decides it is not the pixel count: it is the highest spatial frequency the sensor can sample, and whether the lens still delivers contrast at that frequency. That ceiling is the Nyquist limit, and one division gives it to you. This guide runs the sampling calculation, adds the Kell factor that turns the limit into a realistic design target, works a 6.45 µm example end to end, and shows how to read a lens listing so you can tell before you buy whether the lens holds contrast where your sensor needs it.
Start from pixel pitch, not the megapixel count
A sensor samples the image at its pixel pitch. Divide one millimetre by that pitch and you get the sampling frequency: the number of independent samples the sensor lays down per millimetre of image, in pixels per millimetre. Pixel count alone hides this, because a 12 MP sensor on a 1/1.7" format samples far finer than a 12 MP sensor on a 1" format — the same pixels squeezed into a smaller area. When you are sizing pixels against a field of view, the resolution calculator below does that part; everything after it is about how fine the sampling must be.
The pitches you will meet across the area scan cameras and board-level cameras catalogs fall into a short list, and every number this guide uses comes from it:
| Pixel pitch | Sampling (px/mm) | Nyquist limit (lp/mm) | 0.7 design target (lp/mm) |
|---|---|---|---|
| 6.45 µm | 155 | 77.5 | 54 |
| 3.45 µm | 290 | 145 | 101 |
| 2.4 µm | 417 | 208 | 146 |
| 1.6 µm | 625 | 312.5 | 219 |
Keep the table in reach: every matching decision below reads one of its columns.
The Nyquist limit: half the sampling frequency
A line pair is one dark bar plus one bright bar — a full cycle of a test pattern — and reproducing one cycle needs at least two pixels, one for the dark bar and one for the bright one. That two-pixel minimum is the whole rule:
For the 155 pixels/mm sensor in the table, the limit is 77.5 lp/mm. Finer gratings cannot be represented, and they do not quietly disappear: they fold back into the image as false, coarser patterns. That folding is aliasing, and it is why the limit is a design input rather than a suggestion.
The Kell factor: design to about 70 percent of the limit
Right at the Nyquist limit, whether a grating registers depends on phase: aligned pixel-on, pixel-off, it is captured; shifted half a pixel, it blurs toward uniform grey. Real sensors also lose contrast to light leaking between pixels and to the gap between photodiode and full pitch. The Kell factor rolls both effects into one number, about 0.7: measured contrast holds up regardless of phase at 0.33 cycles per pixel — 0.66 of the limit — with only minor artifacts. The Kell factor reference collects the measured values and the history.
That scaled number, not the raw limit, is what you hand to the lens: 54 lp/mm at 6.45 µm, 101 lp/mm at 3.45 µm, 219 lp/mm at 1.6 µm.
Worked example: matching a lens to a 6.45 µm pixel sensor
The case: a monochrome, 2/3-inch sensor with 6.45 µm square pixels — the pitch made common by the 1.4 MP CCD generation — run at f/4 in green light. The inputs: pixel pitch 0.00645 mm; Kell factor 0.7, the practical measured value; image circle 11 mm, the 2/3-inch diagonal; 550 nm and f/4 for the diffraction cross-check.
The rule, in words: the sensor represents cycles only up to half its sampling rate; size the lens for 70 percent of that, and confirm the aperture's diffraction cutoff sits far above.
The arithmetic:
- Sampling frequency: 1 ÷ 0.00645 mm = 155 pixels/mm
- Nyquist limit: 155 ÷ 2 = 77.5 lp/mm
- Design target: 77.5 × 0.7 = 54.25 — call it 54 lp/mm
- Diffraction cutoff at f/4: 1 ÷ (0.00055 mm × 4) = 455 lp/mm, about six times the Nyquist limit
The result: the sensor tops out at 77.5 lp/mm, the lens has to hold contrast at 54 lp/mm, and diffraction is out of play at f/4 — so aberration correction in the lens, not physics, decides the detail.
The decision: buy the lens for 54 lp/mm on an 11 mm circle. In catalog terms that is a 2/3-inch lens from the 5 MP band upward, corrected for pixels near 3.45 µm with a design target near 101 lp/mm — nearly a factor of two in hand. A 2/3-inch band sized for 6 µm-class pixels aims at your target with nothing in reserve, so treat it as marginal. Keep the aperture at f/11 or wider: f/11 leaves the cutoff at 165 lp/mm, three times the target, while f/22 drags it to 83 lp/mm and starts erasing the very frequencies the sensor can still record.
Reading lens listings: circle, band and MTF
A lens listing carries three resolution facts: the image circle it covers, the MP band it was corrected for, and its MTF curves. The MP band is shorthand for a pixel pitch on the stated format: a lens marked 5 MP for 2/3-inch targets pixels near 3.45 µm — a 145 lp/mm Nyquist limit and a 101 lp/mm design target. Convert the band with the table above and compare it with your sensor's target; that one comparison replaces the megapixel folklore.
The circle must cover the sensor: a 1.1-inch lens covers a 1-inch sensor with room to spare, while a 2/3-inch lens on a 1-inch sensor crops the corners. The formats and their diagonals are laid out in lens sensor format: 1/1.8 inch, 2/3 inch and 1 inch explained, and what "contrast at a frequency" means along the curve is the subject of lens contrast and resolution.
Both checks run as filters in the machine vision lenses catalog, by sensor format and by resolution band.
A camera that needs exactly this treatment: a 1" sensor at 20 MP.
- Sensor format
- 1"
- Lens mount
- C-mount
- Interface
- USB3
- Resolution
- 16-25 MP
The MV-CS200-10UM is the mono variant, which keeps the whole sampling budget for luminance instead of spreading it across a Bayer mosaic.
Two catalog lenses show how the bands read in practice: the LS-FFL-0602 is an 8.5 mm lens on a 1" format at 10 MP, and the LS-FFL-0567 is a 16 mm lens on a 1.1" format at 20 MP with F2.8-22 and <0.3% distortion.
| Specification | LS-FFL-0602 | LS-FFL-0567 |
|---|---|---|
| Resolution | 10 Megapixel | 20 Megapixel |
| Format | 1" | 1.1" |
| Focal length | 8.5 mm | 16 mm |
| Aperture | F2.4 Manual iris | F2.8-22 |
| Filter thread | M77 x P0.75 | M35.5 x P0.5 |
| Price | $429.00 | $679.00 |
All 2 share: M.O.D. 0.1 m; Mount C-mount. Values as printed in each product's data sheet; open a part for its drawing and full specifications.
Both bands clear the worked example's 54 lp/mm target by construction; the 20 MP band is the one that also carries a 3.45 µm-class sensor and its 101 lp/mm target.
When pixels outrun the lens: the 1.6 µm case
Small pixels take this to the other extreme, and the catalog prints one: 1.6 µm × 1.6 µm pixels on a 1/2" format at 12 MP.
- Sensor format
- 1/2"
- Lens mount
- C-mount
- Interface
- GigE
- Resolution
- 12 MP
The LS-BCM-0056 is exactly that part. At its pitch the arithmetic gives 1 ÷ 0.0016 mm = 625 pixels/mm of sampling, a 312.5 lp/mm Nyquist limit and a 219 lp/mm design target. Diffraction is not what stops you — at f/2.8 the cutoff sits at 649 lp/mm — but holding 219 lp/mm edge to edge across an 8 mm diagonal is a correction job that separates 12 MP-rated optics from everything below.
Past roughly 2 µm of pitch, the sensor stops being the bottleneck and the lens takes over: pixels without the matching band add sampling capacity the optics cannot fill. That trade, and the cases where fine pixels still pay, is worked through in small pixel camera selection; where the aperture does become the ceiling — fine pixels stopped down hard — the same 1 ÷ (λ × f-number) arithmetic is in diffraction limit and aperture effects.
Aliasing: what breaking the limit costs
Take the worked example and image a 100 lp/mm grating on it. Sampling runs at 155 pixels/mm, so the 100 cycles/mm pattern cannot be represented: it lands at 155 − 100 = 55 lp/mm, in the middle of the usable band, as moiré. On real parts that is false pitch on threads, shimmer on meshes and screens, phantom stripes on periodic printing. The false pattern and the true one share the same pixels, so nothing downstream can separate them.
Each defence is physical, applied before the exposure:
- Oversample: size the smallest feature you must judge at 3-5 pixels across. The sizing method is worked out in choosing camera resolution for dimensional inspection.
- Angle the camera a few degrees off any periodic pattern, so the beat pattern slides away from the features you judge.
- As a last resort, trade a little sharpness for safety with a deliberate low-pass in front of the sensor: slight defocus or an optical low-pass filter. Never a filter after capture.
When the fix is more pixels, the full trade — bandwidth, optics, light — is weighed in high resolution machine vision cameras: how many pixels.
Checks before you order
Five checks settle the pairing:
- Compute the target: 0.35 ÷ pixel pitch in millimetres is the lp/mm the lens must hold with contrast — 54 lp/mm at 6.45 µm.
- Check coverage: the lens image circle must equal or exceed the sensor's optical format.
- Check the band: convert the lens MP rating to its design pitch on that format and confirm its target beats yours.
- Check the aperture: keep the diffraction cutoff, 1 ÷ (λ × f-number), at least three times the target — about f/11 for the 6.45 µm case and f/5.6 for 3.45 µm pixels, at 550 nm.
- Decide who limits: target well below the band's ceiling means the sensor is the limiter; target at the ceiling means the lens caps the detail; pitch below about 2 µm means buy camera and lens as one set.
The decisions, case by case:
- 6.45 µm, 2/3-inch sensor: a 5 MP-band or better 2/3-inch lens, used between f/4 and f/11. The sensor is the limiter, and the image is good to its 77.5 lp/mm ceiling.
- 3.45 µm-class sensor: a band corrected for that pitch or finer on the same format, kept at f/8 or wider so diffraction still clears the 145 lp/mm limit.
- 1.6 µm-class sensor: choose camera and lens as a set from the 12 MP band. The pitch promises 312.5 lp/mm; the optics deliver less, and the lens band is the number to trust.
If you would rather start from a shortlist, the camera finder and lens finder carry format and resolution band as filters, and the system configurator at the lens step pairs coverage and band for you. For a specific pairing, contact us with the pixel pitch, the format and the working distance — that is enough to answer it. The rest of the sensor-side reading sits on the cameras topic page.
Questions
What is a line pair in machine vision?
One line pair is one dark bar and one bright bar — a single cycle of a test pattern. Resolution quoted in line pairs per millimetre counts cycles, not bars, and each cycle needs at least two pixels: one to catch the dark bar, one to catch the bright one. That two-pixel minimum is exactly why the Nyquist limit sits at half the sampling frequency.
How do I convert a lens megapixel rating to lp/mm?
Work backwards from the pixel pitch the rating assumes on its stated format. A 5 MP 2/3-inch lens is built for pixels near 3.45 µm: 1 ÷ 0.00345 mm = 290 pixels/mm, so it should hold contrast up to a Nyquist limit near 145 lp/mm and a 0.7 Kell target near 101 lp/mm. Compare that number with your own sensor's target.
Does a colour sensor resolve as much detail as mono at the same pitch?
Usually less. A Bayer colour sensor records only one colour per pixel and the camera reconstructs the rest, so fine detail falling between same-colour pixels is interpolated. Mono pixels all sample luminance, so mono uses the full sampling budget without a demosaicing step. If your detail is carried by contrast rather than colour, mono holds detail to a finer pitch.
Can software recover detail above the Nyquist limit?
Not for arbitrary content. Once frequencies above the limit are sampled, they fold into lower frequencies and cannot be separated afterwards. Edges and other geometrically simple shapes can be located to sub-pixel precision statistically, and multi-frame super-resolution needs several shifted exposures of one static scene. Neither recovers real detail from a single aliased frame.
What f-number keeps diffraction off my Nyquist limit?
Keep the diffraction cutoff — 1 ÷ (λ × f-number) — comfortably above the frequencies you need. At 550 nm the cutoff is 227 lp/mm at f/8 and 165 lp/mm at f/11, both above the 77.5 lp/mm limit of a 6.45 µm sensor. For 3.45 µm pixels with a 145 lp/mm limit, f/8 still clears it, while f/16 drops the cutoff to 114 lp/mm and clips the ceiling.