Vision guided positioning accuracy limits: can you hit ±2 µm

A customer asks for ±2 µm positioning in a 10 × 10 mm field of view and wants to know whether current vision technology can do it. The honest answer is that the camera alone can, but the system probably cannot — unless every error source is measured, budgeted and verified. This explainer walks through an error budget for exactly that requirement, shows the arithmetic, and tells you when to quote a 2D telecentric set-up and when to move the requirement to 3D laser metrology or renegotiate the tolerance.

The decision first

Write the tolerance in the same units you will verify it: ±0.002 mm, in a 10 mm field of view, is one part in 5,000 of the field. That single ratio tells you most of the story:

• One part in 5,000 of field of view needs roughly 5,000+ sensor pixels plus sub-pixel measurement — achievable.
• Perspective and distortion errors must stay below a few microns — needs a telecentric lens.
• The motion stage must repeat to better than about 1 µm — that is precision-metrology territory, not a standard actuator.

If all three rows are true, quote it with a documented budget. If the stage row is false — and on most production machines it is — the camera cannot rescue the spec, and the acceptance test will fail for reasons nobody can attribute cleanly. See when you need a telecentric lens for gauging for the optics half of this decision.

Pixel resolution: the first number

Start with the optical resolution, because it is the only term you can compute on paper before buying anything.

Worked example with a 12 MP sensor (4000 × 3000 pixels) on a 10 mm field of view:

• Pixel size on object = 10 mm / 4000 = 2.5 µm per pixel.
• Sub-pixel edge fit at 0.1 pixel = 0.25 µm per edge.
• A center position from two edges = about 0.35 µm (two 0.25 µm errors combined in quadrature).

So the optics contribute roughly 0.3 µm — one seventh of the ±2 µm budget. Pixel count is not the problem. Our camera resolution calculator does this arithmetic for any sensor and field of view, and choosing camera resolution for dimensional inspection covers the sensor-side trade-offs.

A global-shutter monochrome camera is the right family here: color filters halve effective resolution, and rolling shutter distorts anything that moves. The MV-CH120-11UC is the kind of 12 MP global-shutter part that fits this class of task; browse the full range in area scan cameras or the camera finder.

Perspective error: why the lens must be telecentric

With a conventional lens, magnification changes with object distance. The small-angle approximation gives the size error directly:

Example: a 10 mm feature, 0.1 mm of part height variation, at 100 mm working distance gives 10 × 0.1/100 = 10 µm of apparent size change. That alone is five times the whole tolerance — and it appears exactly when the part sits slightly tilted or on differing shims.

A telecentric lens removes this term: magnification stays constant through its depth of field, so height variation no longer masquerades as position error. This is why every micron-level gauging cell uses one. The telecentric lenses category carries bi-telecentric models like the LS-TCL-1970, and large-format bi-telecentric parts such as the LS-TCL-0885 for sensors beyond C-mount size.

Two telecentricity caveats: the depth of field is finite, so clamp the part; and telecentricity error (in the data sheet, typically a few tenths of a micron per hundred microns of height) still belongs in the budget, just at 20 to 50 times smaller than perspective error.

Distortion and calibration residuals

Lens distortion is deterministic and calibratable, but never to zero. After a good polynomial correction you should plan for:

• Residual distortion after calibration: 0.1–0.5 µm over a 10 mm field with a quality telecentric lens.
• Camera-to-world calibration error: 0.3–1 µm, dominated by how precisely you can localize the calibration target. Calibration target field of view and lens coverage margins explains why the target must fill the field.
• Thermal drift: a lens expanding 10 ppm/°C over 100 mm of barrel moves the image by about 1 µm per °C. At micron level, temperature is part of the machine.

For the residual distortion term itself, how to measure barrel distortion correction accuracy gives a measurement procedure you can run before quoting.

Lighting and edge stability

At 0.1-pixel precision, the edge you measure is a brightness transition, and lighting moves it. Budget roughly:

• Edge wander from lighting variation (intensity drift, driver ripple): 0.2–0.5 µm unless the light is strobed from a stable controller.
• Surface-finish effects: a burr or chamfer can shift the apparent edge by microns between part lots.

Collimated backlight is the strongest option for edge-defined parts — collimated backlight or diffuse backlight for edge gauging compares the two — because it projects a hard, repeatable silhouette.

The motion stage: the term that usually decides it

Vision measures where the part is. Positioning also requires the machine to go somewhere, and that axis has its own error. Combine all terms in quadrature:

With the numbers above: optical 0.35 µm, calibration 0.7 µm, distortion 0.3 µm, lighting 0.4 µm gives √(0.35² + 0.7² + 0.3² + 0.4²) ≈ 0.93 µm before the stage contributes anything. For the total to stay under 2 µm, the stage must repeat to about √(2² − 0.93²) ≈ 1.8 µm — 3-sigma, under load, at temperature.

That is the honest engineering answer to the original question: yes on paper, and only in practice if the stage is metrology-grade. The distinction between how a machine's accuracy and repeatability specs should be read is worth clarifying with the customer early — CNC positioning accuracy versus repeatability covers exactly that interpretation for motion axes.

When 2D is the wrong answer: move to 3D

Some ±2 µm requirements are better served by direct metrology than by brightness edges:

A laser profile sensor such as the MIS-DP3850-01P measures the profile itself, so height and tilt stop being error sources. Compare options in 3D sensors; check the sensor's Z repeatability against your tolerance before committing, and remember the scan-axis motion still adds in quadrature. For close-range limits on difficult surfaces, close range depth dropout on fuzzy surfaces and thin parts lists the failure modes to test for.

Checks before you quote

Run this list before putting a number in a proposal:

1. Compute pixel size on object and sub-pixel precision for the actual sensor and field of view.
2. Confirm the lens is telecentric and that the part sits within its depth of field — no shims, no float.
3. Measure residual distortion after calibration on a certified artifact, not just the target.
4. Verify edge stability across 50 frames with the production lighting, driven by a stable controller.
5. Get the stage's 3-sigma repeatability from the machine builder in writing, then subtract it in quadrature.
6. Specify the acceptance method yourself: repeatability study at fixed conditions, artifact-based, 30–50 trials.

When the usual answer is wrong

The reflex answer "more megapixels" fails here: going from 12 MP to 45 MP buys maybe 0.15 µm while the stage contributes 1.8 µm. Similarly, quoting only the vision system's number and letting the machine builder quote only the axis number guarantees a combined failure nobody owns. And if the part presentation is not repeatable to a few microns — vacuum fixture, hard stop, clean seating — no optics will save the spec.

If the budget still closes, the lens configurator steps through telecentric selection with your field of view and sensor, and free expert support with every order means we will review the error budget with you before you commit — start on the contact page.

Error-budget checklist
☐ pixel size on object ≤ 3 µm at 0.1-px fit
☐ telecentric lens, part clamped in DOF
☐ distortion residual measured < 0.5 µm
☐ calibration artifact fills the field
☐ lighting edge wander < 0.5 µm over 50 frames
☐ stage 3σ repeatability documented
☐ acceptance test defined and agreed in writing