Vision system static repeatability: how to measure it

A vision-guided cell picks only as well as the pose the camera hands the robot. Before you blame the robot, the gripper or the hand-eye calibration, measure what the vision system itself contributes: trigger the same stationary part many times and watch how far the returned position and angle wander. That wander is the system's static repeatability. The test needs fifteen minutes, a clamped part and a log file, and it leaves you with a standard deviation in millimetres per axis and a clear pass or fail against your pick tolerance. The worked example below shows the full arithmetic on ten readings.

The number to aim for

Static repeatability is the scatter of the pose your vision system returns when nothing in the scene moves. Trigger the same clamped part 30 times or more, and each axis — x, y, angle — forms a distribution; one sample standard deviation per axis is the number you quote. Accuracy is a different quantity: the offset between the mean reading and the true position, which a calibration removes. Scatter cannot be calibrated out, so it has to be small by design.

The pass gate comes from the 10 to 1 rule in vision metrology: the vision system may consume at most one tenth of the tolerance it serves. For a ±0.2 mm pick tolerance that is σ ≤ 0.02 mm per axis; for a ±0.5° orientation tolerance, σ ≤ 0.05°. Judge the worst axis, not the average.

What you need

• A camera with full manual control: manual exposure, manual gain, fixed focus. Every auto function off, because auto-exposure re-balances images mid-run and masquerades as pose noise.
• A rigid mount. A stand that breathes under cable tension adds scatter no software can remove; the camera stands range exists for exactly this.
• A fixed target: the production part bolted to a plate, at the position and in the plane where picks happen.
• The production light at the production brightness and duty, then untouched. Light changes move detected edges before any code runs — the mechanism behind fiducial pose drift over time.
• Periodic triggering and logging: one row per trigger with x, y, angle and timestamp, plus the exposure and gain readback if your software exposes it.

A compact global-shutter GigE camera and a slide-table stand cover most benches for this test.

Run the test

1. Clamp the part to a rigid plate at the production pose and plane, and hand-check that it cannot rock. Capture two frames ten seconds apart and difference them: any nonzero pixels mean the part, plate or camera moved, and that motion will swallow the σ you are about to measure.
2. Lock the camera down: manual exposure, manual gain, fixed focus, auto functions off. If the lens focuses by motor, focus once and switch to manual — the same discipline as when you calibrate a camera with autofocus.
3. Set the light to the production duty and brightness and do not touch it again during the run.
4. Trigger 30 to 50 times at one to two second intervals, robot parked in its pick pose, cell otherwise idle. Log one row per trigger.
5. Compute the mean and the sample standard deviation of each column, in millimetres and degrees as the robot consumes them (formula below).
6. Compare each axis with your gate and record the verdict together with the run's exposure, gain and light settings. If you move the camera afterwards, the number is void — moved parts drift off their calibration, and a repeatability result expires the same way.

The formula

Divide by n - 1 because the run is a sample of production noise, not the whole population. If your software reports the population standard deviation instead, the two results differ by a factor of √(30/29) ≈ 1.017 at n = 30 — under 2 %, too little to flip a pass or fail. Convert pixels to millimetres with your calibrated scale; σ scales linearly with it, so convert before or after, whichever your tools prefer. Quote every axis and let the worst one speak for the system.

Worked example: x-axis scatter for a ±0.2 mm pick tolerance

Inputs: the cell must hold ±0.2 mm, so the gate is σ ≤ 0.02 mm, one tenth of the tolerance. The vision system is calibrated at the part plane and returns x in millimetres, so no pixel conversion is needed. The ten readings below are example data from a periodic-trigger run on a clamped bracket; your run substitutes its own log.

Arithmetic: mean xbar = 1500.045 mm ÷ 10 = 150.0045 mm. Sum of squared deviations = 0.0007465 mm². Variance = 0.0007465 mm² ÷ (10 - 1) = 0.0000829 mm². Standard deviation s = √0.0000829 mm² = 0.0091 mm.

Results: σx = 0.0091 mm, or 9.1 µm. Peak-to-peak = 150.020 - 149.991 = 0.029 mm. Quoted as ±3σ that is ±0.027 mm. The mean sits 150.0045 - 150.000 = +0.0045 mm from nominal — an offset for calibration to remove, not scatter, so it is not charged against the σ gate.

Decision: 0.0091 mm is under half the 0.02 mm gate and under 5 % of the ±0.2 mm tolerance (0.0091 ÷ 0.2 = 0.0455). The vision system passes with margin, and even the ±3σ quote leaves the tolerance budget dominated by the robot and the gripper. Had σ come out above 0.02 mm, work through the next section in order.

If σ fails, check these in order

1. Auto functions. Read the exposure and gain columns of your log: they must be identical on every row. Any variation means the camera re-balanced mid-run — go full manual and rerun.
2. Light drive. Rerun with the light on a plain DC supply at the same brightness. A large drop in σ convicts the PWM driver, the mains supply or a flickering emitter rather than the optics or the code.
3. Mount creep. Difference two raw frames captured a minute apart: gray-level changes away from the part mean the camera or the stand moved. Retorque every joint, strain-relieve the cable so it cannot pull the head, and rerun.
4. Starved pose estimator. When σ of the angle is much larger than σ of position, the edges or features lack contrast and subpixel estimation is guessing. Aim the light for edge definition, lengthen the exposure, or average three to five frames per pose as described in averaging measurement values across repeated scene runs.
5. Drift, not noise. Plot the reading against trigger index: a slope means the mount or the head is still settling. Warm the bench up for fifteen minutes before measuring and keep HVAC drafts off it.

Checks before you trust the number

• Run the whole test twice. At n = 30 the σ estimate itself moves about ±13 % between runs (one standard deviation on normal data), so two results within roughly 25 % of each other describe one system; a wider gap means one run caught a disturbance, and you must find it before quoting anything.
• Report the worst axis. x and y can pass while the angle fails, and the robot consumes all three.
• Measure re-placement separately. Re-clamp the part five to ten times and take the σ of the returned pose across placements; fixture repeatability usually exceeds the static σ and both land in the pick error. For a reference plate, a manual stage repositions the target between runs without hand placement.

• Audit the log. Timestamp gaps mean dropped frames, and a frame your code filled by repeating the previous pose fakes a better σ than the hardware delivered. Row count must equal trigger count.

What the static number does not cover

A bench pass does not clear the cell. Static σ is sensing noise at one position; production adds errors this test never sees. Hand-eye calibration maps the whole field, not just its centre, so repeat the trigger run with the target parked at five to nine positions across the field of view; offset and σ both grow toward the edges on most lenses.

With the camera on the robot, robot repeatability joins the chain and the combined scatter is what the pick feels. Moving parts add motion blur and rolling-shutter skew that no static run measures. Over weeks, light aging shifts edges slowly — a bias, not noise — which calls for a periodic recheck, not a one-time gate.

The decision path: qualify the sensor with the static test before integration, then qualify the cell with parts presented normally and count pick errors over a full shift. Scatter that appears only once parts rotate in the gripper has its own diagnosis — see vision pick error under rotation.

Set up your bench

Choose the camera from the area scan cameras range or the camera finder, bolt it to a stand that cannot move, and run the 30 to 50 triggers before writing any pick code. The stand and camera configurator walks stand, camera and light in one pass, more system-level procedures sit on the systems topic page, and for help sizing a specific cell, contact us with your tolerance and cycle time.