AprilTag distance off by a constant ratio: find the scale error

You calibrated the camera, the tag node locks on every frame, and every distance it reports is the same fraction of the truth: 0.83 m where the tape measure says 1.00 m, 1.66 m at a taped 2.00 m. Swap the lens and the error disappears, or appears. That pattern, a fixed ratio at every range, is not noise and not a detection problem. It is a scale error, and only two numbers can produce it: the focal length your calibration reports and the tag size in your config. Here is the ten-minute measurement that separates them, the fix for each, and the hardware that keeps the distortion coefficients small enough that this never comes back.

A constant ratio means scale, not noise

A tag solver gets depth from apparent size. In a pinhole camera the width of the tag on the sensor follows w = fx · s / Z, where fx is the focal length in pixels, s the physical tag size and Z the distance. The solver inverts that relation, so the distance it hands you carries the ratio:

Both suspects enter as one product. A wrong tag size and a wrong focal length are indistinguishable from the distances alone, which is why some of your lenses look fine and others do not: the error shows up only when one of the two is off. Noise scatters around the truth, pose ambiguity flickers between frames, and an error that grows with range points at frames and extrinsics instead — that failure mode has its own check of calibration and frames. A fixed multiplier means scale. What fx means physically, and why the sensor sets it, is covered in why sensor size and focal length set the field of view.

The ruler test: measure the focal length you actually have

You need a bar of known length, a tape measure and five minutes.

1. Fix a straight bar — a steel rule of 300 mm is ideal — to a wall, square to the camera.
2. Measure the distance L from the lens to the bar with the tape; about 1.0 m keeps the bar large in the frame.
3. Capture the image your tag node actually consumes. If the pipeline undistorts before detection, measure on the undistorted image; if it detects on the raw image, measure there.
4. Read the bar's length in pixels, w, marking both ends as precisely as the viewer allows.
5. Compute the focal length the camera is really delivering:\n

6. Compare fx_implied with the fx printed in your camera_matrix. Agreement within about 2 % clears the calibration and convicts the tag size; a gap equal to your distance error convicts the calibration.

The bar obeys the same similar-triangle law the tag does, so this test reads the scale directly, with no dependence on the tag, the family or the detector. Measuring barrel distortion correction accuracy applies the same idea to a whole calibration.

Worked example: 1920 × 1080 camera reading 0.83 of the taped distance

The inputs, each with its source:

• Camera matrix from the calibration yaml: fx = 1440 px, fy = 1431 px, cx = 947 px, cy = 536 px, image 1920 × 1080 px.
• Distortion coefficients from the same file: k1 = -0.389, k2 = 0.166, model plumb_bob, five coefficients.
• A taped distance of 1.00 m at which the tag node reports 0.83 m.
• A steel rule, s_bar = 300 mm, fixed at L = 1000 mm; its width in the undistorted image measures w = 518 px.

The rule in words: the bar's pixel width equals the true focal length times the bar's size over its distance. The arithmetic:

The calibration's focal length is 17 % short of what the camera delivers, and the predicted distance error, 0.83, is exactly what the tag node reports. Decision: a gap beyond 5 % convicts the calibration, so recalibrate and leave the tag size alone. Two side checks support the verdict: the principal point sits 13 px and 4 px from the frame centre, so the geometry is otherwise sane, and k1 = -0.389 is a large radial term for a lens whose images look straight to the eye.

Cause 1: the tag size is not the black square

The size the solver wants is the side length of the black square — not the white border, not the outer edge of the print. Measure it with calipers, in metres, and put that number in the config. Print scaling is the classic source: a fit-to-page print shifts the size by a few percent, and a custom family such as 48h12 follows the same definition as the standard families. When the ruler test clears the focal length, the numbers close quickly: a config value of 0.065 m against a black square that measures 0.078 m is a ratio of 1.20 — your exact error — and the fix is to enter 0.078.

Cause 2: five coefficients cannot model that lens

The plumb_bob model carries five numbers: radial terms k1 and k2, tangential terms p1 and p2, and a third radial term the basic fit often leaves near zero. It is a good model for a lens with modest barrel. At k1 near -0.4 it is working hard: your eye tolerates smooth barrel distortion, the solver does not, and when the polynomial runs out of terms the residual trades directly against the focal length. The result is an fx that is believable, stable and wrong, which multiplies every distance by the same factor.

The fix is a recalibration with the rational model. The stock mono calibrator, cameracalibrator.py, fits the five-coefficient form; for the extra terms, call cv::calibrateCamera with CALIB_RATIONAL_MODEL, which enables the radial coefficients k4, k5 and k6, then write the eight values into the yaml as rational_polynomial. The pinhole camera model in image_geometry accepts both plumb_bob and rational_polynomial, so the rest of the pipeline needs no change. Collect the frames the richer model needs: pattern tilted about 30° from parallel, pushed into all four corners of the frame, a few dozen captures. For wider glass, an equidistant model fits better than any polynomial.

Distortion correction for pixel-to-mm measurements covers what the correction does to your measurements, and which distortion setting to pick in a calibration covers the model choice itself.

Cause 3: the matrix does not match the image

A mono calibration file prints both camera_matrix and projection_matrix. If images are rectified somewhere in the pipeline, their geometry follows the projection matrix, and its fx can differ from the camera matrix's. Detect a tag on the rectified image while solving pose with the raw camera matrix, or the reverse, and every distance scales by the ratio of the two fx values, constant at all ranges. For a pipeline that detects on raw images the two should agree; a large gap between them is itself the bug. Open both blocks, diff the fx entries, and treat a difference beyond 1 % as a routing problem: find which node undistorts, and give the detector the image that belongs to the matrix it is handed.

Should you rescale the tag size until the numbers look right?

Only after the ruler test, and then it is not a rescale but the fix: if fx is right and the print measures 0.078 m against a configured 0.065 m, entering 0.078 is simply correct. Forcing a biased focal length to read right through the tag size is different. Depth becomes true, but lateral coordinates stay scaled, because X and Y come from the same fx — a robot guided to the tag lands short or long by that factor even though its distance display is now perfect. The fudge also breaks silently the day you recalibrate. Fix the number that is wrong; the ruler test tells you which.

Hardware that keeps the coefficients small

The cheapest prevention is a lens the five-coefficient model can actually fit: low distortion, so k1 stays small and fx has nothing to trade against. For board cameras, M12 board lenses in low-distortion designs cover the short focal lengths:

On a C-mount camera, pick from machine vision lenses with a printed distortion spec — the LS-FFL-0567 carries <0.3 % distortion:

A camera with a documented sensor and a rigid mount holds the calibration you worked for. Area scan cameras with a C-mount cover this range, and tag work on a robot has its own guide to camera selection for AprilTag pose accuracy:

When the replacement lens changes the field of view, fx moves with it: 90° horizontal over a 1920 px-wide frame puts fx at 960/tan(45°) = 960 px, and 60° puts it at 1663 px.

Checks before you trust the corrected numbers

• Re-run the ruler test after the fix; fx_implied and the calibrated fx should agree within 2 %.
• Sweep the tag from 0.3 m to 3 m in 0.5 m steps and plot reported over taped distance; the slope should sit at 1.00 within ±3 %.
• Confirm the principal point lands within a few tens of pixels of the frame centre.
• For a low-distiction lens only, check hfov = 2 × atan((W/2)/fx) against the data sheet within about 5°; an advertised figure for a wide, distorted lens already includes the distortion, so that comparison is meaningless there.
• Keep enough tag pixels at your working distance — tiny tags fail for their own reasons.

When the diagnosis points at the glass, the shortest path is the configurator started at the lens step with your field of view and working distance. If the numbers still refuse to close, send the calibration file and the ruler-test result through the contact page and we will work it with you.