Camera selection for AprilTag pose accuracy on a robot

A docking robot and a picking arm both navigate by AprilTag, and both miss when the pose estimate wobbles by millimetres. Most of that wobble is decided by hardware before any software runs: pixels on the tag, shutter type, lens distortion and mechanical stability. This guide gives the formulas and worked numbers to choose a camera and lens for tag pose — how many pixels the tag needs, which focal length holds them at your docking distance, why a moving robot needs a global shutter, and where the pixel spend stops paying. The two cases from the shop floor, auto-docking and arm-guided grasping, each close with a concrete build.

How many pixels does an AprilTag need on the sensor?

Size the tag's black border to 100-200 sensor pixels: below roughly 80 pixels the detector starts dropping bit cells, and above 200 the extra resolution adds processing load without moving the pose estimate.

A 36h11 tag packs an 8 × 8 grid of data bits inside a one-cell white border and a one-cell black border — ten cells across. Detectors need about eight pixels per cell to resolve that grid, which is the 80-pixel floor; the 100-200 band adds margin for blur, glare and print wear. Published testing places AprilTag orientation error within about 2° even at off-axis angles up to 90°, so angle is rarely the bottleneck — pixels are.

Pixels convert to position directly. Millimetres-per-pixel at the tag equals tag width divided by pixels across it, and the corner solver refines tag corners to roughly a tenth of a pixel:

• 100 mm tag at 200 pixels → 0.5 mm per pixel → image-derived noise near 0.05 mm
• 100 mm tag at 100 pixels → 1 mm per pixel → near 0.1 mm

Calibration residual, lens distortion and robot mechanics stack on top of that floor; the distortion section below puts numbers on them. This is the same pixels-per-feature budget as sizing camera resolution for defect detection. Tag families and placement conventions for mobile docking are written up for FRC competitors in AprilTag target designation on robots.

Which focal length puts your tag in the pixel band at docking distance?

Work from field of view first and focal length second: at 1 m the lens chosen below, on a 3.45 µm, 2.5 MP-class sensor, images a 442 mm field and puts about 460 pixels across a 100 mm tag — in band with room to spare.

Two formulas decide it. With p the pixel pitch, W the sensor width in pixels, T the tag width, f the focal length and WD the working distance, all in one unit:

Worked example — dock at 1 m, a 100 mm 36h11 tag, robot approach wander of ±150 mm:

1. Sensor: 2.5 MP class, 1/1.7" format at the common 3.45 µm pitch → W = 2048, sensor width 2048 × 3.45 µm = 7.07 mm.
2. Coverage: 100 mm tag + 300 mm wander + margin → FOV ≈ 450 mm → f = 1000 × 7.07 / 450 = 15.7 mm, so the stock 16 mm lens.
3. Pixels on tag: N = 16 × 100 / (1000 × 0.00345) = 464. Field of view = 442 mm, which is 0.216 mm per pixel.
4. Corner precision at 0.1 px: 0.1 × 0.216 ≈ 0.02 mm — the image-side floor only, before calibration and tilt.
5. At a 2 m dock the same lens gives N = 232 in an 884 mm field: the choice survives a distance redesign.

If N lands over 200, shrink the tag or shorten the lens; under 100, lengthen the lens or move in. The lens focal length calculator runs this arithmetic, and the finder at camera lenses filters stock by focal length and sensor format.

The result lands on a standard C-mount length. The LS-FFL-0567 is a 16 mm, F2.8-22 lens covering a 1.1" image circle with distortion under 0.3%.

Global shutter or rolling shutter when the robot moves?

Fit a global-shutter sensor whenever camera or tag moves during exposure: a rolling shutter reads out rows over several milliseconds, and at 1.5 m/s a 10 ms scan shears the tag image by 15 mm — the code may still decode, but the pose bends.

Benchmarks on ground robots report marker detection surviving 1.5 m/s, but detection is not pose accuracy: the row-by-row readout breaks the flat-frame geometry the pose solver assumes. Blur is the second motion tax, and it sets your exposure clock:

At the worked dock that is 0.216 mm per pixel ÷ 1500 mm/s = 0.14 ms. A 150 µs exposure needs help: open the lens to F2.8, or strobe an LED light for the exposure window only.

When tag and camera hold still until the trigger, rolling shutter costs nothing: capture on the robot's in-position signal and the frame freezes. The shutter trade in detail is in global vs rolling shutter sensors, flash timing in rolling shutter global reset with flash, and blur diagnosis in focus the lens at working distance. Docking pose usually fuses tag fixes with wheel odometry, and the companion task is calibrating swerve drive odometry for path planning.

Filter the area scan cameras list to global shutter — 904 models qualify — then to mono, then to the interface your controller speaks. The MV-CU020-90GC is the 2.5 MP, 1/1.7" global-shutter GigE, C-mount class the example assumed.

How much lens distortion can a pose estimate tolerate?

Hold the lens under 0.3% distortion and calibrate before the first production run: on a 3.45 µm sensor, 0.3% displaces a tag corner near the frame edge by about 4-5 pixels, which is close to 1 mm at the tag plane.

The arithmetic: a 1/1.7" sensor's half-diagonal is about 5.3 mm, so 0.3% of the format is 15.9 µm — 15.9 / 3.45 = 4.6 pixels of corner pull, mapping through 0.216 mm per pixel to 0.99 mm at the tag plane. Distortion grows roughly with the square of image radius, so a tag held in the middle third of the frame sees about a quarter of that, 0.25 mm. A 2% barrel wide-angle, the kind bundled with board cameras, runs the same arithmetic to about 6.6 mm.

Calibration removes the systematic share. Cover the frame with a checkerboard, hold the fit's RMS reprojection error at 0.2 px or better, and the residual lens term lands near 0.2 × 0.216 ≈ 0.04 mm. That is why fixed-focal lenses beat zooms for tags: zoom and focus groups shift distortion as they move, so no calibration sticks. See where your fit is weak with lens distortion vector maps, and check the lens resolves the pixels you paid for with the Nyquist budget method.

Should the camera be mono or colour?

Take mono when the choice exists: a mono sensor samples true black-white structure at every pixel, while a colour sensor interpolates each channel from a Bayer mosaic — and corner extraction lives on exactly that fine structure.

AprilTags are black and white, so colour buys the decode nothing; it earns its place only when coloured cues share the frame. Mono also collects more photons per pixel at the same exposure, decisive at the 0.14 ms exposures of a moving dock. In the camera finder, 638 of the area-scan models are mono and 904 are global shutter; start the search at that intersection. The global-shutter cameras featured in this guide are colour variants; the same shutter and interface classes exist in mono.

What changes when the arm carries the camera or the station is fixed?

The pixel math does not change when the arm carries the camera, but timing and calibration take over: trigger the frame from the robot's in-position signal, calibrate eye-in-hand across varied tag angles, and expect the arm's own 0.05 mm-class repeatability to floor the chain.

At a pause point camera and tag are still, so rolling shutter is acceptable there — the skew tax only applies while pixels read during motion. Give the arm a settle beat of a few hundred milliseconds before triggering; capturing the instant motion stops imports vibration into the corners. Published calibration studies anchor their AprilTag ground truth to manipulators with 0.05 mm repeatability, which tells you the mechanical floor is real. The marker-to-grasp chain is walked step by step in ArUco marker pick and place.

• Eye-in-hand: the hand-eye transform joins camera, flange and tool centre point. Choose calibration poses that swing the tag across 30-40° or more so the solver can separate camera pose from robot compliance — the pose-selection method is in eye in hand calibration.
• Fixed over the cell: one static extrinsic, re-verified after any bracket bump. Drift that survives recalibration usually lives in the arm; the walkthrough on industrial arm pose-file and position accuracy traces that side, and rotation-periodic pick errors point at the extrinsic, as diagnosed in vision pick error under rotation.

Close-range grasp reruns the coverage formula at the new distance: at 0.5 m the same focal length on a 7.4 mm-wide, 12 MP 1/1.7" sensor (4000 × 3000, pitch near 1.85 µm) covers 231 mm and puts about 865 pixels on a 50 mm tag. Far past the band — fine, because at short range the field of view, not tag pixels, sets the lens. When the target is a curved casting or a bin rather than a flat tag, 3D sensors take over and tags stay as the anchor for the robot-to-camera transform. A fixed camera may run rolling shutter safely: the LS-CMR-0166 reads 12 MP at 4000 × 3000 over USB3 at 30 fps.

How do you keep focus, mount and calibration stable in service?

Stability failures outnumber spec-sheet failures: clamp the focus ring at working distance, correct C-mount flange error with shims before calibrating, and re-run intrinsics after any mechanical event that touches lens or mount.

Defocus is the forgiving failure. At F2.8 the worked lens holds focus from about 0.93 to 1.08 m at a 1 m setting (thin-lens estimate with a 7 µm circle of confusion), so ±80 mm of dock wander stays sharp. The expensive failure is mechanical — a lens rotating or tilting on its mount moves tag corners by pixels while the image stays sharp. Lock the focus ring, prefer a locking C-mount on vibrating arms, and re-shoot intrinsics after any bump. The flange arithmetic and shim fix are in C-mount flange focal distance tolerance, and the aperture trade-off behind those depth numbers in aperture f-number and depth of field. Park the robot weekly at a reference pose and read a check tag to catch drift between calibrations.

What do you order for each case?

Both cases order from the same core list — a mono, global-shutter GigE or USB3 camera, a fixed-focal lens with distortion under 0.3%, intrinsics calibrated before first run — and diverge on light and triggering.

For docks beyond 3 m, or one camera covering both approach and dock, you want a wider sensor canvas. The MV-CH120-11UC provides it: 12 MP on a 1.1" sensor, global shutter, USB3.

Its larger sensor buys focal length, and focal length buys pixels on the tag: at a 3 m dock with 1 m of coverage it runs about 44 mm of focal length and lands roughly 420 pixels on the 100 mm tag, where a 7 mm-wide sensor manages about 205. Run your own distances through the formulas above, then let the configurator walk camera, lens and light in order — or start at lens configuration once the sensor is settled. The wider stack of controller, trigger and network is set out in machine vision hardware selection basics, the calibration companions live on the systems topic page, and contact gets a build checked before you order.