Planar tag pose jitter reduction for camera localization

Your tag is bolted to the ceiling, the camera is calibrated, and the reported tag position holds five percent accuracy — yet Cam_X and Cam_Y swing wildly, and pitch and roll wander 4-5° while yaw stays inside 0.1°. That pattern is normal for a single planar fiducial viewed from metres away: it is a geometry and sampling problem, not a calibration fault. This guide works through the causes in order of leverage, the test that confirms each one, and the fixes — pixels on the tag, sharp edges, a square-planar solver, and a filtered pose stream. The worked example, a 150 mm tag at 2.5 m, reproduces the numbers you are seeing and then removes them.

Stable yaw, noisy tilt: read the geometry first

Yaw rotates the tag inside its own plane, so every corner travels sideways and the solver reads it directly; 0.1° is what a healthy detector delivers. Pitch and roll only foreshorten the square: the projected side shrinks with the cosine of the tilt, and near head-on the cosine is flat.

Telling 0° from 5° means measuring that projected side to better than 0.4%. A tag imaged at 150 px per side with 0.5 px of corner noise carries about 0.7 px of side noise, roughly 0.5%, which is not enough, so the tilt estimate wanders by degrees. Confirm the cause in your logs: jitter that peaks when the robot passes directly under the tag and falls away to the sides is this geometry. Yaw holding at 0.1° also clears your intrinsics, because a focal-length error scales a pose smoothly rather than shaking it.

For the related failure where the estimate flips between two distinct poses, read planar tag pose ambiguity: diagnose flickering estimates. For how small is too small, see fiducial marker pixel size: why tiny tags fail.

Inverting the rotation smears a good translation

The camera pose in the tag frame is p = −Rᵀ·t, because a rotation matrix is orthonormal and its inverse is its transpose. Your translation t is the accurate part; the rotation is not, and a pitch error σ rotates the whole 2.5 m lever arm: σ_XY ≈ Z × σ. Four degrees is 0.0698 rad, and 2500 mm × 0.0698 = 174 mm. Cam_X can jitter by ±170 mm while the tag coordinates hold five percent, purely from rotation noise multiplied through the inversion.

Test it: regress the Cam_X error against Z·sin(pitch). A slope near one means the rotation is driving the position, and you can stop tuning the translation. Compose the transform properly, too: building the camera position from R times a tag coordinate mixes reference frames whenever that point is fixed elsewhere. Camera pose initial guess: which reference frame it means and fiducial pose wrong distance: check calibration and frames cover that bookkeeping.

One check before any hardware changes: print the pitch angle. A ceiling tag over a level camera commonly sits near ±90° in Z-Y-X Euler extraction, where the arctangent divides two small numbers and hands degrees of noise to roll. If the absolute pitch exceeds 85°, carry quaternions and convert to Euler once, for display.

Fix 1: put at least 400 pixels on the tag

Detection survives from about 40 px per side, but pose quality tracks the pixel count: 150 px buys detection with degrees of tilt noise; 400 px with sharp edges buys fractions of a degree. Tilt noise falls as one over the pixels on target, so doubling the count halves the degrees.

Three levers, cheapest first: print the tag larger — 150 mm to 300 mm halves the jitter — fit a longer lens, or move to a finer-pixel sensor. Search the lens finder and the camera finder, or browse machine vision lenses and area scan cameras. The common wrong move is a wider lens to find the tag sooner: it strips pixels away exactly where the pose is weakest. Pick the focal length for the farthest position at which you still need a fix.

Worked example: 150 mm tag at 2.5 m

Take the LS-CMR-0166: a 12 MP USB3 camera with a 1/1.7-inch rolling-shutter sensor delivering 4000 × 3000 at 30 fps. A 1/1.7-inch sensor at that resolution has 1.85 µm pixels, so its active width is about 7.4 mm. In front of it, the LS-FFL-0567 is a 16 mm C-mount lens rated to 20 MP, F2.8-22, under 0.3% distortion, in a 1.1-inch format that covers the sensor with margin.

σ is in radians and α is the angle between the camera axis and the tag normal; the 1.4 is the √2 from the two corners that define a side. The estimate is good to about a factor of two, because it ignores keystone, which grows above 30°.

1. Magnification: m = 16 / 2500 = 0.0064.
2. Field of view at the ceiling: 7.4 mm / 0.0064 = 1156 mm wide, so the tag stays framed out to about ±0.58 m of lateral travel.
3. Pixels on the tag: 150 mm × 0.0064 = 0.96 mm on the sensor; 0.96 mm / 1.85 µm = 519 px per side.
4. Tilt noise with sharp corners, σ_corner = 0.15 px, so side noise 1.4 × 0.15 = 0.21 px: at 10° off-normal, 0.21 / (519 × 0.174) = 0.0023 rad = 0.13°, which is 2500 × 0.0023 = 6 mm of XY noise through the inversion.
5. Directly under the tag, about 2° off-normal: 0.21 / (519 × 0.035) = 0.012 rad = 0.7°, about 29 mm. That weak zone is geometry, not a tuning error.
6. The before case, 150 px on the tag and 0.5 px blurred corners: 0.71 / (150 × 0.035) = 0.135 rad ≈ 8°, with a lever arm near ±340 mm. That is the 4-5° being measured, plus the position scatter it drives.

Fix 2: remove the blur, then keep it off

Sharp edges give 0.10-0.20 px of corner repeatability; soft edges give 0.5-1 px, and the jitter scales linearly with that number. Estimate edge position accuracy from image sampling puts the limits in numbers.

Focus is the first suspect. A 16 mm lens at F2.8 focused at 1 m images a tag at 2.5 m through a 0.16 mm focus error across its 5.7 mm aperture: a blur circle near 55 µm, or 30 px. Nothing downstream survives that. Focus on the live tag at the working distance and lock the ring, following camera focus during calibration: set a repeatable position. Once set, focus holds: at F2.8, 2.5 m and a 1 px circle, the total depth of field is roughly 250 mm, and a flat ceiling with a fixed-height robot sits inside it.

Motion blur comes next. At 300 mm/s of robot speed and m = 0.0064, each millisecond of exposure smears 300 × 0.001 × 0.0064 / 0.00185 = 1.0 px. Cap the exposure at 0.5 ms, fix the gain, and switch auto-exposure off; contour position jitter under automatic exposure shows how exposure wander turns into position wander.

Rolling shutter is the last term. At 30 fps over 3000 rows, successive rows are read roughly 10 µs apart, so a camera rotating during exposure skews the square into apparent pitch. Straight-line driving at short exposure is fine; if the camera rotates while imaging, move to a global-shutter sensor such as the MV-CH120-11UC, a 12 MP USB3 camera with a 1.1-inch sensor, C-mount and global shutter.

Fix 3: solve with the square-planar solver

Feed the detector corners, as floats, to the OpenCV solver built for square planar targets:

s = 149.5  # measured side of the printed black square, mm
objp = np.array([[-s/2,  s/2, 0],
                 [ s/2,  s/2, 0],
                 [ s/2, -s/2, 0],
                 [-s/2, -s/2, 0]], dtype=np.float32)
ok, rvec, tvec = cv2.solvePnP(objp, corners, K, dist,
                              flags=cv2.SOLVEPNP_IPPE_SQUARE)
cv2.solvePnPRefineLM(objp, corners, K, dist, rvec, tvec)

Skip RANSAC: all four points are inliers by construction. The corners must arrive in the same order as the object points, and most AprilTag-family detectors already refine them to subpixel, so do not round them to integers. IPPE, infinitesimal plane-based pose estimation, returns two candidate poses for any planar target: near head-on the two converge, and when they separate you keep the lower reprojection error and prefer the previous frame's choice. Fiducial tag pose estimation in OpenCV from detected corners covers the pipeline end to end.

Fix 4: filter the pose stream, not the image

A rigid tag on a rigid ceiling cannot jitter; the estimates can. Median-filter five frames first, which deletes single-frame ambiguity flips, then smooth each pose component with a first-order filter. At 30 fps:

The noise factor is √(α/(2−α)) and the lag is (1−α)/(α × 30 fps). Pitch jitter of 4.5° lands at 1.9° with α = 0.3, and at a fraction of a degree once the pixel and focus fixes go in first. Smooth rotation vectors or interpolate quaternions between frames; extract yaw, pitch and roll once, after filtering.

When the usual advice is wrong

A wider lens to find the tag earlier trades the pixels that carry the pose for coverage you cannot spend: with one tag, you pick the focal length for the farthest fix and accept the rest. Recalibrating fails the evidence, because yaw at 0.1° against jittered pitch is not an intrinsics signature. Raising the frame rate leaves per-frame jitter exactly where it is and only feeds the filter faster.

If the robot needs only planar localization — X, Y and heading — take them from t and yaw, and stop consuming pitch and roll near the normal, which are the weakly measured axes. If you genuinely need tilt, engineer the view: 15-30° off-normal gives sin α = 0.26-0.50 against 0.035 at 2°, a 7-14× stronger tilt signal.

Bench check: separate optics from motion

Clamp the camera at its working distance on a camera stand, aim it at the fixed tag, and log 200 frames with everything stationary. Static pitch and roll scatter should sit under 0.5°. The LS-MNT-0040 is a professional slide-table stand, height 700 mm, with a 400 × 450 mm base, and it slides the camera without disturbing focus. If the static log is clean but the robot jitters, the fault is motion — exposure, vibration or rolling shutter — and detecting camera vibration with OpenCV frame tracking separates camera shake from estimation noise.

Print the tag on matte stock, mount it flat, and keep the quiet zone at least one module wide; gloss highlights and a thin border both push corner noise up.

The fix list, in order

1. Count pixels on the tag at the farthest fix distance: 400 px per side minimum, or a bigger print.
2. Focus on the live tag at the working distance, then lock the ring.
3. Cap exposure at 0.5 ms for 300 mm/s of travel; auto-exposure off, gain fixed.
4. Solve with SOLVEPNP_IPPE_SQUARE plus refineLM; corners as floats, tag size measured with calipers.
5. Median five frames, then smooth at α = 0.3 for 30 fps; quaternions in, Euler out, once.
6. Compose the camera pose as −Rᵀ·t; never let rotation noise re-enter the position.
7. Pass the static bench test under 0.5° before commissioning.

Run the geometry through the system configurator, starting at the lens, or send the three numbers that set the design — ceiling height, tag size and robot speed — to our applications team.