Camera localization for a wall-climbing painting robot

A wall-climbing painting robot has to know where it is on the wall, and the usual dead-reckoning tools answer that worse the longer the job runs. This guide is for the builder deciding whether a camera earns its place on the bill of materials. It puts numbers on the decision: how fast encoders and an IMU drift, what pixel scale a fixed camera needs for millimetre localization, how much compute the detection really takes, and which camera class covers a 4 m wall from a 3 m stand-off. You leave with a lens choice, an exposure, a compute placement and a paint check you can defend.

Why do encoders and an IMU drift on a vertical wall?

Because both dead-reckon: rollers that give up 1% of their travel to slip misplace a 5 m path by 50 mm, and a MEMS gyro that keeps a 0.05°/s residual bias turns the heading estimate 3° per minute.

Encoders count roller or track rotation, not chassis motion. Adhesion loads the drivetrain, suction seals leak, and texture under the rollers lets them skid, so recorded distance and true distance separate; roller diameter tolerance adds a slower scale error on top. The slip compounds linearly: 1% of 5000 mm is 50 mm, and every pass adds another.

The IMU drifts in heading instead. Integrating a 0.05°/s residual bias gives 0.05 × 60 = 3° per minute, so a 10-minute pass ends with a 30° heading error that bends every planned line. These two bias values illustrate what small uncalibrated sensors do; the structural point is that both errors integrate with distance and time and never reset. It is the same drift that motivates the localization stacks used on floor robots, covered in how to resolve odometry drift on wheeled machines and in troubleshooting AMCL odometry scale drift.

Does a fixed camera stop localization drift?

Yes — a camera watching the robot from a fixed mount returns absolute x, y and heading on every frame, so the error stays at the detection accuracy, about ±2 mm here, no matter how long the run lasts.

The camera measures rather than dead-reckons. Stick a printed AprilTag on the robot body, and each frame returns the tag's corners and centroid in wall coordinates; because every measurement is absolute, the error after an hour equals the error after a minute. A single fixed view yields x, y in the wall plane plus yaw. Roll and pitch stay with the IMU, since the chassis is held against the wall and those small angles never accumulate into anything you plot. When a pose still comes out wrong, the diagnosis of vision pick error under rotation applies unchanged: separate scale, offset and rotation before blaming the marker.

How much camera resolution do you need at 3 m from the wall?

About 1 mm per pixel: at 3 m stand-off a 12 MP, 1/1.7" camera with a 6 mm lens covers 3.7 m of wall, and 4000 pixels across that field put a tag centroid within roughly ±2 mm.

Worked example, all in millimetres. A 1/1.7" sensor is about 7.4 mm wide. With a 6 mm lens at a 3000 mm stand-off, FOV = 7.4 × 3000 / 6 = 3700 mm, and 3700 / 4000 px = 0.93 mm/px. A centroid found to ±2 px places the robot at ±1.9 mm. Two tags 200 mm apart give a 215 px heading baseline, where 1 px of error is arctan(0.93 / 200) = 0.27°. At 100 mm/s travel, a 5 ms exposure blurs 100 × 0.005 / 0.93 = 0.54 px. Depth of field is a non-issue: at f/2.8 the hyperfocal distance is 6² / (2.8 × 0.00185) ≈ 6.9 m, so focus holds from about 2.1 m to 5.3 m.

For a 4 m wall, drop to a 5.6 mm lens (7.4 × 3000 / 4000 = 5.55 mm) or keep the 6 mm and stand the camera back at 3.25 m, where the field grows to 3700 × 3.25 / 3 = 4008 mm.

The LS-CMR-0166 matches this example directly: a 12 MP, 1/1.7" rolling-shutter sensor delivering 4000 × 3000 at 30 fps over USB3.

Can a low-power board run the vision, or should it stay off the robot?

Keep it off the robot when you can: a smart camera or a small external PC detects tags at 5-10 Hz while the IMU and encoders hold the 100 Hz balance loop, and 0.1-0.2 s between corrections costs only 10-20 mm of coast at 100 mm/s.

Three placements work. First, with no PC on site, one of the smart cameras inspects on the device; where its tool set covers marker or pattern matching, it can output robot pose with no PC in the loop. The MV-SC5060XM-08M-WBN runs 6 MP mono detection on the device over GigE with its fixed 8 mm M12 lens.

Its 1/2.4" sensor is about 5.5 mm wide, so one camera covers roughly 5.5 × 3000 / 8 = 2060 mm of wall at 3 m stand-off — near 0.7 mm/px at 6 MP — so take two for a 4 m wall. Second, a fixed GigE or USB3 camera feeding a small external PC: 5-10 pose updates per second is enough, so decode a region of interest around the tags instead of the full stream. A quad-core ARM board handles that duty cycle with margin; it does not handle the camera's full 30 fps at full resolution. Third, vision entirely on the robot: optical flow at full frame rate is the expensive path and still drifts, so treat onboard vision as an assist between absolute fixes, never the position source.

How do you mark windows and no-go zones and check paint coverage?

Define no-go polygons once in image pixels — 50 mm of margin around a window is about 55 px at 0.93 mm/px — and judge coverage with threshold and blob checks, where a missed 10 mm streak is 11 px wide and an easy classical detection.

Calibrate the pixel-to-wall mapping at each job start from two fixed tags on the wall frame, then store no-go polygons in pixels and clip the planned path against them; re-shoot the calibration after any camera bump. The same frames inspect coverage: threshold painted against bare wall per zone and reject any unpainted blob larger than 3 × 3 px, which is about 9 px² ≈ 8 mm² at 0.93 mm/px — a fingernail-sized miss. Whether a diagonal count joins two blobs is exactly the connectivity 4 or 8 question in blob analysis for circular markers. Wet paint shifts sheen more than colour, so inspect after a drying interval or shoot cross-polarized; the lighting-first method in dark contaminant detection on dark surfaces transfers directly to missed-paint spots.

Which camera fits each station?

A 12 MP fixed camera covers a 3.7 m wall at 0.9 mm/px; choose global shutter if you localize while moving, a smart camera if you want detection on the device, and the 2.5 MP model only for small walls where ±5 mm is good enough.

All three candidates below are area scan cameras; the table pulls their data-sheet rows.

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

• Default: the LS-CMR-0166, a 12 MP, 1/1.7" USB3 camera reading 4000 × 3000 at 30 fps. Rolling shutter is fine when the robot stops for half a second to localize; the worked example above set its scale at 0.93 mm/px with a 6 mm lens.
• Localizing on the move: the MV-CH120-11UC brings a global shutter to the same 12 MP class on a 1.1" IMX304 sensor. A 1.1" sensor is about 14.1 mm wide, so a 12 mm lens at 3 m covers 14.1 × 3000 / 12 = 3525 mm, which is 3525 / 4096 = 0.86 mm/px — no skew while the robot climbs.
• Small walls: the MV-CU020-90GC offers 2.5 MP on 1/1.7" with a global shutter over GigE, and its colour sensor reads painted-versus-bare contrast directly. Around 1800 pixels across a 3.7 m field is about 2.0 mm/px, so ±2 px is ±4 mm — enough for window-frame checks, not for a full paint pass.
• No PC on site: the smart camera card in the previous section covers about 2.1 m of wall per unit; use two or more for wider walls.

When is a single fixed camera the wrong answer?

A single fixed camera is the wrong answer when the robot leaves its field of view, the wall throws specular glare back into the lens, or overspray films the front element; each of these turns the ±2 mm estimate into a guess.

• Path wider or taller than the view: add a second camera with at least 10% overlap and calibrate the hand-off in software; forcing hardware sync is usually the worse choice, as hardware sync gives worse calibration than software shows. Extending one reference across stitched views hits the same wall as template matching a part larger than the field of view.
• Glossy wall or glass nearby: swing the camera 10-15° off-normal and light diffusely, or the specular band swamps the tag corners.
• Out-of-plane checks such as wall flatness or sash depth: a 2D camera cannot measure height; that is a job for 3D sensors.

What do you check before ordering the camera?

Before you order, size four numbers: field of view with 10% margin, pixel scale at or under 1 mm/px, exposure under 9 ms at 100 mm/s for under 1 px of blur, and a mount that repeats to ±2 px on a parked test tag.

• Coverage: for a 4 m wall, keep the 6 mm lens and stand off 3.25 m (4008 mm of field), or fit a 5.6 mm lens at 3 m.
• Scale: past 2 mm/px the ±2 px error becomes ±4 mm and a 10 mm streak shrinks from 11 px to 5 px; add a camera rather than accept it.
• Exposure and shutter: blur px = 100 × exposure / 0.93, so run 2-5 ms with added light, 9 ms maximum, and take rolling shutter only if the robot stops to localize.
• Lighting: two floods at ±45°, no specular band into the lens, tested at the worst ambient you will paint in; tag corners move when lighting drifts, and fiducial pose drift over time explains why the fix is usually the light, not the code. The machine vision light finder narrows the choices.
• Mount and cabling: a rigid steel stand out of walkway vibration; USB3 passive cable is marginal past 3 m, so take GigE and standard network cabling for longer stand-offs.
• Calibration: two fixed wall tags define the pixel-to-wall map; re-verify to within ±2 px at every job start.

Size the camera end to end in the camera finder or start at the camera configurator step; the systems topic page collects the integration guides, and spec questions go through the contact page. In-stock items ship within one business day.