Publish camera_info without a driver: calibrate and serve the YAML

ArUco detection, rectification and hand-eye calibration nodes all wait for a camera_info topic beside the image. A camera that only offers a raw stream — a robot-controller camera, a GigE camera whose ROS driver has no calibration slot, a USB board camera under V4L2 — stalls the pipeline at launch, because the CameraInfo message never arrives. You do not need to touch the driver. This guide shows how to calibrate the camera once with a printed checkerboard, save the intrinsics as a YAML file, and serve that file through a small republisher, then how to check the numbers before a robot trusts them.

What do you do when the camera driver publishes no camera_info?

Calibrate the camera once with a checkerboard and save the intrinsics as a YAML file. Then run a small node that publishes the raw stream with a camera_info topic attached; nothing in the driver changes.

camera_info is a single latched message describing one fixed camera-and-lens state: the image size, the distortion vector D, the 3 × 3 intrinsic matrix K, and the rectification and projection matrices R and P. Nothing in that content requires driver support. Any node may publish it, provided the width, height, header stamps and frame_id match the image stream. Detector nodes pair the topics by stamp, so mismatches drop frames silently. This guide covers the case where no camera_info exists at all; once yours is publishing, ROS 2 camera calibration: validate and publish camera_info covers checking it and serving it in production.

How do you calibrate the camera and save the YAML file?

Print a 9 × 6 inner-corner checkerboard, capture 15 to 20 frames of it at varied positions and tilts, and run the camera_calibration calibrator with the measured square size. It writes the YAML the republisher will serve.

1. Make the target exact. Use 9 × 6 inner corners, odd by even, so the calibrator cannot read the board rotated 180°. Measure a printed square with calipers and enter that value, not the nominal 25 mm: the calibrator converts every result to millimetres through it, so a 2% print error becomes a 2% error in fx. When residuals stay high after enough frames, suspect the target first — checkerboard camera calibration error: fix the grid pitch.
2. Freeze the optics. Mount the lens, set focus and working distance, then lock both; tape the focus ring. A camera stand holds camera and framing still while you move the target, and any bump or focus turn means recalibrating. Any frame source works: area scan cameras over GigE or USB3, a board camera under V4L2, the robot's own camera.
3. Capture 15 to 20 frames. Ten poses is the practical minimum; 15 to 20 converge better. Cover the frame corners and centre, tilt the board 30-45° on both axes, keep it sharp — motion blur drags the corner averages. If no corners are found at all, fix illumination and focus first (chessboard not detected in calibration: lighting and focus fixes).
4. Accept only a good residual. Pass the corner counts and the measured square size; the calibrator optimises and prints the mean reprojection error. Accept about 1 px or less, and expect 0.3-0.5 px from a sharp bench set.
5. Save the YAML at a fixed path. The republisher loads it by path, so keep it in the package's config directory where launch files can find it.

A slide-table type such as the LS-MNT-0040 keeps height and distance fixed across the whole session.

What goes into camera_info, and is the calibration sane?

The YAML carries image_width, image_height, the distortion vector D, the 3 × 3 matrix K and the rectification pair R and P. Sanity-check fx against a pixel-pitch estimate before anything consumes the file.

The republisher maps these fields one-to-one into a sensor_msgs/CameraInfo: K flattened into the nine-element k array, D into d, R and P likewise, with width and height taken from image_width and image_height. Before trusting the file, estimate fx from the data sheet:

Here f is the lens focal length, p the pixel pitch and WD the working distance. Worked example with illustrative numbers: a 1920 × 1200 sensor with 3.45 µm pixels behind an 8 mm lens, 500 mm from a 25 mm checkerboard.

• sensor width = 1920 × 3.45 µm = 6624 µm = 6.62 mm
• fx = 8 mm ÷ 0.00345 mm = 2319 px
• FOV width = 6.62 mm × 500 ÷ 8 = 414 mm
• the board is 10 × 7 squares = 250 × 175 mm, so it fills 60% of the frame width and leaves margin for tilt
• each square should image at 2319 × 25 ÷ 500 ≈ 116 px, which also gives the object-space pixel size: 500 ÷ 2319 ≈ 0.22 mm per px (the object-space pixel size method generalises this)

If the calibrator returns fx = 2330, that is 0.5% off the estimate — sane. If it returns 1550, 33% off, a resolution or square-size input is wrong; fix it before anything reads the file. The lens focal length calculator runs this arithmetic for any focal length and sensor size.

On a 2/3" sensor, an 8 mm lens such as the LS-FFL-1582 spans a 57.6° field angle. That is wide enough to keep the board in frame while you tilt it.

How do you publish image_raw and camera_info together?

Three routes work: an OpenCV-based camera node that takes the Linux device index and your YAML, the camera_calibration package's yaml_to_camera_info_publisher.py work-around script beside your existing image publisher, or a node you write yourself.

Driver-style node. The OpenCV camera node takes a device index and a calibration file path as parameters and publishes image_raw and camera_info itself. This is the least code for a /dev/video* source. Under Docker, pass the V4L2 media nodes through, not only /dev/video0 (camera in Docker: pass V4L2 media nodes, not just /dev/video0).

Work-around script. The camera_calibration package ships a documented work-around node, yaml_to_camera_info_publisher.py, that reads a calibration YAML, converts it to a valid sensor_msgs/CameraInfo message and publishes it on a topic you name. Keep your raw image publisher untouched, and give both messages the same frame_id with stamps taken at grab time.

Your own node. Grab frames with OpenCV's VideoCapture, load the YAML, fill the message fields, and publish both topics from one stamp per frame.

The calibrator's output is already shaped for these readers (illustrative values matching the example above):

image_width: 1920
image_height: 1200
camera_matrix:
  rows: 3
  cols: 3
  data: [2318.8, 0.0, 960.3, 0.0, 2319.4, 601.2, 0.0, 0.0, 1.0]
distortion_model: plumb_bob
distortion_coefficients:
  rows: 1
  cols: 5
  data: [-0.12, 0.09, -0.0002, 0.0001, 0.0]

Note where the numbers sit: fx in the first row of k, fy in the second, and the principal point in the third column. Here cx ≈ 960 and cy ≈ 601, near the centre of a 1920 × 1200 frame as expected. The calibrator labels the five-coefficient plumb_bob model it fitted for a standard lens.

How do you verify the setup before the robot consumes it?

Run four checks before the pick-and-place node subscribes: residuals at about 0.5 px or better, both topics at the frame rate with stamps within one frame period, fx within a few percent of the estimate, and a ruler test within about 1%.

1. Residuals. The mean reprojection error is the first gate: aim for 0.5 px and treat 1 px as the ceiling for marker pose work. Camera calibration residual values: average and max targets sets out the per-target numbers.
2. Stream. rostopic hz on both topics (ros2 topic hz under ROS 2) should show the camera's frame rate, and a matched pair's stamps should differ by less than one frame period — 33 ms at 30 fps. Confirm the frame_id exists in your TF tree; camera intrinsics for hand-eye calibration: fix the pose first explains why intrinsics and frame identity must be settled before hand-eye work starts.
3. Scale. Hold a 200 mm wide target 500 mm away; it should measure 200 × 2319 ÷ 500 ≈ 928 px across. Land within about 1%, and a large miss almost always means the square-size entry was wrong.
4. Marker size. ArUco pose needs the marker about 50-80 px wide at minimum, and roughly twice that for stable orientation. At fx 2319 and 500 mm, an 11 mm marker barely crosses the floor, while a 30 mm marker images at 139 px — comfortable.

With those four green, hand-eye calibration can proceed, and it deserves its own check (hand-eye calibration accuracy verification: test before you trust it). If markers must stay small or sit far away, more sensor pixels put more pixels on each marker. A 12 MP, 4000 × 3000 camera such as the LS-CMR-0166 does exactly that. Over the same field, 12 MP gives roughly twice the sampling of a 6 MP sensor.

When is serving a static YAML the wrong fix?

A static YAML describes one optical state, so it is the wrong fix whenever focus, zoom or the lens itself changes during operation. Recalibrate after any mechanical change and expect small drift with temperature across a day.

One turn of a focus ring invalidates every number in the file, and so does a lens swap or a knock against the stand. Zoom and varifocal optics need a file per setting, keyed by that setting. Even untouched, optics drift slowly with temperature — camera focus drift with temperature: diagnose daytime blur covers the diagnosis. For capture on a moving belt or from a camera carried by the robot, use a global-shutter sensor such as the MV-CU020-90GC; a rolling-shutter frame smears a moving marker and biases its pose.

For production cells, check one thing before writing your own republisher: many ROS camera drivers load a calibration file from a parameter and publish camera_info themselves. If yours does, the calibration is your only remaining task — point the parameter at the YAML and the serving becomes native. From there the marker pose feeds the robot cycle as described in 2D vision guided robot pick place grip: how the cycle works. The mounting decision behind it is covered by eye-in-hand vs eye-to-hand camera mounting for robots.

If you are kitting out the bench, start from the lenses category for the optic, then let the camera configurator walk camera, lens and light in one pass. When the pipeline still starves after these checks, contact the applications team — the systems topic page collects the neighbouring guides.