Getting 3D camera intrinsics into a ROS 2 CameraInfo message

You are writing a ROS 2 driver for a fixed-focus 3D camera, and the tools downstream want a sensor_msgs/CameraInfo message before they will align, project or measure anything. The camera already holds the numbers: its intrinsics were measured at the factory when the unit was built, and on a fixed-focus camera they do not change. This guide shows which number goes in which field, how to pull the values from the SDK once at startup instead of hardcoding them, how to scale them when the SDK reports at a different resolution than you publish, and the checks that prove the loaded message is right. The multi-camera case gets its own section: one message per camera, filled per serial number.

What intrinsics does a fixed-focus 3D camera already know?

Four numbers per stream: fx and fy in pixels, and the principal point cx and cy in pixels — factory-measured constants of the lens-and-sensor stack that a fixed-focus 3D camera reports through its SDK for the unit in your hand.

fx and fy are focal lengths in pixels. A target 1000 mm in front of the lens and 100 mm left of the optical axis lands 0.1 × fx pixels left of cx; that relation, u = fx × X / Z + cx, is everything a projection node does with the matrix. Because the lens neither zooms nor refocuses, the four values hold for the life of the unit — but they belong to the serial number, not the model. Assembly seats each sensor behind its lens with a small placement error of its own, so the next unit of the same model reports different numbers. Read per unit, the same rule stereo depth scale factor and focal length: read them per camera applies to depth scale.

Fixed-optics 3D sensors — laser galvanometer stereo, projected structured light and ToF units such as these — expose their intrinsics exactly this way:

Which fields of sensor_msgs/CameraInfo do you fill?

Six entries carry the intrinsics: the nine slots of K hold fx, fy, cx and cy with fixed zeros and a one, the twelve slots of P repeat K beside a zero column, and width and height take the pixel size of the stream you publish.

K is a 3 × 3 matrix flattened row-major: K[0] = fx, K[2] = cx, K[4] = fy, K[5] = cy, K[8] = 1, and the other four slots zero. P is the 3 × 4 projection matrix flattened the same way; for a single camera it is K with a fourth zero column, so a plain copy works. Set width and height to the frames you actually publish — a mismatch here is the most common way a correct K still produces wrong overlays. Leave the distortion fields at their defaults: the point cloud is already metric, and rectified stereo frames have had distortion removed before the SDK hands them over.

The message exists because consumers do not talk to the camera. A node that lifts a 2D detection into 3D reads K and applies the relation in reverse, the path walked in 2D bounding box to 3D: estimate object extent from depth and 2.5D height map to 3D point cloud: pixels to XYZ.

How do you load the intrinsics at node startup?

Once per connection: open the camera, query its intrinsics through the SDK, fill the message, publish it, then keep the values in memory for the life of the node — they are build constants, not live telemetry.

1. Connect to the camera and read its serial number.
2. Query the intrinsics through the SDK's camera-information call and record fx, fy, cx and cy.
3. Record the resolution those values describe; call its width ResMax in the example below.
4. Scale the values if the stream you publish is smaller — the next section gives the factor.
5. Fill K, P, width and height; leave distortion at its defaults.
6. Publish on the camera's info topic with a quality-of-service profile that keeps the last message for late joiners, and cache the values.

# Worked example, one camera: values already scaled to the 1280 x 800 stream.
fx = 1090.0     # float, px
fy = 1090.0     # float, px
cx = 638.3      # float, px
cy = 401.7      # float, px
width = 1280    # int, px
height = 800    # int, px

K = [fx, 0.0, cx,
     0.0, fy, cy,
     0.0, 0.0, 1.0]        # nine slots, row-major

P = [fx, 0.0, cx, 0.0,
     0.0, fy, cy, 0.0,
     0.0, 0.0, 1.0, 0.0]   # twelve slots, row-major

# Test: a point on the optical axis, 1000 mm out.
X = 0.0                    # float, mm
Y = 0.0                    # float, mm
Z = 1000.0                 # float, mm
u = fx * X / Z + cx        # -> 638.3 px, equal to cx
v = fy * Y / Z + cy        # -> 401.7 px, equal to cy
assert (u, v) == (cx, cy)

Anything else the unit reports about itself follows the same once-at-startup pattern — read IMU data from a 3D camera through its SDK, not ROS makes the same argument for the IMU stream.

How do you scale intrinsics when the SDK reports another resolution?

Multiply every entry by the width ratio: an SDK reporting fx = 1635.0 px at 1920 px, with a stream published at 1280 × 800, gives fx = 1635.0 px × 1280 / 1920 = 1090.0 px in the message.

fx_unit and its neighbours are the reported full-frame values and ResMax the reported width, both read once in step 3; width is the stream you publish. One factor covers every entry while the stream keeps the sensor's aspect ratio — 1280 × 800 inside 1920 × 1200 does, both 8 : 5.

The worked example, start to finish:

• s = 1280 / 1920 = 0.667
• fx = 1635.0 px × 1280 / 1920 = 1090.0 px, and fy the same
• cx = 957.5 px × 1280 / 1920 = 638.3 px
• cy = 602.5 px × 800 / 1200 = 401.7 px
• K = [1090.0, 0, 638.3, 0, 1090.0, 401.7, 0, 0, 1]

The principal point lands 1.7 px left of and below the frame centre at (640, 400) — exactly the size of offset a real unit carries. The other stream changes and what they move:

What field of view do the loaded numbers imply?

Two lines of arithmetic: the horizontal field of view is 2 × atan(width / (2 × fx)) = 60.8° for the worked example, and one pixel spans Z / fx = 0.73 mm at 800 mm, so the view is 0.94 m wide there.

• Horizontal: 2 × atan(1280 / (2 × 1090)) = 2 × atan(0.5872) = 60.8°
• Vertical: 2 × atan(800 / (2 × 1090)) = 2 × atan(0.3670) = 40.3°
• Ground size at 800 mm: 800 / 1090 = 0.734 mm per pixel
• Width on target: 1280 × 0.734 mm = 939 mm, matching 2 × 800 × tan(30.4°) = 939 mm

Use those two numbers — pixels per millimetre and field of view — to size the stream before you lock the intrinsics. The camera resolution calculator turns them into the pixel count a part needs, and its button starts from the result.

Do you still need to calibrate the camera yourself?

No — not the intrinsics of a fixed-focus unit: the factory values the SDK returns are the ones the camera's own firmware used to build the point cloud, so a fresh checkerboard estimate of K makes your overlays disagree with the cloud by the calibration error.

Calibrate yourself only for geometry the factory never measured:

• Aligning depth to a separate colour camera — two devices, two frames; depth-to-RGB alignment off? Run your own checkerboard calibration covers the target and the fit.
• Extrinsic poses between two cameras you paired yourself, with each unit's factory K fixed as an input: stereo calibration with fixed intrinsics: extrinsics only.
• Robot guidance, where the intrinsics feed the pose chain as constants — camera intrinsics for hand-eye calibration: fix the pose first.

How do you serve several 3D cameras from one node?

One CameraInfo per camera, keyed by serial number and published on a per-camera topic: query every connected unit at startup and build a dictionary, because two units of one model report principal points a few pixels apart.

# Filled once at startup, one tuple per connected serial number.
intrinsics_by_serial = {
    'unit-a12': (1090.0, 1090.0, 638.3, 401.7),   # fx, fy, cx, cy in px
    'unit-c56': (1095.4, 1095.4, 641.9, 399.6),
}

The two tuples differ by a few pixels on every entry — that difference is the whole reason the dictionary exists. Name each topic after the serial or the mount point, publish each message as soon as its values are in, and do not average or share one tuple across the fleet: every consumer that projects with it inherits the wrong principal point.

When are the factory intrinsics the wrong answer?

When the stream you publish is not the stream the values describe: a cropped region of interest shifts the principal point by the crop offset, 2 × 2 binning halves fx, and a camera with a changeable lens needs its K from your own calibration after every lens swap.

• Region of interest. Cropping never changes fx or fy; it moves cx by the left crop offset and cy by the top offset. The trade-offs sit in 2D vs 3D camera ROI: where to set it and which one wins.
• Binning. 2 × 2 binning pools four pixels into one: fx and fy halve, and cx and cy halve to within the 0.5 px pixel-centre convention.
• Resolution mismatch. Most SDKs report at the sensor's full size; scale before filling, per the worked example.
• Raw frames. Publish unrectified images and the distortion fields stop being decoration — they must come from your own calibration.
• Your own 2D camera. An area scan camera with a changeable lens, such as the MV-CS200-10GC, carries no factory K at all:

Calibrate that camera after each lens change and again after a refocus, because a C-mount lens seats with a small rotation error every time it is screwed on.

What do you check before you call the node done?

Three checks settle it: a ray on the optical axis must land at cx and cy, a known length at a known distance must give the predicted pixel count, and width and height must match the frames you publish.

1. Centre-ray test. Project X = 0, Y = 0, Z = 1000 mm: u and v come out at cx and cy, 638.3 px and 401.7 px in the example, not at the frame centre.
2. Gauge test. Place a 200.0 mm bar at Z = 800 mm: it spans 200.0 / 0.734 = 272 px; accept ±3 px, which is 1 %.
3. Frame match. width and height equal the published image size, pixel for pixel.
4. Serial keyed. Each camera publishes values read from its own query, never a shared tuple.
5. Late joiners. The message is published at startup under a profile that keeps it, so a node started later still receives it.

A node that passes all five has intrinsics that agree with the camera that produced the cloud — the only standard that matters. From here, pick hardware in 3D sensors or the camera finder, size the stream with the calculator above, and open the system configurator at the camera step when you pair a 3D unit with a colour camera. More set-up pieces sit on the 3D vision topic page.