Decompose a projection matrix into camera intrinsics

A calibration routine, a bundle adjuster or a solver hands you a 3 × 4 projection matrix P, and the code downstream needs plain numbers: focal lengths in pixels, a principal point, a rotation and a translation. Four steps pull all of them out of P, and the arithmetic is small enough to check by hand. This how-to gives the RQ route, the exact library calls, a worked 1280 × 720 example where every entry is verifiable, and the checks that catch a bad decomposition before it reaches a robot controller. All you need is the matrix and a linear algebra routine with a QR.

What a 3 × 4 projection matrix holds

P packs the geometry of one calibrated camera into twelve numbers, defined up to a single arbitrary scale:

(fx, fy) is the focal length in pixels, (cx, cy) the principal point, s the skew, which stays near zero on modern sensors. R is the camera's orientation relative to the world frame, and t places the world origin in camera coordinates, so the camera center in world units is C = -R^T·t. RQ decomposition of the left block M returns the pair directly: an upper-triangular factor can only be K, an orthogonal one only a rotation. With the signs fixed, that pair is unique. The translation is not: it rides the unknown scale of P.

In a vision cell this P is the matrix behind your own calibration of the capture head — a projected structured-light 3D camera such as the MV-DPS200P-02, or any calibrated 2D camera on the line.

What you need

• The 3 × 4 matrix P in consistent storage: 3 rows, 4 columns, with the row-major or column-major order known.
• A QR routine, or a library RQ: OpenCV's RQDecomp3x3, scipy.linalg.rq, or about twenty lines of Givens rotations in C#.
• One known 3D point in the calibrated frame for the final reprojection check.

Decompose P in four steps

1. Fix the scale. Divide the whole matrix by its bottom-right entry P[2,3] (use the Frobenius norm if that entry is zero). P is homogeneous, so nothing physical changes, but every later output becomes reproducible for that one convention.
2. RQ-decompose the left block. Take M as the first three columns of P. An RQ routine returns M = K·R with K upper-triangular and R orthogonal. Without one, reverse the row order and the column order of M^T, run a plain QR, reverse both factors back and transpose them: the transposed triangular factor is K, the transposed orthogonal factor is R.
3. Fix the signs. Build D = diag(sign K00, sign K11, sign K22) and set K ← K·D, R ← D·R. Since D·D gives the identity matrix, the product K·R still equals M and the diagonal of K turns positive. If det(R) now reads -1, negate both K and R; the next step absorbs the sign.
4. Normalize and solve for t. Divide K and p4 by K[2,2], leaving K[2,2] = 1. Solve K·t = p4 by back-substitution from the bottom row. The units of t are the units your calibration target was measured in — millimetres in the example below.

The calls to use, and the .NET gap

Two functions in OpenCV's calib3d module cover the job. RQDecomp3x3(M, R, Q) fills R with the upper-triangular factor — your K — and Q with the orthogonal rotation. decomposeProjectionMatrix(projMatrix, cameraMatrix, rotMatrix, transVect, rotMatrX, rotMatrY, rotMatrZ, eulerAngles) runs the whole split and adds the per-axis rotation matrices and the Euler angles as optional outputs; the classic C entry point cvDecomposeProjectionMatrix takes the same list. If your .NET wrapper has not bound them, declare the export yourself with P/Invoke, following the wrapper's own interop declarations for how managed code calls unmanaged exports. Steps 1-4 above also run entirely in managed code, so no wrapper is strictly required.

Worked example: 1280 × 720 camera, fx = fy = 800 px

The camera looks at a millimetre calibration target and a solver returns:

P = [    0   -800    640  1200000 ]
    [  800      0    360   680000 ]
    [    0      0      1      2000 ]

So M = [[0, -800, 640], [800, 0, 360], [0, 0, 1]] and p4 = (1200000, 680000, 2000). RQ returns:

K = [ 800    0  640 ]      R = [ 0  -1  0 ]
    [   0  800  360 ]          [ 1   0  0 ]
    [   0    0    1 ]          [ 0   0  1 ]

The signs are already clean: positive diagonal in K, K[2,2] = 1, det(R) = +1. R is a 90° turn about the world z axis. Back-substitution gives t:

• Bottom row: 1·tz = 2000, so tz = 2000 mm.
• Middle row: 800·ty + 360·tz = 680000, so ty = (680000 - 720000) / 800 = -50 mm.
• Top row: 800·tx + 640·tz = 1200000, so tx = (1200000 - 1280000) / 800 = -100 mm.

Now prove the split two ways. K·R must rebuild M: its top-middle entry is 800·(-1) + 0·0 + 640·0 = -800, which matches. And the world origin must project identically: through P, the fourth column (1200000, 680000, 2000) divides by 2000 to (600, 340) px; through the decomposition, the origin lands at x_cam = t = (-100, -50, 2000) mm, so u = 800·(-100/2000) + 640 = 600 px and v = 800·(-50/2000) + 360 = 340 px. Identical.

Scale test: feed 2·P. After the division by K[2,2], K and R come back unchanged, while t doubles to (-200, -100, 4000) — the translation carries whatever scale the input carried, which is exactly why a normalized P cannot yield a metric t.

Checks before you trust K, R and t

• Lower triangle of K: every entry below the diagonal under about 1e-6 of K[0,0] — the residue of the factorization itself.
• det(R) = +1 and R·R^T equal to the identity matrix to about 1e-9.
• One known 3D point reprojects to its measured pixel; the origin landed at (600, 340) px above.
• Skew K[0,1] near zero. Square-pixel sensors give a few pixels' worth at most; a skew of tens of pixels means a bad correspondence got folded into the calibration. fx and fy should agree within a fraction of a percent unless the pixels are genuinely rectangular.
• Focal length in millimetres: fx divided by the pixel pitch. fx = 800 px on a 3.45 µm pixel gives f = 800 × 3.45 µm = 2.76 mm; take the pitch from the sensor data sheet.
• Camera center: C = -R^T·t. Here R^T·t = (-50, 100, 2000) mm, so the head sits at (50, -100, -2000) mm, 2 m from the target origin.

Four mistakes that produce a wrong K or t

• Quoting t as metric millimetres after any tool normalized P. Only the direction of t survives; recover the scale from a known distance in the scene first.
• Feeding a transposed matrix. A 4 × 3 layout, or a row-major/column-major mismatch across the wrapper boundary, decomposes the wrong block.
• Skipping the sign fix. A decomposition with det(R) = -1 is a mirrored pose: the camera looks backwards and every later projection flips.
• Ignoring a singular left block. RQ needs M nonsingular; if the routine reports a zero pivot, the calibration behind P is at fault, not the factorization.

Where K, R and t go next

K is what a PnP pose solver consumes, and matching its convention on a rectified image is the subject of the rectified-image pose estimation guide. R and t seed a pose initial guess, provided they land in the reference frame the solver expects. R itself answers how the camera is oriented relative to a fixed frame. Keep the recovered K next to the intrinsics the depth stream reports — the read-don't-guess intrinsics check catches wrappers that hand over a different convention — and treat mismatched clouds and poses downstream with the fine-matching input check.

To choose the capture head the matrix describes, start at the camera finder or the camera step of the configurator; the 3D sensors and area scan cameras categories hold the hardware. For questions on a specific calibration, use the contact page. The 3D vision topic page collects the related guides.