You calibrated a projector against a camera, the toolbox returned two focal lengths, fx and fy, in pixels, and cvPOSIT in OpenCV asks for one number. This guide is for anyone running pose estimation on projected points: structured-light rigs, a projector paired with a depth camera, projection mapping against a known 3D model. You will leave with the value to type in, its unit, the points to subtract before the call, how far the compromise sits from the truth, and a check that proves the pose.
The answer: average the two focal lengths
Take the two entries of the focal-length vector fc from your projector-camera calibration - both are in pixels - and give cvPOSIT their arithmetic mean:
POSIT models an ideal pinhole camera: one focal length, square pixels, zero skew, principal point on the optical axis. There is no second axis to hang fy on, so any value you pass is a compromise, and the mean is the standard one. It leaves each axis off by half the fx-fy split, instead of one axis perfect and the other carrying the whole difference. For this algorithm the mean is the right answer, not a workaround.
Why POSIT takes one number, not two
A calibration returns two focal lengths because the square-pixel pinhole model never fits a real device exactly; separate x and y scales absorb manufacturing anisotropy and fit residuals. The OpenCV calibration module documents fx and fy as focal lengths in pixel units, with cx, cy the principal point - richer than POSIT needs. POSIT (Pose from Orthogonality and Scaling with Iteration) comes from the older C interface and takes a focal-length argument and nothing else; the POSIT tutorial estimates an object's pose from one image with exactly that input.
A projector is a camera run backwards. Its image plane is the pattern buffer - the grid of pixels the device draws - and your calibration fitted intrinsics for that virtual camera, in pattern pixels. One consequence trips people up: the 2D points you feed POSIT must live in that same frame, the projector's pattern pixels, not the watching camera's pixels. The hand-off between the two frames is what structured light 3D cameras: projector pixel mapping to 3D points works through.
What you need from the calibration
- The focal-length vector fc: two entries in pixels. fx is the first, fy the second; calibration toolboxes of this kind store the focal length in pixels as a two-entry vector.
- The principal point cx, cy, in pattern pixels, usually near the buffer centre.
- Distortion coefficients for the projector, when the fit reports them.
- Your 3D model points in one consistent unit; millimetres is the sensible choice.
- The 2D correspondences, measured in pattern pixels.
Feed cvPOSIT the average, step by step
- Read fx and fy from the calibration output. Both arrive in pixels; do not convert them.
- Compute f = (fx + fy) / 2. That single value is what cvPOSIT consumes.
- Check the split before trusting anything downstream: |fy - fx| / f. Under 1 percent, continue. Above 1 percent, recalibrate first - a split that wide usually means a weak pattern fit rather than a genuinely anisotropic projector, and it biases every pose computed after it. Structured light calibration fails on the second shot covers the usual capture faults.
- Undistort your 2D points with the projector's distortion coefficients when they were reported. POSIT has no distortion input; points must arrive already flat.
- Subtract the principal point. cvPOSIT takes no cx, cy - its origin sits on the optical axis - so feed it (u - cx, v - cy) for every correspondence, still in pattern pixels.
- Call cvPOSIT with the model points, the centred 2D points and f. It returns a rotation matrix and a translation vector.
- Read the translation in the unit of your model points: millimetres in, millimetres out. Marker pose units: tvec in the unit of the marker length explains why the unit follows the model, never the camera.
Worked example: a 1024 x 768 projector at one metre
Run a typical calibration output through the arithmetic. Suppose the fit returns fc = [1032.5; 1038.9] px on a 1024 x 768 pattern, principal point (512.3, 384.7) px, small residuals:
- f = (1032.5 + 1038.9) / 2 = 2071.4 / 2 = 1035.7 px
- Split = (1038.9 - 1032.5) / 1035.7 = 6.4 / 1035.7 = 0.0062, or 0.62 percent between the axes; each axis sits 0.31 percent from the mean
- With a 12 um pitch on the projector's imaging device, the physical focal length is 1035.7 px x 0.012 mm/px = 12.43 mm - the figure a lens datasheet prints and the lenses finder sorts by
- Pattern width on the device: 1024 x 0.012 mm = 12.29 mm; half-field angle atan(6.14 / 12.43) = 26.3 degrees, so about 52.6 degrees across
- Magnification at a 1000 mm throw: 1000 / 12.43 = 80.5; projected width 12.29 mm x 80.5 = 989 mm, so one pattern pixel lands as 989 / 1024 = 0.97 mm on the surface
The millimetre figure is for comparing with optics; cvPOSIT consumes the pixel value, 1035.7. Working the other way - recovering focal length and sensor size when no datasheet exists - is what finding the sensor size and focal length with no datasheet does, and why focal length is quoted in pixels settles where the unit comes from.
What the compromise costs in pose error
POSIT's translation scales linearly with the focal length you supply: halve f and every recovered distance halves. The relative error you put in is the relative distance error you get out:
Rotation barely moves; it depends on ratios inside the image, not on absolute scale. With the mean, the bound is half the split:
| fx-fy split | Worst distance error | At a 1000 mm throw |
|---|---|---|
| 0.25 percent | 0.13 percent | 1.3 mm |
| 0.62 percent (the example) | 0.31 percent | 3.1 mm |
| 1.0 percent | 0.5 percent | 5 mm |
| 2.0 percent | 1.0 percent | 10 mm |
Feeding fx alone - 1032.5 px instead of the mean - shortens every distance by 0.31 percent, about 3.1 mm at one metre, all in one direction. That asymmetry is why the mean, not a favourite axis, is the answer. The same linear rule, read off a different camera, drives stereo depth scale factor and focal length.
Check the pose before you trust it
- Reproject: push the model points through the returned rotation and translation, project with f, and measure the residual against your 2D points. Aim for an RMS under about half a pattern pixel; a full pixel means something upstream is wrong. Projecting a 3D point to pixel coordinates with a calibrated camera shows the pipeline end to end.
- Tape-measure one distance. Put a model feature at a known throw and compare the recovered Z; agreement inside the half-split bound - 3.1 mm at one metre in the example - confirms the focal length did its job.
- Rotate the model 90 degrees on the table and confirm the rotation matrix follows. A constant offset means the principal-point subtraction was skipped or wrong.
When not to average at all
When a careful recalibration still leaves fx and fy more than about 1 percent apart, the projector truly does not fit the square-pixel model, and a single-number pose solver is the wrong tool. Newer OpenCV pose routines in the calibration module accept the full cameraMatrix - separate fx, fy, cx, cy - so nothing gets averaged; choosing pose-estimation intrinsics for rectified images walks that choice. Averaging also cannot rescue points captured in the wrong frame: camera-pixel correspondences fed with a pattern-pixel focal length produce a confident, wrong pose.
When the deliverable is a depth map or point cloud rather than the pose of a known model, stop maintaining a projector calibration altogether. Industrial 3D camera selection: stereo, ToF or structured light sorts the options, and the 3D sensors category holds them.
Factory-calibrated units that replace the hand-built rig:
- Type
- Stereo / structured light
- Type
- Stereo / structured light
The MV-DPS200P-02 is a projected structured-light 3D camera - the projector and its calibration stop being your problem. The MV-DLS300P-04 is a laser galvanometer stereo camera, the same idea with a steered beam. Browse the camera finder for the full range.
Quick checklist
- One focal length into POSIT: f = (fx + fy) / 2, in pixels, straight from the projector calibration.
- 2D points in pattern pixels, undistorted, principal point subtracted.
- Split |fy - fx| / f under 1 percent, or recalibrate before believing any pose.
- Translation read in the unit of the model points.
- Reprojection RMS under about half a pattern pixel, plus one tape-measured distance inside the half-split bound.
Spørsmål
Should POSIT get the focal length in millimetres or pixels?
Pixels. cvPOSIT ties image measurements to model geometry through one scale factor, and your 2D points are pixel counts, so the focal length must share their unit. Convert to millimetres only when comparing optics: multiply by the imaging device's pitch, so 1035.7 px at 0.012 mm/px is 12.43 mm. If you rescale the points to another frame, rescale f by the same factor.
What if fx and fy differ by more than one percent?
Recalibrate before averaging. A split that wide almost always means a weak fit - too few pattern poses, poor corner coverage, a board that moved - rather than a projector with truly non-square scaling. Averaging anyway leaves up to half the split as distance error on each axis: two percent of split is one percent of every recovered distance, 10 mm per metre of range. If the split survives a careful refit, move to a pose solver that takes fx and fy separately.
Why subtract the principal point before calling cvPOSIT?
Because the function has no cx, cy input; its camera model puts the optical axis through the coordinate origin. Raw pattern coordinates carry an offset of hundreds of pixels, and the solver reads that as a large lateral shift of the camera relative to the model. Subtract the calibrated cx and cy from every correspondence, and undistort first when the calibration reports distortion coefficients.
How much does the wrong focal length change the pose?
Linearly in distance, barely at all in rotation. Translation scales with the value you supply: pass f one percent high and every recovered distance comes out one percent long. Passing fx at 1032.5 px when the mean is 1035.7 px shortens all distances by 0.31 percent, about 3.1 mm at a one-metre throw. Rotation depends on ratios within the image and stays essentially unchanged.
Can I skip projector calibration by pairing a depth camera?
Only if you do not need the projector's own frame. When the deliverable is a point cloud or depth map, a factory-calibrated structured-light 3D sensor delivers intrinsics and depth as one package and the calibration workflow disappears. Keep the hand-calibrated projector-and-camera rig when you must project onto a specific object and recover that object's pose - precisely the job cvPOSIT answers.