You captured forty chessboard views, findChessboardCorners found every one, cornerSubPix refined the corners, and calibrateCamera returned an RMS reprojection error near 0.2 px. Then you printed the intrinsic matrix and got values like 2.7e-10, 4.29 and 1.6e+15 where fx, cx and cy belong. Nothing in the calibration failed — the read-back did. This page is for anyone reading calibration output in C++: it shows why the returned matrix is CV_64F, what a float read does to its bytes, the one-line type check that proves it, the code that prints the matrix correctly, and the values a healthy 1280 × 800 calibration should show.
The symptom: sane RMS, a nonsense matrix
Everything else about the run reports success. All forty point sets were accepted, the distortion coefficients come out as plausible five-element numbers, and the returned RMS sits near 0.2 px — a strong result for a flat board. What average and maximum residuals should look like is its own subject; see camera calibration residual values: average and max targets.
Only the camera matrix is impossible: near-zero values in the 1e-10 class where fx and fy belong, a lone small number like 4.29, values around 1e+15 where cx and cy belong — instead of the expected [fx 0 cx; 0 fy cy; 0 0 1] layout. When RMS and distortion are sane but the matrix is garbage, suspect the read, not the optimization. Collecting more boards changes nothing: the stored bytes were never wrong.
Most likely cause: the returned matrix is CV_64F
calibrateCamera allocates its output matrices itself, and it returns the camera matrix as CV_64F — 8-byte doubles — even when you hand it a CV_32FC1 Mat. A declaration like Mat(3, 3, CV_32FC1) fixes the type only until the function re-creates the matrix with its own. The one-line test below confirms this on your build.
Reading the result with the float accessor then walks the same 72 bytes (nine doubles × 8 bytes) in 4-byte steps, so every printed value is half of a double. Your distortion coefficients printed correctly for the opposite reason: that Mat was declared CV_64F and read back as doubles, so each 8-byte value was read as one number. Matching the read to the stored type is the whole fix.
Why the print shows 1e-10 and e+15
Take a concrete value. For a 1280 px wide frame, fx near 1280.0 is a plausible calibration result — a horizontal field of view of about 53°. Stored as a CV_64F double, 1280.0 has the bit pattern 0x4089400000000000, which sits in memory as the bytes 00 00 00 00 00 40 89 40 on a little-endian machine. Read those 8 bytes as two floats:
- Bytes 0–3 = 0x00000000 → 0.0. With nonzero low mantissa bits this lands in the denormal range instead, which is where the e-10-class junk comes from.
- Bytes 4–7 = 0x40894000 → sign 0, exponent bits 10000001 = 129, mantissa field 606208. Fraction = 606208 / 8388608 = 0.072265625, so the float is (1 + 0.072265625) × 2^(129 − 127) = 1.072265625 × 4 = 4.290625.
One stored fx therefore prints as 0 or 4.29 depending on which half your accessor lands on. Because the float's 8 exponent bits sit on arbitrary double-mantissa bits, half-reads come out as denormals near 1e-10 or as monsters near 1e+15 — exactly the pattern in your console.
The one-line test: check the Mat type
Print the type code before reading anything:
std::cout << cameraMatrix.type() << '\n'; // prints 6 = CV_64F, 5 = CV_32F
A 6 means the function replaced your CV_32FC1 allocation with doubles, and no float accessor will ever print this matrix correctly. depth() returns the same code for single-channel data. The habit generalizes: boundingRect returns zeros or an error: Mat type and build is the same story one function over — the data was fine, the Mat type was not what the call assumed.
The fix: read doubles, convert only on demand
Leave the outputs empty, assert the type, read with the double accessor:
cv::Mat cameraMatrix, distCoeffs; // empty: calibrateCamera allocates
std::vector<cv::Mat> rvecs, tvecs;
double rms = cv::calibrateCamera(objectPoints, imagePoints, imageSize,
cameraMatrix, distCoeffs, rvecs, tvecs);
CV_Assert(cameraMatrix.type() == CV_64F); // test: 6, never 5
double fx = cameraMatrix.at<double>(0, 0);
double fy = cameraMatrix.at<double>(1, 1);
double cx = cameraMatrix.at<double>(0, 2);
double cy = cameraMatrix.at<double>(1, 2);
- Pass empty output Mats and capture the returned RMS.
- Assert cameraMatrix.type() == CV_64F before any read.
- Read fx, fy, cx and cy with the double accessor.
- Check the values against the sanity list below.
- Convert only if a consumer demands it: cameraMatrix.convertTo(cameraMatrix32f, CV_32F), after calibration.
The parameter list and CALIB_ flags are on the calib3d reference page. Pass the matrix downstream untouched; the undistort and solve-type routines accept CV_64F as returned. A few point-cloud helpers inherit the element type into their output — cv::rgbd::warpFrame, for instance, produces a double-typed cloud from a double camera matrix — so if the next stage insists on 32-bit data, convert once and keep the double original. When results go wrong further down the chain with a correct-looking matrix, the SolvePnP crop trap is the next thing to check.
Sanity values for a 1280 × 800 calibration
A healthy calibration of a 1280 × 800 stream shows:
- fx and fy within about 1–2 % of each other (square pixels). fx sets the horizontal field of view by FOV = 2·atan(width / (2·fx)): fx = 900 → about 71°, fx = 1280 → about 53°, fx = 1900 → about 37°.
- cx near 640 ± 5 % (± 32 px) and cy near 400 ± 5 % (± 20 px). Principal points further off-center usually mean decentered optics or a cropped sensor, not a type bug.
- RMS well under 0.5 px — yours already is.
fx and fy mean the same thing in factory-calibrated depth hardware; reading fx and fy from a ToF calibration model walks the identical matrix layout. Values outside a few hundred to a few thousand pixels on this sensor mean you are still reading the wrong type, or the board views were degenerate — all taken at one board angle leaves fx under-constrained.
The same bug elsewhere
This failure family appears wherever an image buffer changes hands, and the cure is always the same: know the element type where the buffer is created, read with the matching accessor. A colour frame that comes out doubled is Mat type against pixel format; a stereo frame saved wrong is an SDK buffer copied carelessly; 64-bit values in an int64 Mat is storage size against element size.
The trap reaches beyond image buffers. A client that reads a PLC REAL tag with the wrong data size prints the same tiny-and-huge garbage, and the fix is the same check on the other side of the wire: see this float data type mismatch fix for PLC tag reads.
Ordering the camera side of a calibration rig
The matrix is half the system; the other half is a camera whose sensor you can hold square to a flat board. The area scan cameras category lists the GigE and USB3 models used for chessboard work, the camera finder filters by resolution, shutter and interface, and the configurator pairs the camera with a lens and lighting once fx prints sanely.
For a dedicated rig, the LS-CMR-0166 is a typical choice: a 12 MP USB3 camera with an IMX226 sensor at 4000 × 3000, rolling shutter, 30 fps at full resolution — every frame a still, so rolling shutter costs nothing.
- Sensor format
- 1/1.7"
- Interface
- USB3
- Resolution
- 12 MP
- Shutter
- Rolling
If the type check passes and the values still sit outside the sanity list after a fresh capture, bring the printed fx, cx and RMS to the contact page and we will look at the board geometry with you.
Questions
Does calibrateCamera ever return a CV_32F camera matrix?
No. Whatever type you pass in, the function allocates and returns its camera matrix, rotation and translation vectors as CV_64F. A CV_32F matrix is only useful downstream, and then you convert it once, explicitly, with convertTo after calibration. If your printed values are garbage, the matrix itself is still CV_64F — the read is what failed.
What values should fx, fy, cx and cy have for a 1280 × 800 image?
fx and fy should be close to each other, typically between about 900 and 1900 pixels on a 1280 px wide frame — wider lenses sit near the low end, tighter lenses near the high end. cx lands near 640 and cy near 400, usually within about 5 % of the image center unless the sensor is cropped or the optics are decentered.
Is a reprojection error of 0.2 pixels good?
Yes. For a flat chessboard shown sharp, well filled and at varied angles, per-view errors under about 0.5 px are normal and under 0.3 px is a strong result. Since the RMS is already there, the optimization succeeded; the nonsensical matrix on the console is an output-reading bug, not a reason to collect more boards or change flags.
Why did my distortion coefficients print correctly?
Because that Mat was declared CV_64F and read back with the double element type, so every 8-byte value was read as one number. The camera matrix failed only because it was declared CV_32FC1 and read with the 4-byte float accessor after the function had replaced its data with 8-byte doubles. The matrix data was never the problem; only the element size of the read was.
Can I pass the CV_64F camera matrix straight to undistort or solvePnP?
Yes — undistort, initUndistortRectifyMap and solvePnP all accept CV_64F, so pass the matrix as returned. Converting is only needed when a later stage builds a point cloud and inherits the element type, or a function demands 32-bit data; then call convertTo to CV_32F once after calibration. Keep the double original and feed the float copy only where it is required.