Hand-eye calibration solver crash: diagnose the sudden shutdown

A calibration session that runs cleanly through every sample and then kills the whole tool the moment the solver starts is one of the most frustrating failures in vision-guided robotics. The GUI gives you no error, the process exits with a segmentation fault, and the log says nothing useful. This guide walks through the four causes we see in that exact pattern, ordered by how often each one is responsible, with a test for each and the fix that follows. It assumes an eye-to-hand or eye-in-hand set-up where a calibration tool collects robot poses and camera poses and solves the AX = XB system once enough samples exist. If you are still deciding how to collect good samples rather than debugging a crash, start with robot hand-eye calibration: how nine-point alignment works and eye in hand calibration pose selection: robot positions that work; this article is about why the solver itself dies and how to recover without losing your data.

The symptom and what it tells you

The classic signature: you accept the fourth or fifth sample, the tool silently starts the solver, and the whole process dies with a signal-11 exit code. Some implementations also log repeated "CameraInfo msg is NULL" errors earlier in the session — treat those as a red flag, not noise, because the same empty intrinsics message is often what the solver later dereferences.

The key insight is that the crash is rarely caused by the sample you just entered. Most hand-eye tools buffer samples and only run the solver once a minimum count (often three, sometimes five) is reached. The last sample is simply the first moment heavy matrix code executes. Whatever was wrong — missing intrinsics, corrupt poses, a degenerate set, or a broken numeric library — was wrong from the start and only surfaces now.

Work the causes in this order: camera intrinsics first, robot pose data second, sample geometry third, native library conflicts last.

Cause 1: the camera intrinsics message is empty

This is the most common cause and the easiest to check. The solver needs a camera matrix and distortion coefficients to convert image-space detections into camera-frame transforms. If the calibration tool subscribes to a camera info topic that publishes nothing, or publishes an uncalibrated message with a zero matrix, some implementations pass a null pointer or a zero matrix straight into native code instead of raising an error.

Stereo and depth cameras are the frequent offenders: their wrapper nodes often publish rectified images from factory calibration but either omit the camera info topic the tool expects, or publish it on a different namespace than the image topic. The tool then pairs your images with a nonexistent intrinsics message.

Test: from a terminal, echo the camera info topic for a few seconds while the camera streams. You should see a matrix with real focal-length values, not zeros. Then check that the image topic and the camera info topic carry the same frame ID and that the tool's configuration points at both.

Fix: calibrate the camera properly and publish the resulting intrinsics on the topic the tool subscribes to. If the camera node has an option to publish its factory intrinsics, enable it. Our article on single camera calibration when your device has no stereo pair covers getting trustworthy intrinsics out of a device whose wrapper does not expose them conveniently; rectified image pose estimation: choosing PnP intrinsics explains which matrix to feed downstream.

For a rebuild, a mono area scan camera with a proper calibration file removes this entire class of failure:

Cause 2: NaN or empty robot poses

The solver consumes robot poses as 4×4 transforms or quaternions. A pose that arrives with NaN in any component, a zero quaternion, or an unset frame produces garbage matrices, and matrix multiplication with NaN propagates into native code that expects finite values.

Test: print the pose the tool captured for each sample and inspect every component. A quaternion of (0, 0, 0, 0), a translation of NaN, or a timestamp of zero are all disqualifying. Also confirm the tool recorded the pose in the frame you think it did — the base frame, not a wrist or world frame; our camera pose initial guess: which reference frame it means explains how these frames get mixed up.

Fix: reject the bad sample, fix the driver or the frame configuration so poses publish finite values, and re-collect. If the robot driver intermittently publishes empty poses on startup, wait for a valid transform before capturing.

Cause 3: a degenerate sample set

The AX = XB system needs rotation about at least two non-parallel axes to be well-posed. If all your samples are small perturbations of one pose — a target photographed from nearly the same angle four times — the system becomes ill-conditioned. Depending on the implementation this produces either a wild result or a numeric failure inside the solver.

The spread matters more than the count:

• Rotate the target (or the wrist, for eye-in-hand) about two different axes, aiming for 30 to 90 degrees between the extreme orientations.
• Vary translation along at least two axes, not just one slide.
• Ten to seventeen good samples is a practical target; three is the mathematical minimum but leaves no margin to reject a bad one.

Test: compute the rotation between each pair of samples and check the axes are not all parallel. A quick numeric check: if every pairwise rotation axis lies within a few degrees of one line, the set is degenerate.

Fix: re-collect with the spread above, using the pose-selection guidance in eye in hand calibration pose selection: robot positions that work. A rigid target on a known pattern helps; a dot-grid calibration board such as the one below gives the detector stable corners from many angles.

Cause 4: conflicting numeric libraries (the silent segfault)

This is the cause behind the reports where the same matrix operation runs fine in a terminal but crashes inside the calibration tool. Hand-eye solvers lean on a BLAS implementation — the basic linear algebra subprograms underneath matrix multiplication and decompositions. When two incompatible copies of that library end up loaded in one process (one installed by the system package manager, another pulled in by a source-built dependency), the multiplication call can jump into mismatched machine code and segfault.

It is a build-environment problem, not a data problem, which is why your samples look perfectly good and the crash repeats at the same line every time.

Test: export the solver's inputs — the list of robot poses and camera poses — to a file. Run the exact same operations (the matrix multiplications and the final solve) in a standalone script outside the tool. If it crashes there too, you have isolated it. If it only crashes inside the tool, a library conflict is nearly certain.

Fix: make the whole stack use one consistent numeric library. In practice: remove or disable the duplicated copy, install a current release, and rebuild the calibration package and its math dependencies from source so they link against it. After rebuilding, rerun your standalone script before touching the GUI again. This sits in the same family of problems as hand-eye calibration build errors: resolve dependency mismatches and hand-eye calibration plugin load failure: diagnostic steps — when you build a calibration stack from source against a different environment than the one it loads at runtime, native code fails in ways the application cannot report.

Recover your samples instead of re-collecting

If the tool crashed mid-session, your samples may still be recoverable. Many calibration tools write each accepted sample (image, robot pose, camera pose) to disk as the session progresses. Check the tool's output directory before restarting the robot.

1. Locate the saved sample files from the crashed session.
2. Reload them into the tool, or feed them to your standalone solver script.
3. Solve offline and compare against a known reference transform if you have one.

Solving offline from saved samples is also the fastest way to validate a fix: once the library conflict is resolved, rerun the solve on the old data and see whether it completes. Keep that data after calibration too — vision calibration recovery after camera or robot changes explains how saved calibration artifacts shorten recovery when hardware moves.

Verify the fix end to end

A solver that completes is not yet a calibration that works:

• Check the reprojection or residual error the solver reports; for a well-collected set with a good target, expect small residuals, not orders of magnitude larger than your target size.
• Project a known robot-held point into the image with the resulting transform and confirm it lands on the target within a few pixels.
• Run the full verification procedure in eye-to-hand calibration: verify the camera-to-base result before trusting the transform in production.

For context on setting up the capture side — camera, target and mounting — the machine vision camera stands category covers rigid mounting, and the camera finder filters area scan cameras by interface and resolution once you decide to replace an inconvenient capture device. A stand like the one below keeps the camera rigid between sessions, which is what makes saved calibrations stay valid.

Checklist before you blame the solver

• Camera info topic publishes real intrinsics, same frame as the image topic.
• Every captured robot pose is finite and in the frame you intended.
• Sample set spans rotation about two non-parallel axes and translation along two axes.
• Solver reproduced standalone outside the GUI before changing system libraries.
• One consistent numeric library version across the whole stack, everything rebuilt against it.
• Samples saved to disk so a crash never costs a collection session.

If the crash persists after all four causes are cleared, the contact page reaches our applications team; bring the standalone reproduction script and the solver's saved sample file, because those two artifacts narrow almost any remaining case in minutes. For the wider picture of integrating the resulting transform with a robot cell, the systems topic page collects our related integration guides.