Binary image registration: compute the 2D displacement vector

A hand-held or re-mounted camera produces binary masks of the same scene a few pixels apart, and the control loop needs one number from them: the (dx, dy) that maps the first mask onto the second. Intensity feature matching fights you here — SIFT and SURF have almost nothing to describe in a two-valued image. A full affine fit is more freedom than the problem has. This guide shows the pipeline for exactly that case: detect blobs, vote on candidate translations, confirm with RANSAC arithmetic, refine to a tenth of a pixel, and convert the result to millimeters on the part. It runs in any language with an FFT and a connected-component labeling, on a vision controller next to the line. The worked example ends at a committed vector: (+37.2, −12.6) px, which is (+2.01, −0.68) mm on the part.

Why do SURF and SIFT fail on a binary image?

Intensity descriptors such as SURF and SIFT fail on binary masks because they summarize local gradient texture, and a two-valued image has almost none. Nearly every patch produces a near-identical descriptor. The matcher returns scattered pairs and the affine fit inherits the noise.

SIFT and SURF build a histogram of gradient orientations over local patches. In a mask, the interior is constant and every gradient lives on a one-pixel outline. The outline of a hole and the outline of a silhouette corner can produce nearly the same descriptor. Match scores come back weak and spread across the frame, and a transform fitted to those pairs wanders with them.

Two exits exist. If the grey frames the masks came from still exist, run the descriptors there and transfer the transform to the masks. If only masks survive — for instance when an on-camera tool emits them directly — stop feeding descriptors two-valued input and match the geometry the mask already contains.

What should you match instead of intensity features?

Match the geometry the mask already contains: connected-component centroids with their areas, or Harris corners along the mask outline when blobs are scarce. A 5 MP mask of a machined part typically yields 30-150 stable blobs, and each centroid repeats to a fraction of a pixel.

Connected-component labeling gives every blob a centroid, the mean of its pixel coordinates and fractional by construction, plus an area to match on. Boundary flicker barely moves it: flips of half a pixel shift a blob's centroid by about perimeter × 0.5 / area in pixels. That is 71 × 0.5 / 400 ≈ 0.09 px for a 400 px blob but 0.4 px for a 20 px speckle. Keep blobs of 20 px area and up to kill speckle. Match on blobs of 400 px and up for sub-pixel stability.

Sort by area and keep the largest 60 per frame, and the vote stage stays bounded without losing the anchors. Measuring blob area in a binary image covers the area side. When a scene segments into few blobs — long edges, no islands — fall back to a Harris corner detector on the mask outline. Corners carry coordinates but no area, so matching leans entirely on the consensus fit.

Should you run phase correlation or blob voting?

Run phase correlation when the second mask is the same scene, purely shifted — overlap above 70% and rotation under about 0.1°. Switch to blob voting with RANSAC when blobs appear or vanish or the camera may have turned: voting survives content change, correlation does not.

Phase correlation is one FFT pair. Transform both masks, whiten the cross-power spectrum, invert, and a single peak marks the integer shift. Upsampling the correlation peak refines it to 0.1 px at an upsample factor of 10. It is indifferent to uniform intensity change, so a mask re-thresholded from the same grey frame still correlates. But it assumes one scale and a small angle, and rotation or noise smears the peak.

Crop both masks to the shared region first — a shifted border step biases the peak, the same ROI discipline that breaks template matching when the template is smaller than the search area. Blob voting needs no pixel-wise common content, tolerates blobs appearing or vanishing, and extends to rotation later. That makes it the default for a hand-held camera. If the stage moves on one axis only, sum each mask's columns into a 1-D profile and correlate the two profiles — far cheaper than a 2-D search; the row-sum profile method shows the mechanics.

How many RANSAC trials does a translation-only fit need?

A pure translation has two unknowns, so one matched point pair defines a candidate model, and the trial count for 99% confidence at inlier fraction w is N = ln(1−p)/ln(1−w); at w = 0.78 that is 4.61/1.51 ≈ 3.0, so 4 trials.

Here w is the fraction of candidate pairs that are true correspondences and p is the confidence of drawing at least one all-inlier sample — for a translation, a single pair. Even the pessimistic cases stay small:

Brute-force voting is already a consensus method — every pair votes and the histogram peak is the model — so RANSAC is optional there. Keep it when you want a formal inlier set for the refit, or when you match corners with no area attribute and need distance-gated hypothesis testing.

What are the steps from two masks to one displacement vector?

Label both masks, vote every centroid pair's offset into a 2-pixel histogram, take the peak and refit the median over its supporters: 58 × 61 = 3,538 candidate pairs are enough to pin the worked example at (+37.2, −12.6) px.

1. Label connected components (8-connectivity) in both masks; drop blobs under 20 px area.
2. Compute centroid and area per blob; sort by area and keep the largest 60 per frame.
3. Vote every pair: add (b.cx − a.cx, b.cy − a.cy) to a 2-D histogram with 2 px bins. The worked set casts 58 × 61 = 3,538 votes.
4. Take the peak bin: 43 votes at bin (19, −6), centre (38, −12) px, against 6 votes for the runner-up — a ratio of 7.2.
5. Collect pairs within ±2 px of the peak and refit the translation as the per-axis median: (+37.2, −12.6) px over 45 inliers.
6. Gate on support: at least 70% of kept blobs must find a partner within 3 px after the shift; here 45 of 58, or 78%.
7. Refine on the overlap with upsampled phase correlation, factor 10, for 0.1 px steps; the vector holds at (+37.2, −12.6) px.

# Illustrative pseudo-code: these variable names are our own, not a library API.
blobs_a, blobs_b = label(mask_a), label(mask_b)  # each blob: (cx: float, cy: float, area: int px), area >= 20
votes = {}                                       # dict keyed by (dx_bin: int, dy_bin: int), value: int count
for a in blobs_a:
    for b in blobs_b:
        key = (round((b.cx - a.cx) / 2), round((b.cy - a.cy) / 2))
        votes[key] = votes.get(key, 0) + 1
peak = max(votes, key=votes.get)                 # worked: (19, -6) -> centre (38, -12) px with 43 votes

The test that proves the result: 43 votes in the peak bin against 6 in the next (ratio 7.2), and 45 of 58 centroids supported within 3 px after the refit. Both gates pass, so the vector ships.

How do you convert the vector from pixels to millimeters?

Multiply the pixel vector by your millimeters-per-pixel value: on the worked station, a 132 mm field of view across 2,448 pixels gives 0.0539 mm per pixel, so (+37.2, −12.6) px becomes (+2.01, −0.68) mm — a 2.12 mm move on the part.

Worked station, illustrative values: 2448 × 2048 mono camera, 3.45 µm pixels, 16 mm lens, 250 mm working distance. Sensor width is 2448 × 3.45 µm = 8.45 mm, so the field of view is 8.45 mm × 250 / 16 = 132 mm and one pixel is 132 / 2448 = 0.0539 mm. The pixel vector converts as 37.2 × 0.0539 = 2.01 mm and 12.6 × 0.0539 = 0.68 mm, a straight-line shift of √(2.01² + 0.68²) = 2.12 mm. The 0.1 px refinement step is worth 5.4 µm here.

Working backwards: a task that needs 0.05 mm per pixel across a 132 mm field calls for 132 / 0.05 = 2,640 pixels. The calculator below sizes the camera:

How much compute does this add per inspection cycle?

Budget roughly 0.1-0.3 s of single-core CPU for a full 5 MP phase-correlation pair and a few milliseconds for the blob vote: three 2560 × 2048 transforms cost about 1.8 GFLOP, which a desktop-class core at 5-20 GFLOPS clears inside that window.

Phase correlation pads each mask to a fast FFT size — 2560 × 2048 for a full 5 MP frame — and runs three transforms (two forward, one inverse). At 5N log2(N) floating-point operations with N = 5.24 million pixels, each transform costs about 0.6 GFLOP, so the pair costs about 1.8 GFLOP. The 3,538-vote histogram adds milliseconds. Halving both axes first leaves about a fifth of the work — roughly 20-60 ms — at the price of doubling the pixel step to 0.108 mm; downscaling after capture covers that trade.

The controllers below carry this load with room for the inspection itself.

The MV-VC3301H-128G66 pairs an Intel Core i3 with 8 GB of memory, 5+ LAN ports and 4 built-in light channels. Its PCIe expansion leaves room for extra frame-grabber interfaces.

The MV-VC5525-256G70-NN steps up to an Intel Core i5 with 32 GB of memory, PoE camera ports and 8 built-in light channels for multi-camera cells.

The MV-SC5060XM-08M-WBN is a 6 MP mono smart camera with an 8 mm M12-mount lens and a GigE interface. Its onboard matching handles feature positions at the camera; route the whole-mask vote and FFT to a controller. The smart cameras range covers the rest of the on-device inspection cases.

When does translation-only registration give the wrong answer?

Translation-only breaks once rotation or distance change creeps in: at 0.5° the corners of a 2,448-pixel frame sit about 11 px from where a pure shift puts them, and a 5% working-distance change moves them about 61 px — both smear the single-shift model.

The differential is arithmetic, not a rule of thumb. Rotation θ in radians displaces a point at frame half-width 1224 px by θ × 1224 px, so 0.5° = 0.00873 rad × 1224 ≈ 11 px. A scale error s moves the same point s × 1224 px, so a 5% distance change scatters the frame by about 61 px. Invert for your tolerance: a 2 px tolerance allows θ up to 2/1224 = 0.0016 rad ≈ 0.1°, and scale up to about 0.16%.

Below that, the translation model holds. Between about 0.1° and 1°, fit a rigid model — rotation plus translation — on the same matched centroids. The minimal RANSAC sample becomes two pairs, so trials rise to ln(1 − p)/ln(1 − w²), which at w = 0.78 is 4.61/0.937 ≈ 4.9, so 5 trials. Beyond 1°, or when the two images come from different lenses or working distances, no rigid model fits: re-mount the camera or calibrate scale per shot.

How do you verify the vector before you trust it?

Prove the pipeline on a commanded move before trusting an unknown one: shift the part exactly 1.00 mm and expect 1.00 / 0.0539 = 18.6 px back. Accept a result only when the top bin holds at least three times the runner-up and 70% of blobs find partners.

Acceptance gates, per run:

• Peak bin holds at least 10 votes and at least 3× the runner-up (worked: 43 vs 6).
• At least 70% of kept blobs find partners within 3 px (worked: 78%).
• Inlier offsets sit within 1 px RMS of the per-axis median.
• A commanded 1.00 mm move returns 18.6 ± 0.4 px on both axes, a 2% gate.
• The rotation differential, θ × 1224 px, stays inside the application tolerance.

After two frames register, change detection downstream can finally assume alignment: the two-buffer frame differencing pattern breaks exactly when registration drifts, so run it after this pipeline. Watch the inputs too — a live binary preview catches broken masks before they vote.

The decisions, in one line each. Fixed station, overlap above 70%, rotation under 0.1°: run phase correlation with upsample factor 10 and gate on peak ratio. Hand-held camera or changing content: vote blob centroids, refit the per-axis median, verify with the commanded-move test, and escalate to a rigid fit once θ passes 0.1°. For the parts that run it, browse the vision controllers or start at the systems topic page.