Align images before background subtraction: garment masking

Two studio shots of the same mannequin — one dressed, one bare — should differ only in the garment. They never quite do: the camera moved a few pixels between frames, so a plain subtraction lights up the mannequin's own edges, the stand and the backdrop seams, and the mask comes out useless. The steps below estimate that shift from the static parts of the scene, move one frame back into register, and threshold the difference into a clean garment mask. You also get the displacement budget: the arithmetic that tells you, before you shoot, how much camera movement a translation-only correction absorbs for your working distance and scene depth. Everything runs in a standard vision library on a PC, or on an x86-class smart camera, and it assumes nothing about the garment except that it was absent in the second frame.

Why a few pixels of camera shift break a plain difference

A difference image assumes pixel (x, y) shows the same scene point in both frames. Move the camera between shots and every scene point lands a few pixels away, so the difference flags the static content — mannequin edges, stand, backdrop seam — as strongly as the garment. That is the ghosting: a halo around every silhouette you wanted to cancel. The same fixed-viewpoint requirement underlies frame differencing for motion detection, which subtracts two frames from one camera that must not move.

Optics fixes the size of the shift. A camera that translates sideways by Δx at working distance d displaces the image of content at that distance by f × Δx, where f is the focal length:

Two consequences drive the whole job. The shift is proportional to the bump: a 2 mm nudge at 600 mm with a 16 mm lens already moves the image about 15 pixels. And the shift varies with depth — nearer surfaces move more than far ones, which is why image registration is its own field. A dressed mannequin in front of a backdrop spans a few hundred millimeters of depth, so no single offset aligns both planes exactly. The residual between the planes decides whether translation alone survives:

Here d_near and d_far are the depths of the nearest and farthest static surfaces. Hold the residual at or under about 1 pixel and the leftover halo stays thin enough for morphology to strip; past 2-3 pixels, fix the rig instead of the image. The displacement budget at the end of this guide gives the number for your setup.

One point saves most of the pain: the garment never needs to align. It exists in only one frame, so it shows up in the difference no matter what. Only the static surfaces — mannequin, stand, backdrop — have to cancel.

What you need

• The two frames, dressed and bare, in identical pose. Accessories must match: jewelry and footwear belong in both shots or in neither, or they land in the mask.
• Locked capture settings. Manual exposure and manual white balance, set before the dressed shot and never changed; fixed studio lights and no daylight in the mix.
• A PC with any vision library that offers feature detection, descriptor matching, image warp and morphology — or an x86-class smart camera if the inspection must run on the device.
• Approximate focal length, working distance and pixel pitch, for the cross-check in the next section.
• A rough garment location. Even a hand-drawn rectangle is enough to keep the fabric out of the matching step.

Step 1: estimate the offset from the static content

1. Keep the dressed frame fixed as the reference; the bare frame is the one that moves.
2. Exclude the garment from matching. Fabric that exists in only one frame produces wrong match vectors, and even a handful of them drags a least-squares fit. Your rectangle from the list above marks the exclusion zone.
3. Detect features on the remaining static area: mannequin head, arms, stand, any textured backdrop or a patterned target taped at mannequin depth. On a smooth studio set the mannequin itself supplies the texture; if it is glossy and featureless, add the target.
4. Match descriptors between frames and keep mutual nearest neighbors. Convert each surviving match into a shift vector (dx, dy), take the median of the set, and keep only matches within a few pixels of that median. The median shrugs off outliers; a plain average does not.
5. Fit the translation to the survivors — a least-squares offset or simply the median itself. With a few hundred good matches the estimate typically lands within a few tenths of a pixel, well inside the residual budget.
6. Warp the bare frame by the negative of that offset, with bilinear interpolation. Sub-pixel offsets are normal here; the slight blur does no harm in a difference image.

Predict the offset before you measure it, then compare. With f = 16 mm, a 2 mm bump at d = 600 mm gives 16 × 2 / 600 = 0.0533 mm on the sensor, which is 15.5 pixels at a 3.45 µm pitch. A measured offset near that value confirms a pure translation. A much larger one points at rotation or a false match on repeated texture; a near-zero one means focus or zoom changed instead. The worked example below runs the full arithmetic.

For the pixels-per-millimeter side of that cross-check, the field-of-view calculator below turns focal length, sensor and working distance into the field of view; divide your horizontal pixel count by it to get pixels per millimeter at the mannequin.

Step 2: difference, threshold and clean the mask

7. Take the absolute difference per channel between the dressed reference and the shifted bare frame. For a colored garment on a neutral mannequin, convert both images to Lab first and difference the a and b channels: shading and gray-level drift barely register in chroma, and automatic threshold segmentation in Lab color with Otsu's method handles the split.
8. Threshold the difference. Start with Otsu on the histogram; fall back to a fixed level if Otsu latches onto shadow instead of fabric. Working without a library, threshold basics covers the trade-off.
9. Strip the residual halos. A misalignment of 1-2 pixels leaves ridges of the same width along static edges. Remove them with a morphological opening of 2-3 pixels, or with a two-level threshold that admits weak difference pixels only where they touch the strong garment core. Keep kernels small: an aggressive opening eats lace trim and thin straps.
10. Close pinholes, drop components below a size floor, and keep the largest component as the garment mask. A hole appears where garment color matches the mannequin behind it — the difference is genuinely small there — so fill interior holes rather than raising the threshold until they disappear.

Step 3: verify the mask before you ship it

• Overlay the mask contour on the dressed frame. The edge should hug the garment silhouette, with no straight lines or offset bands betraying a bad warp.
• Sample the difference along a known static edge, such as the mannequin's shoulder or the stand. After cleanup, a healthy registration leaves no mask fringe wider than about 2 pixels there.
• Toggle the mask over the whole frame. Blobs away from the garment mean lighting drift, a moved accessory or a pose change. No offset removes those; reshoot them.
• Compare the mask area against a hand outline once, on a few frames, to set your acceptance band. After that, a plain area check catches gross failures in production.

The displacement budget: how much camera move a translation absorbs

Worked example, on a common 5 MP camera of the 2/3-inch class: 2448 × 2048 pixels at a 3.45 µm pitch, a 16 mm lens, the mannequin at 600 mm, a backdrop 300 mm behind it, and a 2 mm sideways bump between shots.

Shift at the mannequin's depth: 16 mm × 2 mm / 600 mm = 0.0533 mm on the sensor, and 0.0533 mm / 0.00345 mm per pixel = 15.5 pixels.

Cross-check through the field of view: active width = 2448 × 3.45 µm = 8.45 mm, so FOV = 8.45 mm × 600 / 16 = 317 mm, and 2448 pixels / 317 mm = 7.7 pixels per millimeter. A 2 mm bump at that rate reproduces the 15.5-pixel shift within rounding.

Shift at the backdrop: 16 mm × 2 mm / 900 mm = 0.0356 mm, or 10.3 pixels. Register on the mannequin plane and the backdrop keeps a residual of 15.5 − 10.3 = 5.2 pixels — too much for a textured backdrop, harmless on plain studio paper. Put the leftover on whichever static plane is smoothest.

The budget for holding the residual to 1 pixel:

Δx_max = 0.00345 mm × 600 mm × 900 mm / (16 mm × 300 mm) = 0.39 mm. With this lens, sensor and scene, the camera must stay within about 0.4 mm of its first position; a hand brushed against the tripod exceeds that. What moves the number:

• Backdrop 100 mm behind instead of 300 mm: 0.00345 × 600 × 700 / (16 × 100) = 0.91 mm — more than twice the freedom.
• A 30 mm lens instead of 16 mm: the budget scales by 16/30, down to about 0.21 mm.
• Pixel pitch and working distance scale it roughly in proportion: coarser pixels and longer distances forgive more.

Decision: if you cannot hold the bump under your budget — and in a studio with people around, you usually cannot guarantee it — prevent rather than correct. Lock the rig and shoot the pair untouched. Reach for a model with scale terms (affine, keystone) only when the geometry forces it and you have hundreds of good matches on both depth planes; a camera tilt that changes scale with depth is the one thing a translation can never absorb.

Shoot the pair so registration has less to do

Prevention beats correction by a wide margin. Clamp the tripod head, trip the shutter remotely and do not touch the camera between frames. Fit a manual lens and tape the focus and iris rings: refocusing alone rescales the image by a few pixels even when nothing else moves. Keep the lights fixed and block daylight, which drifts during a shoot.

Any color area scan camera or smart camera covers the capture; resolution only sets how fine the mask edge lands. Both cameras below take C-mount lenses, so you can fit a manual-iris model and lock it down.

The MV-CU020-90GC pairs a 2.5 MP global-shutter sensor with a C-mount and suits a torso at these working distances; the MV-CS200-10GC resolves 20 MP over a 1-inch sensor when lace and printed detail must survive into the mask.

If the mask must be produced on the device instead of a PC, the camera below runs an x86 platform with an 8.8 MP global-shutter sensor and a C-mount, so the same pipeline executes where the image is captured.

Common mistakes

• Matching on the garment. It changed between frames, so its match vectors are wrong by construction. Exclude it before detecting anything.
• Auto exposure or auto white balance left on. The whole frame shifts between shots; see the warning above.
• Refocusing between shots. Focus breathing rescales the image from the center outward, and no translation removes a scale change. Symptom: a near-zero measured offset with halos that grow away from the image center.
• Trusting translation after a tilt. Symptom: the residual grows from one side of the frame to the other. Level the camera or move to a model with scale terms.
• Morphology that eats the garment. Kernels much above 3 pixels thin lace, straps and hems. Keep the opening at 2-3 pixels and let the size floor, not the kernel, do the cleaning.
• Chasing shadow with the threshold. If the level that catches a pale hem also captures the shaded torso band, fix the lighting or move to chroma channels; do not tune past the point where static edges reappear.

From mask to cut-out

Apply the mask as an alpha channel over the dressed frame and crop to its bounding box with transparent padding; the polygon mask crop with a transparent background guide shows that final step. If your problem is the reverse geometry — a fixed camera over a scene that itself moves — the guide on object detection on a moving background with registered subtraction takes that case.

For a fixed studio rig, start at the camera finder or step through the configurator. Questions about a specific bench reach the applications team through the contact page.