Training data for detecting partially occluded objects

The part reaches the camera half covered by its neighbor and the detector stays silent. The reflex is to blame the training data and go collect hard images before anyone has measured anything. Do it in the cheaper order: run a baseline test with the model you already have, label the occluded examples correctly if that test fails, size the occluded share of the dataset with real numbers, then stage a capture that produces those images on your own parts. The bench this assumes is standard machine vision: area scan cameras for the dataset, a smart camera for the on-device check and one of the vision controllers for the training run.

Will a model trained only on fully visible parts miss the hidden ones?

Sometimes yes and sometimes no, and the honest answer for your line comes from a measurement, not from a rule: convolutional detectors often carry moderate occlusion on their own, while misses, when they appear, cluster where less than half the part stays visible.

Detectors vote on local evidence: edges, corners, texture boundaries. Cover a fifth of a phone and the remaining four fifths still carry nearly every feature the network learnt, so the box often lands where it should. Cover half or more and the model is looking at fragments it never saw as positives, which is why misses, when they exist, pile up in the deep-occlusion band. Computer-vision references treat occlusion as a core failure mode of detection and tracking for exactly this reason.

That uncertainty is the argument against pre-collecting. A model that passes the test below costs nothing; a model that fails tells you which visibility band to buy data for.

Which baseline test do I run before collecting more data?

Hold out an occluded slice before you annotate anything: gather 50-100 production images where the part is genuinely covered, run your current model on that slice alone, and let its miss rate there — not a feeling — decide whether you need new training data.

Run the deployed model unchanged: same thresholds, same pre-processing, same confidence cutoff. Then work the steps:

1. Collect the slice: 50-100 images from production or a staged bench, with real occluders — neighboring parts, fixture walls, a gloved hand. No labels are needed for the test itself.
2. Record every miss and every box that lands on the wrong visible fragment.
3. Score each miss by visible fraction: visible height ÷ full part height at that location.
4. Read the pattern. Misses under about 2% above 50% visible but climbing below it: the data gap is real, continue with the sections below. Misses flat across every band: the problem is threshold, lighting or focus, and no dataset will fix it — tune first, then re-run this test.

Check the labels before blaming the model. A box that never covered the fragment in the first place is a labeling error, and Trained YOLO model has no detections: fix labels first walks that diagnosis.

How do I label a part that is half hidden?

Draw the box around the whole object, hidden area included, exactly as the standard YOLO-style labeling convention requires, and record the visible fraction next to it; a box clipped to the visible sliver teaches the model that the part is smaller than it is.

The box defines the object's extent, not the visible pixels' extent. Draw it around the full silhouette so every image of the class carries the same geometry, and the visible fragment teaches the network that a partial phone is still a phone. Clip the box to the sliver and you have taught the model that phones come 30 mm long.

Record the visible fraction per image to the nearest 5%, in the filename or a sidecar field: phone_vis060_0217.jpg, where 060 is the visible share, needs no extra tooling. The fraction exists so you can slice results by band later, and so the same tag filters truncated poses if you also run 6D work, as Visibility fraction is occlusion only: filter truncated poses too explains.

The tooling side — box placement, review passes, sampling — is its own workflow; Occluded object bounding box labeling for training covers the mechanics step by step.

How much occluded training data do I need?

Start at 10-20% of the training set with 30-50% of the part hidden: on a 500-image set that is 50-100 occluded images, and you raise the share only if the held-out miss rate is still above target after the first retrain.

Treat the split as the opening bid. It keeps the set dominated by clean images while the occluded share stays large enough to move the decision boundary, and it usually converges in one retrain. Move it in 5-point steps after that if the re-measured miss rate stays above target. The severity band matters more than the exact share: a model that has seen one 40%-covered part handles a 25% overlap and a 55% overlap far better than a model that only ever saw slivers.

The arithmetic is share × set size:

A 500-image set needs 50-100 occluded frames: 0.10 × 500 = 50 at the floor, 0.20 × 500 = 100 at the ceiling. Size the whole set first — How many images to train an object detection model works through the total, and the occluded share rides on that number.

What does a capture set-up for occlusion data look like?

Keep the production camera, lens, height and lighting fixed and stage the occluder instead: a simple fixture that slides a neighboring part across 30-50% of the target gives repeatable occluded shots at about 15-20 s per placement, so 100 images take under an hour of bench time.

Vary the occluder, never the optics: same camera model, same lens, same working distance, same lighting recipe as the production cell, or the network learns the bench instead of the line. Slide different neighbors across the part at 30%, 40% and 50% hidden, with a few 20% and 60% shots at the band's edges. Lock the lighting: When lighting changes force retraining a vision model prices that rule, and Lighting for AI vision training: stop retraining your model shows how to keep training light stable from the start.

Decide the smallest detail a label must resolve before you point a camera at the fixture. For a 150 mm phone behind a neighbor, give the labeler 10 px across a 1 mm visible edge — a 10 px/mm requirement. A 200 mm field of view then needs 200 mm × 10 px/mm = 2,000 px minimum width; the calculator below runs that check on your own part.

Worked example, sensor first: a 4000-px-wide chip on a 1/1.7" format is about 7.4 mm across, so ground sample distance = 200 mm ÷ 4000 px = 0.05 mm/px, or 20 px per mm — twice the requirement. The phone spans 150 mm ÷ 0.05 mm/px = 3000 px, and the 1 mm edge gets 20 px.

Lens next: magnification m = 7.4 mm ÷ 200 mm = 0.037. With the part plane 300 mm out, f = 300 mm × 0.037 ÷ (1 + 0.037) ≈ 10.7 mm, so the pick is the next standard up, a 12 mm C-mount lens, and the working distance moves to 12 mm × 1.037 ÷ 0.037 ≈ 336 mm. Export and label at full resolution even when the viewer shows a preview; Full resolution capture while showing a low res preview is the failure mode to avoid.

Two area scan cameras that cover this brief:

The LS-CMR-0166 lands on the worked example's numbers: 4000 × 3000 pixels on a 1/1.7" rolling-shutter sensor at 30 fps over USB3. The MV-CS200-10UM buys margin at 20 MP on a 1" sensor with a C-mount over USB3.0.

Can augmentation replace real occluded images?

Treat augmentation as a supplement, never the source: cut-and-paste and random erasing teach a network that boxes survive holes, but the occluders it then knows are rectangles and noise, not the parts, gloves and fixture walls your line actually produces.

The gap is distribution. A pasted rectangle carries no part texture, no cast shadow, no manufactured edge, and the network pockets the difference as a shortcut. Use augmentation to multiply what you staged: paste the real occluder crop from your own bench shots onto other backgrounds, which keeps the occluder statistics genuine while multiplying placements. Add background-only images without the part so cluttered scenes stop producing false positives; Negative samples reduce false positives: train with background images works that end of the problem.

One honest exception: if production occlusion never exceeds roughly 15-20% hidden, generic erasing augmentation often closes the gap alone, and the baseline test above will have told you so before you built a fixture.

Where do the frames and the retraining run live?

Keep the whole pipeline on one machine: raw frames on hard drive, labeled exports and the training run on SSD, and the final check on the hardware that runs in production, because a model that only passes on the training PC has not passed.

Storage arithmetic for the worked example: one raw frame is 4000 × 3000 px × 2 bytes = 24 MB, so a 500-frame set is 24 MB × 500 = 12 GB and a 2,000-frame set is 48 GB. Raw frames belong on the hard drive; exports and training data on the fast drive.

The MV-VC2140-128G40-1T carries that split in one box: a 128 GB SSD for exports, a 1 TB HDD for raw frames and 4 built-in light channels for the fixture.

Sets past a few thousand frames, or two cameras feeding one run, want more compute: the MV-VC5525-256G70-NN steps up to a 12th-gen Intel Core i5 with 32 GB of memory, PoE camera ports and 8 built-in light channels.

The last word on hardware is the device that runs the model in production. If that is a smart camera, re-run the held-out slice on it before sign-off: the MV-SC5060XM-08M-WBN runs that check on the device, with 6 MP mono, GigE and an 8 mm M12-mount lens.

What do I check before retraining?

Work the list top to bottom and retrain once: untouched held-out slice, 10-20% occluded share, whole-object boxes with visible-fraction tags, capture optics identical to production, and a final miss-rate measurement on the same slice you baselined.

• Baseline saved. The held-out occluded slice is untouched since the first measurement, with model version and thresholds written next to it.
• Labels consistent. Every occluded box spans the whole object, visible fraction tagged at 5% steps, no clipped boxes anywhere in the set.
• Share sized. Occluded images are 10-20% of training, severity concentrated at 30-50% hidden, validation mirrors the share.
• Backgrounds present. Background-only images sit in the set so cluttered scenes do not become false positives.
• Optics matched. Same camera, lens, working distance and lighting as the production cell, full-resolution exports only.
• Re-measured. After the retrain, run the same held-out slice and compare miss rate per visibility band. Accept when misses above 50% visible stay under 2% and the 30-50% band meets the line's target.

Close the loop where you started. A baseline that already passed means the budget goes to lighting stability and background samples, not occlusion data. If you are specifying the capture bench from scratch, the camera configurator screens sensors and lenses from the field-of-view and pixel numbers above, the surrounding capture-to-training guides sit on the applications topic page, and the contact page takes a baseline miss table if you want the split checked before you annotate.