Open-vocabulary detection class mismatch after fine-tuning

Your fine-tuned detector crashes at inference with RuntimeError: shape '[1, 65, -1]' is invalid for input of size 441600, and the traceback ends inside the prediction head's reshape. The weights are fine. Fine-tuning on the five classes in your dataset YAML rebuilt the head for exactly those five names, and your predict script registered only one, so the head and its class bookkeeping disagree about how many class channels every anchor carries. What follows works the arithmetic behind the two numbers in that message, ranks the causes, gives the fix in a few lines, and states the decision for every case where you need a different vocabulary. It is written for engineers deploying trained detectors on lines fed by area scan cameras and smart cameras, with inference on a vision controller.

The answer: register the trained names, all of them

Pass the registration call the complete class list you trained on — here the five names from the dataset YAML: OK, quemian, dirty, toufa and heijietou, in that order, because the list position assigns the class id of every result. If the station only acts on one of them, register all five and filter the output; never trim the input list. A fine-tuned checkpoint also runs with no registration call at all, since the trained names travel inside the weights and a plain predict call uses them.

names = ["OK", "quemian", "dirty", "toufa", "heijietou"]  # the YAML list, verbatim
model.set_classes(names, model.get_text_pe(names))
results = model.predict(source)
# need one class? filter results by class id, never trim the prompt list

Class lists belong with the rest of the task settings, kept in one place between training and serving — vision model task configuration for reliable inference covers that discipline.

Why the class count breaks the reshape

A pretrained open-vocabulary detector scores each candidate region against text embeddings of whatever names you hand it, so any number of names works. Fine-tuning on a fixed dataset ends that. The trainer reshapes the prediction head to the N classes named in the YAML and stores the prompt embeddings for those N names inside the checkpoint; from then on the model is a closed-set detector for exactly that list.

At inference the registration call writes the new name list into the model's bookkeeping, including the per-anchor channel count the post-processing reshape expects. The forward pass still produces the trained channel layout. When the two counts differ, the reshape that flattens each feature level cannot divide the tensor evenly and stops with the shape error — on this stack it fires during warmup, on the 640 px dummy pass, before your first real image.

N is the class count the bookkeeping holds after the registration call, and M is the segmentation head's mask-coefficient count — zero on box-only heads.

Worked example: 441 600 elements against 65 channels

Inputs, each taken from the failing set-up: inference size S = 640 px, the warmup default; a three-level head at strides 8, 16 and 32, so grids of 80 x 80, 40 x 40 and 20 x 20 — 6 400, 1 600 and 400 anchors; five trained classes; one registered name; and an error that expects 65 channels per anchor over a 441 600-element tensor.

The arithmetic:

1. The failing tensor is the first level, the stride-8 grid: 441 600 / 6 400 = 69 channels actually produced per anchor.
2. Those 69 split as 4 box + 5 classes + 60 mask coefficients — the trained layout, five class channels wide.
3. The reshape expects 65 = 4 + 1 + 60: registering one name rewrote the bookkeeping to a single class, four channels fewer.
4. 65 x 6 400 = 416 000, not 441 600, and 441 600 / 65 = 6 793.8 — not a whole anchor count — so the view is refused.

The decision: produced 69, expected 65, difference 4 — exactly the four names the script dropped, since 5 - 1 = 4. Register five names and both counts read 69; 69 x 6 400 = 441 600 divides cleanly and inference runs. The tolerance is zero: a mismatch of even one name, in either direction, fails this reshape.

Causes in order, with the test that separates them

Rank 3 is the dangerous one:

The fix, step by step

1. Open the training YAML and copy the names block verbatim — five entries here, 0: OK through 4: heijietou.
2. Load the fine-tuned checkpoint: the best weights from the training run, not the pretrained file you started from.
3. Call the registration once per model load with the full list, in YAML order.
4. Predict, and confirm the results carry ids 0 through 4.
5. Filter to the class the station needs by id, in post-processing.
6. Validate on a hand-labeled batch: each id must resolve to the name you expect before the station gates product.

Success looks like this: the warmup pass completes, the first image returns boxes with ids 0-4, and filtering to one id leaves the other detections untouched.

When you need a different vocabulary

Fine-tuning closes the vocabulary — that is its job. Pick your case:

• Fixed line, fixed defect set — this case. Treat the model as closed-set: register the five names or skip the call, filter at the output, and re-check the mapping after every retrain.
• Occasional new names, no retraining. Keep the pretrained checkpoint for those runs — its head was never reshaped, so text prompts of any length still work. Run it beside the fine-tuned model, not in place of it.
• The five classes plus new ones. Retrain with the expanded YAML: six names rebuild the head at 70 channels per anchor (4 + 6 + 60), one channel more, so training time and memory barely move. Training anomaly detection across multiple product classes covers the dataset work that adding classes drags along.

One dataset decision belongs here. The YAML above carries an explicit OK class for good parts; good parts can instead enter as unlabeled background frames, and background images in object detection training: why no labels explains when negatives need no labels at all.

Where the model trains and where it runs

The training run behind the error is modest: a 10.1 million parameter segmentation model, batch 4, AdamW at a 1e-3 learning rate, 150 epochs, on a 48 GB workstation GPU. Parameter count does not size that machine — activations and optimizer state do, which is why small models still fill large cards; why small neural network models use large GPU memory works that math. For the machine itself, GPU workstation sizing for fine-tuning and inference budgets the memory for segmentation models of this size.

Serving is lighter. A 10 million parameter head runs comfortably on fanless vision controllers at line rates. The camera side does not change: the area scan cameras that shoot the line today keep feeding the model, and the camera finder is the place to start when the station is new. When the class set never changes at all, on-device inspection in smart cameras sidesteps the whole vocabulary question.

A controller such as the MV-VC3301H-128G66 carries the retrained head with capacity to spare for the camera links and the reject I/O:

To compare the range directly:

Checklist before the next run

• Registration list copied from the training YAML — same count, same order.
• Checkpoint path points at the fine-tuned best weights.
• One registration call per model load, never per frame.
• Id-to-name mapping confirmed on a hand-labeled image.
• Output filtered by class id, not by trimming the vocabulary.
• Training and serving package versions aligned.
• Good parts handled deliberately: a labelled OK class or unlabeled background frames, never an accident.