Bounding boxes can look plausible while referring to the wrong pixels, frame or class. This makes output-contract errors harder to spot than a failed inference call. For inspection pipelines, separate tensor decoding from coordinate conversion and display rendering before changing the model. The checks below help you locate the first stage that disagrees with a saved reference frame, then verify the correction without relying on a convincing overlay.
Start with the symptom, not the model
Keep the camera stationary and process one saved image repeatedly. This removes changing scenes and acquisition timing from the first investigation. Inspect numerical detections before drawing anything.
Use this order to narrow the fault. It prioritizes inexpensive checks, not a universal ranking of failure rates.
| Symptom | Check first | Discriminating test | Corrective action |
|---|---|---|---|
| Boxes have consistent vertical displacement | Letterbox padding | Draw boxes on the padded model input | Subtract padding before reversing resize |
| Boxes stretch differently across the image | Independent horizontal and vertical scaling | Compare input and source aspect ratios | Reverse the actual resize operation |
| Boxes appear transposed or inverted | Coordinate order and box representation | Decode one record manually | Match the documented field order |
| Positions look right but labels are wrong | Class identifiers and label mapping | Print numeric identifiers beside labels | Use the matching label map |
| No detections survive | Parsing, score interpretation and filtering | Count records before each filter | Correct the first stage losing valid records |
| Boxes trail moving objects | Frame association | Compare image and inference frame identifiers | Render detections on their originating frame |
A display error is not evidence that the detector needs retraining. Establish which stage first produces an incorrect value.
Freeze the image and write down the contract
Save the exact source image, the actual preprocessed input, raw output tensors and decoded detections from the same inference. Record tensor shapes and data types, not just printed values.
Your diagnostic record should include:
- Model artifact identity and the associated configuration version.
- Source image width, height, pixel format and frame identifier.
- Crop origin, resized dimensions and padding on each side.
- Input tensor axis order, channel order and numerical scaling.
- Output tensor names, shapes, field meanings and coordinate units.
- Class mapping, score interpretation and suppression settings.
- The coordinate frame promised by the postprocessor's output.
Do not infer that contract from a model filename. Export options can change output layouts or include postprocessing inside the exported graph.
If the wrong artifact might be loading, resolve model file paths and deployment errors before adjusting coordinate arithmetic. When the saved image itself looks wrong, use the frame-conversion validation checks first.
The verification target is repeatability: the same saved input and configuration should produce equivalent decoded records within the runtime's expected numerical behavior.
Verify tensor layout before flattening anything
Read the documented output layout before indexing its first record. A tensor containing candidate rows is different from separate tensors for boxes, scores, classes and valid counts. Raw prediction heads need their own decoding; they are not necessarily ready-to-draw detections.
For example, a candidate tensor could contain 100 rows with 6 fields per row. A transposed tensor could contain the same 600 values, but arrange those fields along a different axis. Flattening either tensor produces an equal-length array while discarding the distinction.
Test the parser with one manually inspected record. Confirm all of the following:
- The batch dimension selects the intended image.
- The parser uses the correct candidate and field axes.
- Any valid-count tensor limits iteration correctly.
- Each count is finite, integral and within the allocated capacity.
- Coordinates and scores are interpreted using the documented data type.
- Quantized outputs are converted using the provided quantization metadata when required.
Where affine quantization applies, the usual conversion is:
Use the metadata for that output, including per-axis parameters where applicable. Do not apply this conversion again to values the runtime already exposes as floating-point data.
Reject malformed records with a clear diagnostic. Silently stopping halfway through a tensor can make a broken parser appear merely conservative. Likewise, an all-zero remainder is not a universal end marker; use documented counts or padding rules.
Check class mapping and confidence filtering separately
Correct box locations do not prove that class labels are correct. A model with 4 foreground classes might use identifiers from 0 through 3, but another export can reserve an identifier for background or retain sparse dataset identifiers.
Print the raw identifier, mapped label and score together. Check several known objects rather than only the first label. A consistent one-position label shift suggests an indexing mismatch, not a coordinate problem.
Do not treat a missing label as permission to invent a new class. A diagnostic label such as unmapped_class can expose the fault, but the inspection decision should remain invalid until mapping is corrected.
Next, trace candidate counts through filtering. An illustrative log might show:
- 12 valid decoded records.
- 7 records after the confidence threshold.
- 5 records after duplicate suppression.
This identifies where records disappear. Temporarily lower the threshold on a saved diagnostic image, then restore the validated operating threshold before deployment.
Confirm what the score means. Some outputs provide a final detection score; others expose separate objectness and class values or unnormalized logits. Multiply or transform them only when the output contract requires it. Applying an extra score operation can remove otherwise valid detections.
Also determine whether suppression has already occurred. Running another suppression stage can change counts, particularly when its class handling differs from the original stage.
Reverse letterboxing with the actual resize metadata
Name each coordinate frame explicitly: source image, cropped image, resized image, padded model input and display viewport. A number such as 240 is ambiguous without its frame and axis.
First identify the box representation. Corner coordinates, center-plus-size and top-left-plus-size require different decoding. Determine whether coordinates are normalized or expressed in pixels, and whether their order starts with the horizontal or vertical axis.
For normalized corner coordinates relative to the padded model input, the inverse mapping is:
These equations assume resizing the complete source image before padding. If preprocessing starts with a crop, use the crop dimensions for scaling and add the crop origin afterward.
A typical aspect-preserving letterbox starts with:
However, integer rounding can make the realized horizontal and vertical scales slightly different. Store the actual resized dimensions and actual padding rather than reconstructing them from the nominal scale.
Worked example
Suppose the source image is 1280 × 720 pixels and the model input is 640 × 640 pixels. Aspect-preserving resizing gives:
Assume a decoded box uses normalized corners relative to the padded input:
Convert to model-input pixels:
Then reverse padding and resize:
The source-space box is therefore [320, 200, 960, 520]. Simply multiplying normalized coordinates by the source dimensions would give incorrect vertical positions.
Give each transformation one owner
Choose one stage to own conversion into source-image coordinates. The renderer should receive either source-space detections or clearly marked model-space detections, never an undocumented mixture.
Use explicit field names such as bbox_model_pixels and bbox_source_pixels. Include the frame identifier and preprocessing metadata with each inference request. Avoid shared mutable metadata that another request can overwrite.
Before testing another real detection, inject synthetic box records directly into the coordinate-conversion function. Test the center, each corner and a box crossing the padded region. This isolates geometry from detector accuracy.
Perform a round-trip test: map a source point into model space and then map it back. For a floating-point transformation test, you might require an error below 0.01 pixel. That is an arithmetic check, not a claim about detector accuracy.
Keep coordinates as floating-point values until rendering or an interface requires integers. Also document whether maximum box coordinates are inclusive or exclusive; array slicing and drawing APIs can use different conventions.
Clip boxes only after diagnosing the original values. Early clipping can hide an axis swap or missing padding subtraction. Record both unclipped and clipped coordinates during verification.
Separate overlay geometry from frame association
A correct source-space box can still look wrong when the preview adds another resize, crop or offset. Test the numerical box against the original saved image before investigating the live display.
If the source image is shown at half size, its overlay needs the same display transformation. If the viewport adds margins, apply those offsets too. Keep this display mapping separate from model preprocessing.
The related coordinate-indexing checks for detection overlays help distinguish horizontal-versus-vertical indexing mistakes from display offsets. For live diagnostics, camera preview overlays for motion tracking explains how visible references can expose inconsistent motion.
When the error appears only on moving objects, compare frame identifiers before changing scale factors. Render detections on the retained originating frame, not whichever image is currently available. Follow the timestamp and frame-matching checks when acquisition and inference run asynchronously.
A decisive test is simple: if the overlay is correct on its saved source frame but wrong on the live preview, investigate display transformation or frame association next.
Keep hardware selection separate from output decoding
Changing acquisition hardware will not repair an incorrect tensor parser. Likewise, a faster host cannot correct a duplicated coordinate transformation. Establish a passing saved-image test before attributing the symptom to capture or compute hardware.
If you also need acquisition hardware, browse area scan cameras and use the camera finder to compare acquisition requirements. The LS-CMR-0166 is a USB3 area scan camera candidate; confirm its suitability for your imaging task separately from the model-output investigation.
- Sensor format
- 1/1.7"
- Interface
- USB3
- Resolution
- 12 MP
- Shutter
- Rolling
For an integrated camera-processing installation, evaluate vision controllers against the runtime, interfaces and sustained workload you actually need. The MV-VC2140-128G40-1T is a vision controller candidate, not evidence of compatibility with a particular model runtime.
- LAN ports
- 3-4 LAN ports
- USB 3 ports
- 2 USB 3 ports
- Light outputs
- 4 built-in channels
- Memory
- 8 GB
Before ordering, confirm operating-system support, runtime architecture, camera connectivity and any required software licensing. Run the saved-image contract test on the proposed runtime rather than assuming equivalent output behavior across deployment environments.
Know when coordinate correction is the wrong fix
Once parsing and geometry pass, poor localization may genuinely come from the image or model. Compare the saved preprocessed input with the source: excessive resizing, incorrect channel order or inappropriate numerical scaling can damage detection quality without breaking the parser.
A box that correctly encloses the wrong object is not primarily a scaling problem. Investigate training coverage, class definitions and decision thresholds after verifying label mapping.
Also distinguish image coordinates from machine coordinates. Correct pixel boxes do not establish a robot position or measurement in physical units. Lens distortion, perspective, calibration and object height require separate treatment. A linear resize correction cannot reverse those effects.
Do not move a machine based solely on a visually convincing overlay. Validate the complete control interface and retain the machine's independently designed safety functions.
Finish with a repeatable acceptance checklist
Before trusting the deployment, retain evidence that each boundary behaves as intended:
- Artifact identity: the model, configuration and label map belong together.
- Parsing: tensor axes, valid counts and record fields match the documented contract.
- Scores: quantization, score conversion and thresholding occur only where required.
- Geometry: normalization, padding, resizing and crop offsets are reversed exactly once.
- Display: overlays follow the displayed image's transformation and coordinate convention.
- Timing: every detection remains associated with its originating frame.
- Regression: center, edge, corner, empty-scene and multiple-object cases remain covered by saved tests.
Keep numerical assertions alongside reference overlays so later changes cannot pass on appearance alone. Record the settings used for acceptance and repeat the tests after changing preprocessing, export options or runtime versions.
For a hardware review, bring a saved source image, preprocessed input, tensor description and timing requirements to applications support. If acquisition selection is still open, start with the camera configurator after defining the image requirements.
Questions
Why are all my detection boxes shifted by a similar amount?
A consistent offset often indicates padding, crop origin or display margins that have not been accounted for. First draw the decoded box on the exact model-input image. If it fits there, inspect the inverse preprocessing transformation before changing the model or retraining.
Can I multiply normalized coordinates by the original image size?
Only when the documented output uses normalized coordinates relative to that original image. Coordinates normalized against a padded model input need padding removal and inverse resizing instead. Confirm whether your runtime already performs this conversion, because applying it twice also produces incorrect boxes.
Why are detection labels wrong when the boxes look correct?
The label map may belong to a different model export, use a different starting identifier or omit a reserved background entry. Compare raw class identifiers against the model's documented mapping. Correct geometry and high scores cannot verify that the displayed class name is correct.
Should I lower the confidence threshold to fix missing detections?
No. A lower threshold can reveal candidates for diagnosis, but it cannot repair layout errors, incorrect score conversion or bad geometry. Count candidates before and after each filtering stage on saved images, then choose the operating threshold using validated inspection performance.
Do I need different hardware when detections display incorrectly?
Test the same saved image on both runtimes and compare raw outputs, decoded records and preprocessing metadata. Investigate hardware performance when frame delivery or latency fails requirements. A camera or controller replacement should follow measured acquisition or compute needs, not an unexplained overlay error.