Rotated ROI OCR: teach, read and keep the text upright

You draw a rectangular region of interest, teach the characters, and the read works. Then the part sits at an angle, someone rotates the ROI, and the read fails, returns the wrong string, or returns nothing, even though the rotated rectangle draws correctly on screen. The failure appears at every layer of the stack: a PC pipeline, a smart camera, a fixed code reader with an OCR tool. The cause is never the classifier itself. Teach stores one pixel patch; read hands it another. Below are the five places that substitution happens, ranked by how often they occur, the crop geometry that fixes it with the arithmetic shown, and the checks to run before you re-teach anything.

The rule: rotate the crop, not just the rectangle

A teach is a reference patch plus a character set trained at one orientation and one scale. The classifier never learns that a rectangle rotated; it only sees pixels. Rotating the drawn ROI changes which pixels you must extract, and nothing else. So the patch that reaches the classifier at read time has to be the taught patch reconstructed: same content, same scale, characters upright.

That takes two steps. Extract a window that contains the whole rotated rectangle. Rotate that window by minus the ROI angle about the rectangle center, then take the central region at the taught size. Recognition models trained on near-horizontal text absorb a few degrees of residual tilt and fall over beyond that, so aim to land within about 5 degrees of the taught orientation.

Five failures at a glance

Cause 1: the angle never reaches the crop

The most common break, and the quietest. A rotated rectangle on screen is five numbers: center x, center y, width, height, angle. Most crop code accepts four. The extraction then runs on the un-rotated rectangle, or on coordinates captured before the rotation, and the classifier reads stale pixels while the display shows the new rectangle.

Log the inputs at the moment of the crop call, not at the user interface. If the angle is missing there, you have found it. In a block-diagram environment such as LabVIEW, inspect what the ROI data actually carries into the crop routine: when only position and size arrive, the angle was dropped upstream, and no amount of re-teaching will follow a rotated region. For character-set training on the same hardware, see training OCR on a USB camera in LabVIEW.

The fix is mechanical. Carry all five numbers together, compute the rectangle center in image coordinates, and pass the angle into the same call that positions the crop.

Cause 2: you cropped the bounding box of the rotated rectangle

Second most common. Here the code does use the rotation, but only to enlarge the crop to the axis-aligned box around the tilted rectangle. The patch now contains the text, diagonal. A recognizer trained on horizontal baselines has nothing stable to segment, and detection models keep finding rotated text long after recognition fails, which is why a system can locate the string and still decode nothing.

The fix is the two-step extraction from the rule above: crop the outer box, rotate the patch by minus the angle about its center, take the central taught-size window. If your ROI arrives as points rather than numbers, the minimum-area-rectangle function (minAreaRect in OpenCV) converts a point set into center, size and angle, and boxPoints returns the four corners. Take the minimum and maximum of those corners for the outer crop and you never depend on the library's angle convention.

The arithmetic of one rotated crop

Taught ROI: 400 px wide, 80 px tall, axis-aligned. The operator rotates it by 20 degrees.

• Outer box width: 400 × 0.940 + 80 × 0.342 = 375.9 + 27.4 = 403.3, so crop 404 px.
• Outer box height: 400 × 0.342 + 80 × 0.940 = 136.8 + 75.2 = 212.0, so crop 212 px.
• Half diagonal: √(400² + 80²) / 2 = √166,400 / 2 = 408 / 2 = 204 px.

The half diagonal is the safety radius: every corner of the rotated rectangle stays within 204 px of the center at any angle. When the angle is only approximate, crop a 408 × 408 square centered on the ROI center instead of the tight box. Rotate the patch by minus 20 degrees about its center and take the central 400 × 80 window; that window is the taught patch, reconstructed. Use a sub-pixel extractor such as getRectSubPix for the patch position when the ROI center lands between pixels, then apply the rotation with an affine warp.

Cause 3: the patch arrives rotated the wrong way

Two conventions fight you here. Image coordinates run y downward, so a rotation that looks counter-clockwise on screen is negative in the pixel array. And a rectangle is symmetric under 180 degrees while text is not, so the crop can be geometrically perfect and still hand the classifier upside-down glyphs. A third trap sits at the diagonal: a rectangle rotated plus 80 degrees is the same rectangle rotated minus 10 degrees with width and height swapped, so the value reported as the angle can jump as the ROI crosses 45 degrees.

Constrain the taught and live angles to within 45 degrees of each other. When the part can land near 180 degrees, run the classifier on both orientations of the upright patch and keep the higher score; a patch-sized classification runs twice in negligible time.

Cause 4: the scale moved between teach and read

Rotation correct, patch upright, reads still garbage: compare character height in pixels. A character set trained at one height is a mismatch at another, and rotation gets blamed for what is really magnification drift after a lens swap or a working-distance change.

A 12 MP area scan camera such as the LS-CMR-0166 delivers a 4000 × 3000 pixel frame. Put a 200 mm field of view across that frame and you have 4000 / 200 = 20 px per mm, so a 3 mm character stands 3 × 20 = 60 px tall inside an 80 px ROI. If a second station delivers 15 px per mm, the same character is 45 px and the taught patch no longer matches. Hold the magnification, or resample the patch to the taught pixel height before classification. The camera resolution calculator sizes this from the other end: field of view and smallest feature in, pixels required out.

For what actually crosses into the classifier after cropping and filtering, see what reaches the classifier from an OCR character region.

Cause 5: the patch was rotated twice

Every resampling softens edges; a bilinear rotation spreads a stroke edge over roughly one pixel. Rotate once, from the native frame to the upright patch, and count every resampling step between sensor and classifier: onboard digital rotation in the camera, frame rectification for display, your own rotation for OCR. Two rotations of the same text is the classic slow decay, where reads work after commissioning and fade once someone enables a display rotate.

Use bilinear interpolation for the rotation, never nearest-neighbour, on text. If the taught strokes are only two pixels wide, no interpolation setting saves them; fix that with magnification, not with resampling.

Where the rotation belongs

On a fixed code reader with an on-device OCR tool, rotate the ROI inside the device tool and re-teach there. The device then crops, rotates and classifies in one consistent place, and the layer mismatch described above cannot occur. Running several text regions in one image is the same discipline; see OCR on a fixed code reader with multiple text strings.

On a smart camera, the ROI angle should come from the part, not from an operator: a pattern match locates a stable feature, returns its angle, and the ROI follows. That approach is laid out in anchoring an OCR ROI to a pattern match.

With an area scan camera and your own code, the two-step crop from Cause 2 is the whole fix, the same discipline as any detect-then-crop OCR pipeline. Pick the camera in the camera finder, or start at the camera step of the system configurator.

When the crop-rotation fix is the wrong fix

• The angle varies run to run. An operator-rotated ROI is then the wrong mechanism; let a pattern anchor supply the angle instead of a person.
• The text is curved around a cylindrical part. No single angle straightens an arc; reading curved text as an annulus works without unwrapping.
• The characters are randomly oriented, digit to digit. Correct each region before training, as in deskewing randomly rotated digits before you train.
• The text is vertical by design. Rotate the segment 90 degrees and constrain the character set, per reading vertical text with OCR.
• The orientation comes from a deep-learning detector. Crop from the rotated box after detection, then map result corners back: rotated bounding boxes in live inference and mapping rotated barcode result points cover both directions.

If you square the part up mechanically instead, treat it as a machine modification: lock out the machine before you reach inside the frame to move a fixture.

Checks before you re-teach

1. Save the patch at read time and place it next to the taught patch; orientation, height and contrast must match by eye.
2. Log the five ROI numbers at the crop call, not at the interface.
3. Measure character height in pixels in both patches; they should agree within a few percent.
4. Count the resampling steps from sensor to classifier; the target is one.
5. Clamp the outer box to the frame, and treat a clamped crop as a no-read.
6. Push a bold letter L through the pipeline once; input and patch must agree in orientation.
7. Fixed angle: re-teach at that angle. Varying angle: a pattern anchor supplies the angle.

If the patch is upright, at taught scale, and the read still fights you, send the taught and live patch pair to us through the contact page. Two images usually settle it in one reply.