Detection to OCR pipeline: crop detected regions then read text

Text on a production line rarely sits still or alone. A serial number lands somewhere new on every casting, a shipping label carries five fields, and plates arrive at an angle. Full-frame OCR struggles in those scenes: it merges neighboring strings, invents characters, or returns nothing. The pattern that holds up is two-stage — a detector you train finds each text-bearing region, a padded crop of that region goes to whatever OCR engine you choose, and the detection index ties every string back to its place on the part. This guide gives the steps, a backend-agnostic code pattern, and the camera math that makes the crops worth reading.

What each stage owns

The detector answers where; the recognizer answers what. Keep that boundary hard. Train the detector to return one box per readable region and name the classes by role — plate, shipping_label, serial_number, meter_display — never by content. A detector asked to read characters starves the recognizer of context and fails at both jobs.

Two-stage detect-then-recognize is also how the big OCR engines are built internally: a text detector finds each textual region, and the recognizer transcribes one crop at a time. Putting your own detector in the first seat moves that stage under your control — exactly what plates and meters need. The license plate OCR pipeline walkthrough and the digital meter capture guide both stop at this same seam.

Put the engine behind a callable you own: a function that takes a crop and returns text plus a score. Engines get replaced, licenses change, and one plant wants offline decoding while another already runs a service. The orchestration — crop, pad, clamp, join — is the part worth writing once.

The pipeline step by step

1. Capture training frames at the station. Same lens, same lighting, same working distance as production. Detection accuracy collapses when training frames are sharper or better lit than the real thing.
2. Annotate one box per readable region. Draw just outside the text, not around the whole part; the padding rule below adds the margin. One class per region type.
3. Add background frames with no text at all. They teach the detector what an empty frame looks like; why negative images need no labels covers the mechanics.
4. Train, then set the confidence floor. Start at 0.30, raise it until the first real region drops out, step back one notch and freeze. A missed region is an unread part; a false region costs one wasted call you reject on score.
5. Fix the two interfaces. detect(frame) returns a list of [x1, y1, x2, y2, class_id, score] with float coordinates; recognize(crop) returns (text, ocr_score). Everything else is plumbing you write once.
6. Crop with padding and clamp to the frame. If your detector letterboxes frames to a square input, map boxes back to full-frame pixels first — input dimensions and padding shows that arithmetic.
7. Read one crop per accepted detection, in order. The detection index is the join key.
8. Log every failed read with its crop. Misses are your next training batch. A pipeline that cannot show its failures cannot be tuned.

Crop with the right padding

Pad every box by the larger of 8 px and 4% of its longer side. A 40 × 20 px box gets 8 px; a 546 × 328 px box gets 22 px. Padding keeps ascenders, descenders and tilted strokes inside the crop. Recognizers resample every crop to a fixed input height, so a stroke cut at the edge stays cut — no later stage can repair it.

Clamp the padded box to the frame on all four sides, round to integers, and slice rows before columns: frame[y1:y2, x1:x2]. Slicing with float coordinates or swapped axes is the most common bug in these pipelines. Flag any detection whose box already touches a frame edge before padding: detectors return out-of-frame coordinates when a region straddles the border, the crop comes out partial, and the OCR output looks confident while being wrong.

A reader you can point at any engine

# Every name below belongs to this guide. Plug your detector into `detect`
# and your OCR engine into `recognize`; neither is a third-party API name.
# frame: 2-D uint8 grayscale array. box: [x1, y1, x2, y2, class_id, score].

def crop_with_padding(frame, box, pad_px=8):
 # Crop box from frame, padded and clamped to the frame edges.
 height, width = frame.shape[0], frame.shape[1]
 x1, y1, x2, y2 = (int(round(v)) for v in box[:4])
 pad = max(pad_px, int(0.04 * max(x2 - x1, y2 - y1)))
 x1, x2 = max(0, x1 - pad), min(width, x2 + pad)
 y1, y2 = max(0, y1 - pad), min(height, y2 + pad)
 return frame[y1:y2, x1:x2].copy # rows first, then columns

def read_detections(frame, detect, recognize, min_score=0.30):
 # One record per accepted detection, joined on the detection index.
 records = []
 for index, box in enumerate(detect(frame)):
 if box[5] < min_score:
 continue
 crop = crop_with_padding(frame, box)
 text, ocr_score = recognize(crop)
 records.append({'index': index, 'class_id': box[4], 'box': box[:4],
 'detector_score': round(box[5], 3),
 'text': text, 'ocr_score': ocr_score})
 return records

The test that proves the padding and the join:

import numpy as np

def test_padding_and_association:
 frame = np.zeros((100, 100), dtype=np.uint8)
 frame[10:30, 10:50] = 200 # the label patch, 40 x 20 px
 seen = []
 def detect(f):
 return [[10.0, 10.0, 50.0, 30.0, 1, 0.92], # keep
 [70.0, 70.0, 90.0, 90.0, 2, 0.05]] # below min_score
 def recognize(crop):
 seen.append(crop.shape)
 return 'SN-2041', 0.97
 records = read_detections(frame, detect, recognize)
 assert seen == [(36, 56)] # 20 + 2 x 8 rows, 40 + 2 x 8 columns
 assert len(records) == 1
 assert records[0]['text'] == 'SN-2041' and records[0]['index'] == 0

The two callables are the whole contract. Point recognize at a local engine, a decoding service, or OCR options already inside an HMI — the orchestration above does not change, and that is the argument for a callable over one baked-in library.

The pixel budget that picks the camera

Design so the smallest character in the crop stands at least 16 px tall, and treat 10 px as the floor. Below that line the loss is unrecoverable: a 16 px character upscales 2–3× to the recognizer's typical 32–48 px input and strokes survive, while a 10 px character stretches 3–5× and stroke edges go soft. Why small text fails covers the failure modes below that line.

Work the budget from the part, not the sensor: measure the smallest print with calipers, choose the field of view the station needs, and let the two lines above return the frame width in pixels. Any camera in the area scan cameras category that clears that count can carry both stages — the detector asks for nothing special.

When the field of view cannot shrink, frame pixels are the only lever left. The MV-CS200-10UM is a 20 MP USB3.0 camera with a 1" sensor. Count pixels across the field of view rather than megapixels alone: a larger sensor behind the same lens widens the field and hands the extra pixels straight back.

Worked example: 3 mm print at 600 mm on a belt

Inputs, each with its source:

• Smallest print to read: 3 mm character height, from the label drawing, confirmed with calipers on the lightest print run.
• Pixel budget: 16 px per character height, the design value above.
• Working distance: 600 mm, fixed by the machine guard over the belt.
• Belt speed: 0.5 m/s, from the line spec.
• Sensor width: about 7.4 mm, the nominal active width of a 1/1.7" sensor — the one estimate in this example.
• Camera: the LS-CMR-0166, a 12 MP USB3 area scan camera with a 4000 × 3000 resolution.

The arithmetic:

1. Sampling needed: 16 px ÷ 3 mm = 5.33 px/mm.
2. Field of view: 4000 px ÷ 5.33 px/mm = 750 mm wide; 3000 px ÷ 5.33 px/mm = 563 mm tall.
3. Magnification: m = sensor width ÷ FOV = 7.4 mm ÷ 750 mm = 0.0099.
4. Focal length: f = WD × m ÷ (1 + m) = 600 mm × 0.0099 ÷ 1.0099 = 5.9 mm, so a 6 mm lens is the one to order.
5. Check the real lens: m = 6 ÷ (600 − 6) = 0.0101; FOV = 7.4 ÷ 0.0101 = 733 mm; sampling = 4000 ÷ 733 = 5.46 px/mm; smallest character = 3 × 5.46 = 16.4 px. That passes with about 2% margin — keep the label zone near frame center, where the lens is sharpest.
6. Exposure: one pixel spans 1 ÷ 5.46 = 0.183 mm, so half a pixel of blur at 0.5 m/s takes 0.0915 ÷ 500 = 183 µs. Fix exposure at 150 µs and light the belt hard enough to hold it.
7. Crop size: a 100 × 60 mm label images at 546 × 328 px, padding is max(8, 4% × 546) = 22 px, and the recognizer receives 590 × 372 px.

The decision: a 12 MP camera, a 6 mm lens at 600 mm, 150 µs exposure, 22 px padding — every check passes for a stop-and-read station, the normal case for label inspection. One hardware split: this sensor has a rolling shutter, so a belt that never stops takes a global shutter model from the camera finder instead.

When the detector in front is the wrong answer

Dense, fixed-layout pages — forms, invoices — already carry a layout model inside full-frame OCR engines, and a detector in front of them mostly adds missed regions. Run one pass over the page and use its own segmentation.

Barcodes and 2D codes never need this pipeline: dedicated decoders find and read many symbols per frame by themselves. The LS-CRD-0028 is a 2560 × 2048 fixed code reader rated at 60 codes/s with an 8 mm lens. Even mixed labels stay a decoder problem — reading multiple barcodes in one image is iterative decoding, not detection. Browse the fixed code readers category first.

One text field at a fixed position needs a fixed ROI, not a detector — until the fixture drifts, and then it needs one badly. Where no PC is allowed at the station, smart cameras run detection and OCR on the device; AI versus rule-based inspection helps you pick. Short strings can even stay on a code reader entirely — OCR of several text strings on one reader image shows that route.

Checks before you ship it

• Zero missed regions over a replayed production day. A false region is one rejected call; a miss is an unread part.
• Every crop at 16 px or more over its smallest character, measured on real crops, not the drawing.
• Edge boxes padded, clamped and flagged when the box starts at the frame boundary.
• One join key: the detection index travels inside the record, and nothing sorts crops between slicing and reading.
• Exposure inside the blur budget at top belt speed — 150 µs in the example, recomputed for your line.
• Failure logging on: every empty or rejected read saved as a crop, ready to become the next training batch.

From here, size the whole station in the configurator, or bring the label, the print spec and the belt speed to our applications desk and we will run this arithmetic against your parts.