Tesseract digit accuracy: fix resolution, psm and whitelist

A digit read that works one cycle and fails the next is rarely a model problem. The pattern is familiar: the full frame finds almost nothing, a cropped strip finds the digits most cycles, a 6 vanishes now and then, and a 3 comes back as a look-alike letter. Three settings are fighting your image: the resolution you hand the engine, the page segmentation mode, and the character set it may output. This guide gives you the numbers that matter — a 30-50 px glyph band with the arithmetic to hit it, the psm value for each crop type, and the whitelist line that makes look-alike swaps impossible. Apply them in that order, then prove the fix on your worst frames, not your best one.

The symptom pattern and the order to attack it

Intermittent digit reads split into four signatures, and each points at its own cause:

1. Nothing found on the full frame. A 1653×2338 image drowns the layout stage: thousands of candidate text regions, speckle everywhere, no line structure it recognizes.
2. Region found, digit missing on some frames. The glyphs sit outside the size band the engine classifies well, so binarization or the line finder eats them when the capture varies.
3. Digit swapped for a look-alike. A 3 that returns as a Cyrillic З means the letter is in the active alphabet; the shapes are near-identical, so the model flips between them.
4. Shape area found only sometimes. The first contour in the list is not always your figure, and a worn marker leaves hairline gaps that break the mask differently on every frame.

Work in that order. Scale fixes 1 and 2, the segmentation mode makes a cropped strip readable at all, the whitelist kills 3, and a mask pass settles 4. A whitelist cannot rescue text the line finder never found, so tuning the alphabet before the scale wastes an afternoon.

Fix the scale first: land glyphs at 30-50 px

Tesseract rescales each detected text line to a fixed internal height before classification, and its binarization and line-finding stages behave best around text near 30 px x-height. In practice the reliable band runs from about 20 px to roughly 100 px of glyph height, with 30-50 px as the target where every stage keeps margin. Below about 20 px the strokes alias away — the same reason the minimum font size for OCR sits where it does.

A full frame at high resolution fails for the opposite reason: it buries the digits in speckle and hands the layout stage thousands of false text candidates. Downscaling works because it is a low-pass filter. Shrink the frame and every gap, dirt speck and hairline break below the new pixel pitch averages away; strokes close up and the line finder sees clean shapes.

Worked example: a 1653×2338 frame whose digits measure 130 px tall (an example measurement — read yours off any image viewer). Shrinking the width to 600 px gives scale = 600 / 1653 = 0.363, so the digit lands at 130 px × 0.363 ≈ 47 px — inside the band. That is why a 600×825 resize suddenly finds everything: arithmetic, not luck. If you crop first instead, aim the crop itself at 40 px: scale = 40 / 130 = 0.308.

If you control the optics, work from pixel pitch. Digits 4 mm tall at a 40 px target need 4 mm / 40 px = 0.1 mm per pixel. A 1440 px wide sensor then covers at most 1440 px × 0.1 mm/px = 144 mm across; at a fixed 60 mm field the same digit spans 4 mm / (60 mm / 1440 px) ≈ 96 px — still readable, but spending pixels.

Match the psm mode to what is in the crop

The default, --psm 3, runs full automatic page layout analysis. On a tight crop that is wasted freedom: the analyzer hunts for columns, blocks and margins that are not there and can return nothing at all. On a page of scattered marks it under-finds badly. Mode choice moves results as much as resolution does — switching from automatic page layout to a mode that matches the crop (a single line, or sparse text) is regularly the difference between a read and nothing, which is why you should change psm and resolution together, one at a time, on your worst frames.

Set the mode from what the crop contains:

The decision rule: crop the digits into their own small image, then pick the mode that describes that crop. A strip reading 5/6 wants 7. The value 7,2 on its own wants 8. Sparse mode 11 belongs on the full frame, where the digits sit wherever the layout puts them — expect junk tokens from empty regions and filter candidates by position, not by hope.

Close the character set with a digit whitelist

Reading with lang='rus' puts every Cyrillic letter in play, so З competes with 3 on every frame and wins sometimes. The whitelist closes the alphabet: with -c tessedit_char_whitelist=0123456789 the engine cannot return a letter at all, so the 3-to-look-alike flip disappears instead of getting rarer. It is the same principle as the fixes for look-alike OCR characters: do not let the wrong answer be typeable.

Add every separator you expect to read. A decimal value like 7,2 needs 0123456789,. and a ratio like 5/6 needs 0123456789/. A digit-only list returning 56 from a 5/6 strip is the proof the whitelist is active; add the slash when it carries meaning. Whitelists bite hardest in the tight modes (7, 8, 13) on a tight crop. If the whitelist seems ignored, the engine build is old — update it, because current builds apply it to the classifier output.

The pipeline, end to end:

import cv2
import pytesseract

img = cv2.imread("frame.png")     # 1653 x 2338 input frame
band = img[500:700, 800:1120]     # digit band only; measure your own coordinates once
scale = 40 / 130                  # target 40 px / measured 130 px glyph height = 0.308
band = cv2.resize(band, None, fx=scale, fy=scale, interpolation=cv2.INTER_AREA)
text = pytesseract.image_to_string(
    band,
    config="--psm 7 -c tessedit_char_whitelist=0123456789",
)
print(repr(text))                 # '56' on a 5/6 strip proves scale, mode and whitelist

Then the mask: why the shape area comes and goes

The shape half of the script has its own intermittence. cv.findContours returns contours in traversal order, so contours[0] is the first one found, not necessarily your figure. Index by size instead: keep the contour with the largest cv.contourArea, or filter to an area window that matches the part.

A worn marker adds a second failure: hairline gaps break the filled mask differently on every frame, so the area jumps or the shape vanishes. Close those gaps before findContours with a morphological closing pass — dilate, then erode by the same amount. The downscale you already did for OCR helps here too: it merges gaps below the new pixel pitch, which is why the area started reading reliably at the smaller size.

Prove it on your worst 20 frames

Tuning against one image overfits: a psm that rescues frame 1 can break frame 2. Test like this:

1. Collect the 20 worst frames from recent runs — every historical miss, every swap, plus a few good ones as controls.
2. Write the expected string for each frame once, in a list beside the images.
3. Change one variable at a time: resize first, then psm, then whitelist, then the mask pass.
4. Have the wrapper report a confidence value per word and log it beside each read.
5. Pass means every expected string comes back exactly; anything else is a miss and stays in the folder.

A confidence floor turns the log into a guard: flag any read under your floor for a second look, and set the floor from this folder rather than from folklore. The same expected-string discipline scales to production — see checking OCR output against an expected string.

When the read moves onto the line

For a bench script, Tesseract is the right tool. When the same marking is read every cycle in production, move the read onto hardware built for it — the problem stops being tuning and becomes configuration.

A smart camera runs acquisition, lighting control and the inspection on the device with no PC in the loop, so the pipeline above collapses into one configured job. The LS-SMC-0063 carries a 16 mm lens with white light and reads out at up to 60 fps over GigE.

Where the marking is a real barcode or DataMatrix code, a fixed code reader decodes 1D and 2D codes on the line with optics and illumination matched to the job. The MV-ID3013PM-12M-WBN pairs a 12 mm focal length with a 1.3 MP sensor on Fast Ethernet.

For free-form human-marked digits that no reader claims, keep the pipeline but feed it from an industrial camera with controlled light instead of a phone frame: fixed exposure, a dedicated machine vision light, and the glyph band held by the optics. Reading several strings in one frame has its own pattern — see OCR on a fixed code reader: multiple text strings in one image.

What does not work

Three moves that feel right and are not:

• More resolution. Past the band, extra pixels buy speckle, not accuracy. The 600 px frame outread the 1653 px frame because the engine normalizes lines internally — a 47 px glyph reads as well as a 130 px one, without the dirt.
• Upscaling a small crop. Interpolation cannot restore strokes that were never sampled. If the crop lands below about 20 px, move the camera closer or lengthen the focal length instead.
• Training a custom font first. A model trained on your glyphs, loaded like any other language, is the correct last resort for genuinely nonstandard characters — seven-segment digits, stencil marks, dot-matrix print. Train after crop, scale, mode and whitelist are right; training on a broken pipeline only bakes the errors in.

Checklist before you trust the read

• Crop tight to the digit band. Everything else is noise the layout stage can trip over.
• Glyphs 30-50 px tall in the image you pass. Measure once, compute the resize factor once.
• psm matches the crop: 7 for a line, 8 for one token, 10 for one character, 11 for scattered marks.
• Whitelist names every expected character, separators included, and nothing else.
• Engine build current, so the whitelist is actually applied.
• Light and exposure fixed, so the frames your settings were tuned on are the frames you get.
• Failure folder scored after every change, expected versus read logged, misses kept.

The decision, case by case

• Bench script, occasional reads: crop to the band, land glyphs at 30-50 px, set psm to the crop (7 for a line), whitelist digits plus separators. That combination is the fix.
• Same code on every part: a fixed code reader, not OCR.
• Digits plus inspection on the line: a smart camera on the device.
• Free-form marked digits: the pipeline, on an industrial camera frame with controlled light.
• Nonstandard glyphs after all of the above: train your own model, last.

More reads like this sit under code reading. If a station still resists after the failure folder comes back clean, the /contact/ page reaches the applications team — send the worst frame, the crop size and the exact config string you ran.