Deskew a rotated table before OCR with OpenCV and Python

You scanned or photographed a data table and the page came out tilted a few degrees. The table extractor you had in mind — Camelot, or any lattice detector that hunts for horizontal and vertical rules — assumes an upright table and returns nothing usable on a tilted one. The fix is geometric and simple: measure the tilt with OpenCV, rotate the image once, then extract. This guide commits to the numbers: the residual worth hitting (±0.5°), two estimators that reach it on real pages, the rotation call with white padding, and the cell grid plus per-cell OCR that follows. Everything runs in Python with OpenCV and a Tesseract engine, and the capture section sizes a camera station for an A4 sheet.

The tolerance that matters: ±0.5°

Cell finding by projection needs every table row to land inside one band of the row-ink profile. Skew smears each text line as it crosses the page: the far end of a line sits a drift below the near end, where drift = W × tan(θ) and W is the line length. Keep the drift under half the row pitch and the bands stay separate.

On an A4 sheet in portrait a text line spans about 210 mm. Twelve-point rows have a pitch of 12 × 25.4 / 72 = 4.23 mm, so the budget is 2.12 mm and the ceiling is:

θ_max = atan(2.12 mm / 210 mm) = 0.58°

Work to ±0.5°. At 0.5° the drift is 210 × tan(0.5°) = 1.83 mm, inside the budget; at 1° it is 3.67 mm, enough to push the outer rows into their neighbours' bands. The tolerance is geometric, in millimetres, so it holds at any resolution — it scales only with the type size: 10 pt rows give 0.48°, 6 pt rows give 0.29°.

Text PDF or image: pick the path first

Camelot reads tables out of a PDF's text layer, and its documentation states the limit plainly: text-based PDFs only, not scanned pages. That splits the job into three cases, each with one right answer:

• Upright table in a text PDF — Camelot alone: lattice mode for ruled tables, stream mode for whitespace-aligned ones. No OpenCV stage exists here, and none is needed.
• Scan or camera frame, any skew — the path in this guide: deskew, locate the rules by projection, OCR each cell. A deskewed raster has no way into Camelot, so the image path builds its own grid with Tesseract through pytesseract.
• Skewed table inside a text PDF (a scanned page whose OCR text layer froze the skew in) — render that page to a 300 dpi PNG and take the image path.

What you need

• Python 3 with the opencv-python package, numpy, and pytesseract bound to a Tesseract engine.
• The page as a raster at 300 dpi or better — about 12 px/mm — for 10-12 pt text.
• Ten or more legible text runs, or a ruled table: the estimators need one of the two.

The deskew steps

Run the stages in this order, once per frame:

1. Load one channel of the page.
2. Binarize the ink adaptively.
3. Estimate the tilt (blobs, rules, or sweep).
4. Rotate once, with white padding.
5. Prove the sign and residual with a synthetic assert.
6. Cut the grid and read each cell.

1. Load one channel

import cv2
import numpy as np

page_path = "tilted_table.png"                      # str: your frame
gray = cv2.imread(page_path, cv2.IMREAD_GRAYSCALE)  # uint8 ndarray, H x W

2. Binarize the ink adaptively

Lamps shade a page, and one global Otsu level splits it into light and dark halves. Adaptive thresholding evens the background first — the same lesson as thresholding white text on a busy background for OCR in Python:

def ink_of(img):
    """uint8 page in -> 0/255 mask, ink white."""
    return cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C,
                                 cv2.THRESH_BINARY_INV, 31, 15)

ink = ink_of(gray)

3. Estimate the tilt

Method A merges each text line into one blob and takes the median long-side angle across the blobs:

kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (40, 1))

def long_side_deg(rect):
    """Box long side vs horizontal, folded into (-45, 45]."""
    _, (w, h), a = rect
    if w < h:                       # long side sits in the height slot
        w, h, a = h, w, a + 90
    while a > 45:
        a -= 90
    while a <= -45:
        a += 90
    return float(a)

def estimate_skew_deg(img):
    """Median line tilt, degrees, clockwise-positive; 0.0 if empty."""
    blobs = cv2.morphologyEx(ink_of(img), cv2.MORPH_DILATE, kernel)
    contours, _ = cv2.findContours(blobs, cv2.RETR_LIST,
                                   cv2.CHAIN_APPROX_NONE)
    angles = []                     # list[float]
    for c in contours:
        x, y, w, h = cv2.boundingRect(c)
        if w >= 200 and h <= 60:    # whole-line blobs only
            angles.append(long_side_deg(cv2.minAreaRect(c)))
    return float(np.median(angles)) if angles else 0.0

At 300 dpi a 20-character line runs about 500 px and a 12 pt row is 50 px tall, so w >= 200 and h <= 60 keep line blobs and drop logos, stamps and noise. Length matters for resolution: a 0.5° tilt drops a 500 px line by 500 × sin(0.5°) = 4.4 px, and 1.7 px at 200 px — both above the 1 px binarization noise floor.

Method B fits the ruling lines directly, when the table has them:

lines = cv2.HoughLinesP(ink, 1, np.pi / 720, threshold=200,
                        minLineLength=gray.shape[1] // 3, maxLineGap=20)
rule_angles = []                   # list[float]
if lines is not None:
    for x1, y1, x2, y2 in lines[:, 0]:
        a = np.degrees(np.arctan2(y2 - y1, x2 - x1))
        if abs(a) <= 10:           # near-horizontal rules only
            rule_angles.append(float(a))
skew_deg = float(np.median(rule_angles)) if rule_angles else 0.0

The 0.25° theta step plus a median over six or more rules lands within ±0.1°. For sparse pages with no rules, sweep −5° to +5° in 0.1° steps and keep the rotation with the highest level score from step 4 — a text-only page still answers.

4. Rotate once, with white padding

Image y grows downward, so a line with its right end lower gets a positive angle from arctan2, and cv2.getRotationMatrix2D rotates counter-clockwise for positive angles — the estimate goes in unchanged. The level score settles any remaining convention doubt: a level page stacks its rules and text rows into a few dark profile rows, so the row-ink variance peaks at the correct sign.

h_img, w_img = gray.shape[:2]
center = (w_img / 2.0, h_img / 2.0)

def rotate_deg(img, angle):
    """Rotate about the frame centre onto a white border."""
    M = cv2.getRotationMatrix2D(center, angle, 1.0)
    return cv2.warpAffine(img, M, (w_img, h_img),
                          flags=cv2.INTER_CUBIC,
                          borderMode=cv2.BORDER_CONSTANT,
                          borderValue=(255,))

def level_score(img):
    profile = ink_of(img).sum(axis=1, dtype=np.float64)
    return float(profile.var())     # peaks when rows align

skew_deg = estimate_skew_deg(gray)  # float, degrees
deskewed = rotate_deg(gray, skew_deg)
if level_score(deskewed) < level_score(rotate_deg(gray, -skew_deg)):
    deskewed = rotate_deg(gray, -skew_deg)
    skew_deg = -skew_deg

White padding is not cosmetic: black corners threshold into solid blobs that drag both the estimator and the OCR. The rotate-then-read order is the same discipline as correcting camera frames before barcode decoding.

5. Prove it before it runs unattended

level_page = cv2.imread("level_table.png",   # uint8, same frame size
                        cv2.IMREAD_GRAYSCALE)
tilted = rotate_deg(level_page, 3.0)         # known +3.0° warp
assert abs(estimate_skew_deg(tilted) + 3.0) < 0.5   # sign pinned

assert abs(estimate_skew_deg(deskewed)) < 0.5       # residual budget

The first assert pins the sign and magnitude of your OpenCV build to a number: if a release flips the minAreaRect convention, CI fails here instead of rotating every page 90° on the line. The second asserts the ±0.5° budget on real output.

6. Cut the grid and read each cell

With the page level, the rules are straight peaks in the ink projections:

row_ink = ink_of(deskewed).sum(axis=1, dtype=np.float64) / 255.0
col_ink = ink_of(deskewed).sum(axis=0, dtype=np.float64) / 255.0
rows = np.where(row_ink > 0.5 * w_img)[0]    # >50% ink = rule row
cols = np.where(col_ink > 0.3 * h_img)[0]

def centers(idx):
    """Group consecutive rule pixels into band centres."""
    bands = np.split(idx, np.where(np.diff(idx) > 5)[0] + 1)
    return [float(np.mean(b)) for b in bands if b.size]

Cells are the rectangles between adjacent centres. Crop each with a 3 px inset so rule edges stay out, then read the crops one at a time — the crop-then-read order that also drives the detection-to-OCR pipeline:

import pytesseract
text = pytesseract.image_to_string(cell_img, config="--psm 7").strip()

--psm 7 reads the crop as a single line; use --psm 6 for wrapped header cells.

Worked example: photographing the page

Numbers for a camera station reading an A4 sheet, portrait, with 15 mm of margin per side — a 240 mm × 327 mm field of view.

The stop-and-read station camera is the LS-CMR-0166: a 12 MP USB3 camera with a 1/1.7" sensor, a C mount and 30 fps, printing 4000 × 3000 px frames.

The 3000 px side across the 240 mm width gives 12.5 px/mm; 4000 px across 327 mm gives 12.2 px/mm, so the width sets the scale. A 1/1.7" format spans roughly 7.6 mm on its wide side, so the magnification is m = 7.6 mm / 240 mm = 0.032. At a 400 mm working distance the focal length is f = 400 mm × 0.0317 / 1.0317 = 12.3 mm: fit a 12 mm C-mount lens and refocus, and the working distance becomes 12 × 1.0317 / 0.0317 = 390 mm.

Text size: a 10 pt body face has an x-height near 1.8 mm, which at 12.5 px/mm is 22 px. Hold x-height above 20 px and binarized strokes stay 2-3 px thick. The camera resolution calculator resizes this for other sensors and type sizes.

Motion: at 100 mm/s a sheet travels 100 mm/s × 0.001 s = 0.1 mm in a 1 ms exposure — 1.25 px of blur at this scale, so cap the exposure at 0.8 ms for 1 px. Make it a stop-and-read station: this sensor is a rolling shutter, and a rolling shutter images a moving sheet sheared, a distortion no single rotation removes. Lines that must read on the fly pick a global shutter model from the area scan camera finder.

Where the station has no PC, run everything on an x86 smart camera. The LS-SMC-0017 is an x86-platform device with an 8.8 MP global shutter sensor and a C-mount; an x86 host is PC-class, so the Python, OpenCV and Tesseract stack from the steps above runs on the device — deskew, grid and cell reads included.

When a rotation is the wrong correction

• Keystone, not tilt. Shoot the page square-on. When the left and right rules differ in length by more than 2%, the camera sees the sheet in perspective and the far edge is genuinely smaller — rotation cannot restore it. Detect the four table corners and remap with cv2.getPerspectiveTransform and cv2.warpPerspective.
• Curved, not tilted. A page curled in the scanner or wrapped on a cylinder bends the baseline; per-line unwrapping is the tool, as in deskewing curved cap text into one line for OCR.
• Level table, rotated glyphs. Header rows printed at 90° need their own crop rotation — segment first, then rotate, the order the vertical text with OCR guide uses.
• Random digit rotation inside cells. Deskew the page, then each digit crop before reading or training, as in reading randomly rotated digits with OCR.

Mistakes that cost read rate

• Estimating from one line. An underline or the table frame is just one line, and one bad measurement wins when n = 1. Every estimator above needs at least six lines.
• Deskewing cell crops. A cell is 200 px wide or less — below the blob length that resolves 0.5°. Deskew the page once; crop after.
• Cropping to the table before rotating. Rotation loses the far corners first, and the corners are where the drift budget is spent. Rotate the full frame, then cut the cells.

Pre-line checklist

• Capture or render at 300 dpi or 12 px/mm: strokes 2-3 px, line blobs 200 px and longer.
• Synthetic assert green: a known +3.0° warp returns −3.0° ± 0.5°.
• Residual on 20 production frames: every one inside ±0.5°.
• Warp output on a white border, saved as PNG, cells inset 3 px.
• Ten cell reads spot-checked against keyed entries before the station goes live.

Where to go next

The decision per case: an upright table in a text PDF goes to Camelot, lattice or stream; a scan or photo at a stop-and-read station runs the six steps above with the camera arithmetic as sized; a station without a PC carries the same Python stack on an x86 smart camera; a barcode column instead of printed text goes to fixed code readers, which need none of this geometry.

The code-reading topic page collects the OCR-side guides, and the camera configurator sizes the capture side in one pass. Bring a sample frame and its row pitch to the contact page and we will run the estimate with you.