Find object orientation angle from a camera for pick rotation

You have a part sitting in a nest or on a stopped belt, a servo or robot holding a vacuum cup, and one question: which way is it pointing? One top-down camera answers that from a single image. Here is the method — threshold the silhouette, fit a rotated minimum-area rectangle, resolve the 180-degree flip — with the numbers that tell you whether the angle is good enough to drive the rotation, and the compute tier each variant needs. On a 2 MP sensor over a 113 mm field, a 40 mm part resolves to about ±0.2 degrees, ten times finer than the suction pick needs. The same math covers cloves, connectors, cut pieces and any mostly convex part whose two ends differ.

The method, and when it is the right one

Commit to the method before the hardware. If the part reads as one connected silhouette whose outline carries the taper — a clove, a wedge, a connector shell — a rotated minimum-area rectangle fitted to the binary mask returns the heading in one pass, to a fraction of a degree. A principal-axis fit on the same mask (the second-order image moments) does the same job when the outline is oval with no straight edges.

Three cases need something else. A round silhouette carries no heading at all: shape gives nothing, so you are detecting a feature instead. Ends that differ only in colour or texture make the mask symmetric, and no rectangle will sign it. Overlapping parts merge into one blob whose angle is meaningless — singulate first.

Two related guides cover adjacent ground: the screw-orientation walkthrough applies the same fit to one specific part, and symmetric-part orientation matching handles the taper-too-subtle case. For the full pick cycle around this angle step, see building a robot vision system for object location and gripping.

What you need

• A camera mounted square over the nest, optical axis vertical to within about 2 degrees.
• A dark matte backdrop or a backlight aperture, plus a diffuse light: the mask lives or dies here.
• Compute for a threshold, a rectangle fit and a serial write — far less than most people budget.
• A rotation axis (servo, stepper or robot wrist) with the cup offset measured from the rotation centre.

The compute decision splits by method, not by brand. A silhouette fit is a few thousand integer operations per frame, so an entry vision sensor does the whole job on-device and hands the controller one number. The MV-SCA002M-03S-WBN-SR is that tier: a 0.3 MP mono global-shutter sensor with a fixed 3 mm lens focused at 240 mm, an M5.8 mount and Fast Ethernet. For one part on a fixed nest, that is ample resolution.

If you would rather not spec a separate light, an integrated-light smart camera covers it. The LS-SMC-0063 combines a 1.6 MP sensor, a 16 mm lens, 60 fps and a built-in white light with polarized and transparent modes, on GigE.

When the field must be wider or the part smaller, add pixels and keep the shutter global. The MV-SC3050XC-16M-ATN-H is a 5 MP color vision sensor in 1/1.45" format with global shutter, M12 mount and Fast Ethernet. The extra pixels let you widen the field before the part drops below the pixel length that sets angle accuracy.

And when the focal length must match the field math exactly, a C-mount body accepts any lens: the MV-CU020-90GC is a 2.5 MP color GigE camera with a 1/1.7" global-shutter sensor. The smart camera and area scan camera ranges, filterable by sensor, shutter and mount, sit in the camera finder.

Sizing the camera for a 40 mm garlic clove at a 200 mm working distance

The inputs, and where each comes from:

• Part: a large clove, about 40 mm long and 20 mm wide — typical for the species; measure three from your own line and use the largest.
• Field of view: the part's diagonal is √(40² + 20²) = 44.7 mm; doubled for nest margin and placement drift that wants about 90 mm, so take a 100 mm square field.
• Working distance: 200 mm, set by the bracket height.
• Sensor: 1/1.8-inch class, 7.2 mm wide, 1600 × 1200 pixels (2 MP, 4:3) — the format the worked numbers below assume.

The rule, in words: magnification is sensor width divided by field width, and focal length follows from magnification and working distance.

The arithmetic: m = 7.2 / 100 = 0.072, so f = 200 × 0.072 / 1.072 = 13.4 mm. C-mount primes come in standard steps and 13.4 mm is not one of them, so take 12 mm and check the field it gives: FOV = 7.2 × (200 − 12) / 12 = 112.8 mm, call it 113 mm. Ground sample distance is 112.8 / 1600 = 0.0705 mm per pixel, so the clove's 40 mm length spans 40 / 0.0705 = 567 pixels.

Angle accuracy follows from the pixel length. A one-pixel edge error at each end of the blob swings the heading by their sum over the blob length, in radians, so in degrees:

Here: 57.3 × 2 / 567 = 0.20 degrees. The same line makes a working rule — angle error ≈ 115 / L_px degrees — so keeping the part at least 300 pixels long puts you at ±0.4 degrees or better before lighting effects.

Decision, with the threshold: a suction pick on a part this size tolerates about ±2 degrees, because a cup line offset 20 mm from the centroid drifts 20 × sin(2°) = 0.70 mm at that error and the placement never notices. The camera's ±0.2 degrees is ten times finer, and a stepper at 1.8 degrees per step with 16-times microstepping gives 0.1125 degrees, a matching axis. Widen the field to 400 mm and the blob falls to 142 pixels, so error rises to 57.3 × 2 / 142 = 0.8 degrees — still inside the pick tolerance, past comfortable. Solve the focal length for your own field in the lens focal length calculator:

From image to signed angle in six steps

1. Mount and frame. Fix the camera square to the nest, set the working distance from the sizing above, and draw the ROI around the nest. If parts wander inside the field, anchor the ROI to a detected feature — the blob-anchored dynamic ROI method is the standard trick.
2. Light and threshold. Dark matte backdrop with a diffuse ring light, or backlight through an aperture in the nest. Set the threshold midway between part and background grey levels on the histogram, not at a remembered constant.
3. Clean the mask. Fill holes (a specular highlight on waxy skin punches them), drop specks below about 1 percent of the part area, and keep the largest connected blob only. Measuring blob area by pixel counting is the tool for the area checks.
4. Fit the rectangle. A rotated minimum-area rectangle on the mask returns width, height and heading; read the long side's angle against the camera x-axis and normalize to (−90°, +90°], because every library orders the convention differently — OpenCV included. For an oval with no straight edges, the principal axis from the central moments is equivalent: θ = 0.5 × atan2(2·mu11, mu20 − mu02).
5. Resolve the flip. The rectangle heading is defined modulo 180 degrees. Split the blob along the line through its centroid perpendicular to the long axis and compare half areas: on a tapered part the blunt half owns more pixels, and that signs the angle.
6. Transform and command. Add the taught camera-to-tool offset, add the target heading, and normalize the command to the shortest turn — the tool-rotation normalization method stops a 170-degree correction from running the long way round. The smart-camera socket handoff for a straight-down pick shows the controller side of this step.

Verify before it drives anything

Present a part at known headings in 30-degree steps across 180 degrees — an indexable nest, or one part glued to a protractor disc under a clear plate. Run twenty repeats per step. Expect repeatability within about ±0.3 degrees and a systematic offset under 0.5 degrees, which the taught offset absorbs. Error that grows with angle rather than staying constant means the camera is not square to the nest: fix the mount, not the math.

Then close the loop and watch for drift. A vision pick error that appears under rotation usually traces to the transform, not the fit, while error that grows across a shift traces to lighting — controlling ambient light variation matters the moment a skylight or an open door enters the picture.

Where the silhouette answer fails

A round part is the clean failure: nothing in the outline carries a heading. For the tomato case — orienting fruit by the stem — detect the stem instead. Under light tilted about 45 degrees from vertical, a stem 3 mm tall casts a 3 mm shadow beside the blob, and the shadow's centroid gives the direction. If the ends differ only in colour or texture, the mask cannot help; you are into grayscale cues or a small learned classifier. The pragmatic shortcut is marking one end with coloured tape and using a colour-blob sensor — it works, at the cost of touching every part.

Three environment failures are cheaper to fix than to tolerate. Waxy skin reflects the ring light into the lens and punches holes in the mask; a polarizer or a more diffuse source removes it. A moving belt smears the taper: at 100 mm/s belt speed with a 1 ms exposure, blur is 0.1 mm, about 1.4 pixels at the worked example's resolution — shorten the exposure, strobe the light, and prefer a global shutter. Overlapping parts merge into one meaningless blob; singulate upstream or detect several rectangles and keep the well-separated ones.

Which compute tier runs the fit

The silhouette tiers need nothing exotic — a threshold, a rectangle fit and a serial write run on an entry processor. The learned tier is different: inference wants memory and a real CPU. The MV-VC2140-128G40-1T fits that step, with an Intel N-series processor, 8 GB memory, 3-4 LAN ports, 2 USB 3 ports and 4 built-in light channels. Picking the network is its own exercise — choosing a lightweight CNN backbone for embedded inference starts there — and the vision controller range covers that tier.

Checks before you order

• Field: the part spans at least 300 pixels along its length — that is your ±0.4-degree line.
• Shutter: global if the part moves during exposure; rolling is fine on a stopped nest.
• Mount: M12 smart cameras take fixed focal-length lenses only; if the field math lands between standard focal lengths, C-mount wins.
• Backdrop: a dark matte surface or a backlight aperture, drawn into the fixture, not added later.
• Light: diffuse; add a polarizer for waxy skin and an oblique channel if a feature carries the heading.
• Compute: the silhouette fit runs anywhere; a learned cue needs the controller tier.
• Cup offset: measured from the rotation centre, not guessed — it sets what each degree of error costs in millimetres.

Spec the camera and lens first, then the light, then the compute; the configurator follows that order. The other applications guides cover every link in this chain, and if a set-up refuses to cooperate, the contact page reaches the applications team directly.