Move a robot relative to a vision-detected part frame

You taught the process path once — screwdriving, sealing, dispensing, marking — with the part sitting at its nominal position. On the floor the same part lands 10 mm to the side and 7° rotated, and every point of the path misses. The fix is not re-teaching: store each taught point relative to a frame fixed to the part, let a camera measure the part's offset and rotation every cycle, and transform the path before the robot moves. Here you get the transform itself, a worked example of what 7° of rotation does to a point 85 mm from the part origin, the locator accuracy the method actually needs, and the cases where a part frame is the wrong tool. It is written for the integrator moving beyond pick-and-place into feature-localised robot processes.

What the camera must deliver: a part frame, not a presence signal

The detector's job in this method is to return three numbers about a fixed reference feature set on the part: x, y and in-plane rotation θ. A presence result cannot drive a path, because it carries no geometry. The frame origin must also be a point that exists on every part you will run: a machined corner, two holes, a printed fiducial. If the origin wanders from part to part, the whole method inherits the wander.

How a fixed camera's raw measurement becomes a pose the robot can use is covered step by step in the eye-to-hand object pose transform for robot picks; the same pipeline feeds this method. Many industrial controllers shorten it further: the robot can build a user frame directly from the detected part, and converting an existing taught job to run against a reassigned frame is a built-in controller feature rather than a script you maintain. Whether the arithmetic runs in your code or in the controller, the maths below is the same.

The transform: one line of arithmetic per taught point

Teach the path with the part at a reference pose. Every taught point p_part is then stored relative to the part origin; at run time the camera returns t, the detected translation from that reference pose, and θ, the detected rotation about the camera's viewing axis. Each waypoint transforms as:

R(θ) is the planar rotation matrix. Compute the result once per cycle for all waypoints, then run ordinary absolute moves to the transformed points. Keep millimetres throughout, and convert before the trigonometry: θ in radians = θ in degrees × π / 180.

Two conventions cause most commissioning faults. Camera images put y pointing down while robot bases put y up, so a naive copy inverts the rotation; and rotation can be reported either as the measured part angle or as the correction angle — pick one definition and write it into the program header. With a wrist camera nothing changes downstream: the controller combines the flange pose it already knows with the snapshot and hands you the same t and θ.

Worked example: 7° of rotation is worth 10 mm

Teach time: the dispense start point P1 is stored at (80, 30) mm in the part frame. Its radius from the part origin is r = √(80² + 30²) = √7300 ≈ 85.4 mm.

Run time: the camera detects t = (+12.0, −5.0) mm and θ = +7.0°. cos 7° = 0.9925, sin 7° = 0.1219.

• x' = 80 × 0.9925 − 30 × 0.1219 + 12.0 = 79.40 − 3.66 + 12.0 = 87.7 mm
• y' = 80 × 0.1219 + 30 × 0.9925 − 5.0 = 9.75 + 29.78 − 5.0 = 34.5 mm

The robot moves to (87.7, 34.5) mm. A translation-only shortcut would aim at (80 + 12, 30 − 5) = (92.0, 25.0) mm and miss the correct target by √(4.3² + 9.5²) ≈ 10.4 mm — nearly as much as the entire 13.0 mm translation it did compensate (√(12² + 5²) = 13.0 mm).

The rotation term is what you cannot eyeball. Any point at radius r swings by 2 · r · sin(θ/2); here 2 × 85.4 × sin 3.5° = 10.4 mm, matching the calculation above. At 10° the same point swings 14.9 mm. Translation error stays fixed at the part's displacement; rotation error grows linearly with angle and with distance from the part origin. Points far out on the part are the ones the orientation accuracy must protect.

Size the locator from the worst pose, not the nominal one

Example part: 120 × 80 mm, landing anywhere within ±20 mm in x and y, rotated up to ±10°.

Field of view. The half-diagonal is √(60² + 40²) = 72.1 mm. Worst case, the farthest corner sits 72.1 + 20 = 92.1 mm from the window centre, so the field must cover 2 × 92.1 ≈ 184 mm across. Specify 200 mm and the geometry is settled for any pose in the window.

Resolution. Over a 200 mm field, a 5 MP sensor with 2448 px across resolves 200 / 2448 = 0.082 mm per pixel. Let the locator consume a fifth of the process budget: for a ±0.5 mm process that means 0.1 mm per pixel or better — met with margin — and a 0.5 mm part edge spans 6 px, enough for a stable subpixel edge fit.

Orientation accuracy. A rotation error δθ displaces a feature at radius r by r · δθ, so the requirement follows directly from the process:

With tol = 0.5 mm at r = 85.4 mm: δθ ≤ 0.5 / 85.4 = 0.0059 rad = 0.34°. Pixels are not what limits you: the taught pattern spans a radius of 72.1 / 0.082 ≈ 879 px, so 0.1 px of edge repeatability corresponds to 0.1 / 879 ≈ 0.00011 rad ≈ 0.006°. Lighting and calibration set the installed accuracy — budget the locator at 0.05–0.1° of rotation repeatability, a factor of 3–7 inside the 0.34° requirement — and hold the light steady, because a 50× pixel margin evaporates under lamps that swing; the controls are laid out in ambient light variation in pick and place vision. With a calibration plate and a rigid mount, residuals of 0.05 mm or better across a 200 mm field are routine; re-run the hand-eye calibration whenever the camera or its bracket is disturbed.

If the part moves during capture, add a blur budget of speed × exposure ≤ 1 px: at 100 mm/s and 0.082 mm per pixel, exposure stays under 0.082 / 100 ≈ 0.8 ms, which is why a global shutter and a short, bright flash belong in this spec.

The camera resolution calculator re-runs the field-to-pixel arithmetic on your own part and sensor; its button opens the camera finder set to cameras with enough pixels.

Fixed-mount or wrist-mounted camera

The fixture usually decides, not the camera. A defined window — located pallet, clamped nest — favours the fixed camera: no travel, no fingers in the image, one calibration. Loose arrival favours the wrist camera despite its snapshot time. Where parts arrive on a moving belt, the 2D or 3D camera for conveyor pick and place: choosing the system extends this comparison to the conveyor case, including trigger and encoder questions.

Parts for this set-up

The worked example needs a 200 mm field, 0.082 mm per pixel and a fraction of a degree of rotation repeatability — comfortably inside a modern 2D locator. Two candidates from the smart cameras category:

The MV-SC3050XC-16M-ATN-H vision sensor fits the class of job: 5 MP colour, global shutter, M12 lens mount, on Fast Ethernet.

The MV-SC6050C-08M-XBN packages the locator as one housing: a 5 MP colour smart camera with an 8 mm M12 lens, GigE interface, global shutter and a built-in multi-colour light source.

All 2 share: Sensor type CMOS, global shutter; Pixel size 3.45 μm × 3.45 μm; Sensor size 1/1.45"; Resolution 2432 × 2048; Mono/color Color; Power supply 24 VDC; Humidity 20% RH to 95% RH (no condensation); Certification CE, KC. Values as printed in each product's data sheet; open a part for its drawing and full specifications.

When parts tilt — more than a degree or two out of plane, or a process following a curved surface — move to a 3D sensor. The MV-DPS200P-02 projected structured-light 3D camera measures the surface itself, so the frame carries tilt and height as well as planar pose; the 3D sensors category holds the profiler and stereo alternatives.

The compute is modest either way: one template match plus a transform per cycle. The MV-VC2140-128G40-1T vision controller runs the detector and holds the taught programs — an N97 processor, 8 GB of memory, a 128 GB SSD plus 1 TB HDD storage, 3-4 LAN ports, 2 USB 3 ports and 4 built-in channels to drive lights. Cells with several cameras step up to the VC3000 or VC5000 series; the MV-VC5525-256G70-NN, for example, pairs an Intel Core i5 with 32 GB of memory, 5+ LAN ports, PoE camera ports and 8 built-in light channels. The full range sits in the vision controllers category.

Teach once, run anywhere: the procedure

1. Place a reference part at the nominal pose and capture the teach image. This image defines zero; every later t and θ is measured against it.
2. Hand-eye calibrate a fixed camera to the robot base (a wrist camera skips this; it uses the flange pose at each snapshot). Hold the residual around 0.05 mm across the field.
3. Teach the path with the part at the reference pose and store every point in the part frame — at teach time the part sits at the reference pose, so the stored base points are already part-frame points.
4. Gate the path on the detection: locate, check the result sits inside the window (|t| within ±20 mm, |θ| within ±10°), reject otherwise. The gate is what stops a bad frame from becoming a bad path; the layout thinking in structuring a vision-guided pick and place program around drop targets applies directly.
5. Dry-run at reduced speed with the warning below in mind, then release at full speed.
6. Prove the transform on a shifted part: move the part +10.0 mm in x and +5.0° with a staged fixture, run, and measure where the tool lands. It should track the shift within about a tenth — ±1 mm on a 10 mm move. The residual is calibration and detection noise, and it repeats: if the tool lands 0.8 mm short on every shifted test, correct the calibration, never the taught points.

Pitfalls that shift the path

• Sign and axis flips. Test before first production: shift the part +10 mm in x, run, and confirm the tool follows +10 mm. A flipped y or an inverted θ shows up in one test.
• A frame that went stale. Detection happens once per cycle; if a clamp, a chip or an operator moves the part afterwards, every transformed point is wrong by that motion. Locate after clamping, and if the process itself touches the part mid-path, re-locate before the touch.
• Symmetric parts. The detector can lock a square part 90° or 180° off, and the transform follows faithfully onto the twin feature. Either make the path identical under the symmetry — how much rotation tolerance a symmetric part really has tells you when that is legitimate — or break the symmetry for the camera with a notch, label or corner feature.
• Height drift. The planar transform leaves every taught Z untouched; a surface that wanders out of its process window leaves the path planar-correct and contact-wrong. Add a touch-off move or a 3D Z measurement.
• Slow drift over weeks. If paths start missing after months of good service, suspect bracket creep and lamp ageing before software; the diagnosis flow in vision pick error under rotation: diagnose position drift applies unchanged to paths.

When a part frame is the wrong tool

Three cases break the method, each with a known exit.

The feature moves on the part. Stampings, castings and weldments hold loose tolerance between the part outline and the feature you must hit; a perfect part frame then tracks the wrong thing precisely. Detect the feature itself — its own hole, notch or edge — and teach against that; the argument in teaching long thin parts to a vision system by feature, not whole part carries over to any loose-tolerance part.

The part tilts. The planar transform assumes one plane. A part rocked 2° out of level displaces a feature 85 mm from the origin by 85 × tan 2° ≈ 3.0 mm — six times the 0.5 mm budget. Measure the tilt with a 3D sensor and carry it in the frame, or fixture the part level.

The tolerance is tighter than any frame can carry. A process needing ±0.05 mm will not be held by a camera locator over a 200 mm field. Use the camera to bring the tool within reach, then finish on a mechanical datum, a probe touch-off or force-guided contact; the locator's job shrinks to coarse placement.

The decision, case by case

• Flat part, planar process, variation within ±20 mm and ±10°: a 2D locator and the part-frame transform. At the example's scale — 200 mm field, 0.082 mm per pixel, 0.05–0.1° installed rotation repeatability — a ±0.5 mm process at a feature 85 mm out holds with a factor of 3–7 in reserve.
• Tilted parts or curved, multi-face processes: a structured-light or stereo 3D sensor, with full pose carried in the frame.
• Feature position loose on the part: detect the feature, not the part.
• Several robots or several paths on one part: publish one measured frame per cycle with its identifier; every consumer transforms its own taught points from that single result.
• Fixed fixture, tight cycle: run detection during clamping; the match adds tens of milliseconds of compute and the robot never waits idle for it.

Build the chain starting at the camera finder, or step through the configurator below; the smart camera, 3D sensor and vision controller categories above carry the parts.

For a second opinion on a live cell, bring the part drawing and the process tolerance to the contact page. The wider integration set sits on the applications topic page.