Variable waypoint reference frame: fix wrong-direction robot picks

Calibration passed, the camera finds the part, the taught grip is right — and the first vision-driven move still goes somewhere else. Shift the part across the field of view and the robot drifts off at an angle; rotate the part and the base joint swings while the gripper never turns. Both symptoms share one root cause: the pose numbers the vision step returns and the reference frame the move applies are not the same frame. This page shows how to read each symptom, which frame mismatch produces it, the 10 mm and 5° tests that isolate the fault in minutes, and the pose composition that puts the part rotation on the gripper where it belongs.

What a variable waypoint actually references

A taught waypoint stores joint angles and repeats the same motion forever. A variable waypoint receives a pose from a variable at runtime — the part pose your vision step writes — and the controller converts it to joint targets through inverse kinematics on every cycle.

A pose is six numbers, and the numbers answer nothing by themselves. Measured from where? That is the reference frame: the base frame, the active tool frame, or a named feature frame defined for the cell. The frame is not inside the numbers and rarely visible in the line that assigns the variable. It is a property of the move instruction — the coordinate-system or feature selector on the Move node — or of the pose type itself, depending on the controller family.

The vision step returns the part pose in the camera's own coordinate system, and nothing converts it automatically. Somewhere in the data path one transform — the calibrated camera-to-base result — must turn camera coordinates into base coordinates, and the move must consume numbers that match its own reference frame. Template pick programs from the camera vendor usually run that transform inside their library, so the numbers in the log look right; the frame selector on your own move node is the part most often left wrong. The same holds when the pose comes from one of the 3D sensors: stereo and structured-light models return a part pose in the sensor's frame just as a 2D camera does, and the camera finder filters them when the pose source itself is up for review.

If you are building the transform rather than debugging a template, the eye to hand object pose transform for robot picks guide shows the composition end to end, the guide on using eye-to-hand calibration results to compute robot-base picks covers the transform's destination, and the object pose in camera frame at runtime after hand-eye calibration guide shows what the pose variable should contain.

Symptom 1: the pick moves the wrong direction

Work the causes in this order; the first two explain most cases seen with template pick programs.

1. The move node references a different frame than the numbers. The library writes calibrated base coordinates, but the Move node still points at the frame the taught waypoints were created in, or at the raw camera frame. The numbers are right; the robot applies them from the wrong origin or a rotated one.
2. The camera-to-base transform never ran, or ran twice. Missing, the pose stays in camera coordinates and the robot chases it through the rotated frame; doubled, the robot overshoots by roughly double the part offset.
3. Axis conventions. An image axis runs opposite to its base axis, or the camera sits rotated so that base +X is camera +Y. The distance is right and the direction is a constant 90°, 180° or mirror image off.
4. Units. Some controllers carry pose translations in metres while vision software writes millimetres. A 20 mm part shift then arrives as 20 m: the controller faults, or the arm lunges.

The test that separates them takes five minutes. Park at the taught pick pose, shift the part exactly +10.0 mm along a base axis you choose, measured with a ruler against a fixtured edge, then read the pose variable on the teach pendant.

• Pose variable rises 10.0 mm in the matching component and the robot still moves elsewhere: the numbers are right, so fix the move node's frame selector.
• Pose variable itself is wrong — an extra component moves, or the sign flips: fix the transform, starting with the mounting angle and axis signs.

On controller families that let a program value set a move's base frame, the vision pose can drive that assignment directly; the knowledgebase page on configuring a robot base frame with a program variable shows the pattern.

For a camera rotated ψ about the vertical, with its origin at (X₀, Y₀) in base coordinates, the transform you are checking is:

A ψ of 90° swaps the axes, so a 10 mm shift along the camera's X drives the robot 10 mm along base Y; a mirrored Y flips the sign. When the observed motion matches one of those shapes, the mounting angle or axis sign inside the calibration data disagrees with the physical camera. Correct ψ and the signs once, in the transform, and the wrong-direction motion disappears.

Symptom 2: the rotation turns the base joint instead of the gripper

The part's angle θ about the table normal is an orientation change, and only one of the three places it can end up is right.

• Right: the pose orientation. Compose θ onto the taught orientation about the tool axis — equivalently about base Z when the tool points straight down, sign handled below. Inverse kinematics distributes the turn across the wrist joints, the gripper spins in place, and joint 1 does not move.
• Wrong: joint space. The template exposes the angle as a separate number and the glue code adds it to a joint field or a joint target built from the taught joints. Joint 1 turns, the arm swings, and the gripper keeps its orientation relative to the arm while the part rotates under it.
• Wrong: base-frame rotation of the whole pose. Rotating the taught pose about base Z rotates position and orientation together. The TCP traces an arc around the base centre at the taught radius, joint 1 visibly turns, and the gripper arrives beside the part instead of on it.

The tell for the base-frame composition is geometric: the TCP error grows with the taught radius, at r × θ per radian of part rotation. At a 400 mm radius, 5° of part rotation swings the target 34.9 mm — a guaranteed miss that reports as 'rotation sent to the base'. If the spin is correct but picks still drift with part rotation, the residual is usually position rather than frame, and the vision pick error under rotation guide separates the two.

The fix is identical in both wrong cases: keep the target a pose, put θ into its orientation about the tool axis, and let the controller's inverse kinematics choose the joints.

Worked example: a 5° part rotation must spin the gripper, not swing the base

Inputs, from a typical table-top cell of this kind: taught pick pose in base coordinates X = 400 mm, Y = 0 mm, Z = 120 mm, tool pointing straight down, taught orientation Rz = 0°. The part centre sits at the taught point, and the vision step reports a part rotation θ = +5° counterclockwise viewed from above, that is +5° about base +Z.

The rule: rotating a pose about a base axis moves its position along an arc of radius r through the angle θ, while rotating about the tool axis keeps the position and spins the orientation. A part that turned in place demands the second.

Wrong composition — rotate the whole pose in the base frame. X_wrong = 400 mm × cos 5° = 400 × 0.99619 = 398.5 mm. Y_wrong = 400 mm × sin 5° = 400 × 0.08716 = 34.9 mm. The TCP ends 34.9 mm from the part centre and joint 1 turns about 5°. The orientation is right and the position is not, so the pick misses.

Right composition — spin the orientation in the tool frame. Position unchanged: X = 400 mm, Y = 0 mm, Z = 120 mm. Orientation: Rz_base = 0° + 5° = +5°, which enters the tool frame as −5° about the tool Z, because the down-pointing tool axis runs opposite to base Z and the right-hand sign flips with it. The wrist joints take the 5°, the gripper turns with the part, and joint 1 stays at its taught value.

Decision: the part angle goes into the pose orientation about the tool axis — never into a joint field, never as a base-frame rotation of the whole pose. The threshold worth memorising: if a ±5° test rotation moves the TCP more than 1 mm, the rotation is composed in the wrong frame, because any taught radius above 11.5 mm already yields 1 mm of arc error (r = 1 mm ÷ 0.0873 rad). Every real pick radius is larger.

Fix procedure: six steps in order

1. Log the pose variable the vision step writes for an untouched part at the taught position. The numbers must equal the taught pose in the move node's frame; if they do not, repair the transform before anything else.
2. Run the +10 mm and −10 mm shift tests along base X and base Y. Each matching component must track the shift exactly; note any extra component that moves with it.
3. Run the +5° and −5° rotation tests at the taught point. The TCP must stay within about 1 mm and the gripper must spin the correct way.
4. Set the Move node's coordinate-system selector to the frame of the numbers you just verified, or transform explicitly to base coordinates and leave every move in the base frame.
5. Wire the part angle into the pose orientation about the tool axis with the sign check from the worked example, then rebuild the approach and retract points from the corrected target so the Z clearance survives; the variable pick approach offsets for vision-guided picking guide keeps those offsets consistent.
6. Run five consecutive live picks with the part placed at three different stations in the field. All five must pick before the cell is called done.

If the program faults on an undefined waypoint variable instead of moving, the pose assignment runs after the move or in a scope the move cannot see; the walkthrough on resolving an undefined waypoint variable with a pose-add call matches that case.

When recalibrating is the wrong turn

Re-running the calibration plate does nothing for a frame-selector bug, and it is the most common time sink in this fault. The camera was already reporting sensible numbers: the part was found and the taught grip matched. Recalibrate only when one of these holds.

• The reported pose misses by a nearly constant offset at every position in the field. That is an origin error: adjust the transform's translation and keep the plate result.
• The camera or its mount was bumped, replaced, re-focused, or the lens changed.
• The robot changed — a joint swap, a collision, a new tool mass shifting the kinematics.

After any of those, work through the vision calibration recovery after camera or robot changes sequence before trusting old taught poses. If the gripper was changed too, re-check the tool frame first with the robot-held camera calibration with a TCP tool frame guide: a wrong TCP bends every composed rotation the same way, which mimics a frame bug.

Prevention: one frame convention per cell

Four habits keep this fault out of the next cell.

• Transform once, immediately after the vision step, into base coordinates; every downstream move consumes base-frame poses.
• Keep the move node's coordinate-system selector equal to the numbers' frame across the whole program; where feature frames are mandatory, define them from the calibration result instead of teaching them by jogging.
• Record the camera mounting convention — rotation ψ and any flipped axes — on the cell drawing; it is the first thing to check when a pick drifts.
• Add the 10 mm and 5° tests to the commissioning sheet, and log the pose variable beside the received target for the first production runs.

For parts that look alike at several angles, set the rotation tolerance before chasing sub-degree behaviour; the symmetric part orientation matching guide shows how to make the vision step report one decisive angle instead of an ambiguous one.

Where the fix loop runs

The tests, the logging and the transform all execute in one machine: the vision controller that hosts the vision software, holds the calibration result and talks to the robot over its own LAN port. Start from the vision controllers category page and filter by LAN port count, PoE camera ports and memory.

A single-camera pick cell fits the VC3000 line, which starts with 5+ LAN ports, an Intel Core i3, 8 GB of memory and the VM6200Pro vision software licence included.

Cells reading several cameras or 3D sensors at once fit the VC5000 line, which adds PoE camera ports, 32 GB of memory, 8 built-in light channels and a 12th-generation Intel Core i5.

The decision, symptom by symptom

Every row reduces to the same discipline: know which frame the numbers are in, make the move agree, and put the part's spin in the orientation where inverse kinematics turns it into a gripper rotation. If the six steps leave the fault hidden, send the pose log and a sketch of the cell layout through the contact page and we will read the numbers with you.

To spec the controller that runs the test loop, the configurator starts at the vision controller step:

The systems topic page collects the neighbouring faults: hand-eye transforms, runtime poses and calibration recovery.