Intermittent vision pick errors: diagnosis by cycle phase

An intermittent vision pick error is the fault that resists instinct: it passes 499 cycles, fails one, and by the time you reach the cell it has moved to a different part. When failures land on different shapes, sometimes between waypoints and sometimes right after a pick, the pattern itself is the diagnosis. Geometry faults stay put; timing faults wander. This guide gives you the wandering-fault method: tag every failure with its robot cycle phase, budget the vision loop's worst-case time, then fix what the phase points to — handshake timeout between waypoints, trigger overlap after a pick, cable integrity anywhere. The hardware that matters is the vision machine that owns the trigger, cables built for the motion and a bench stand that makes the fault repeatable.

Why does the pick fail on a different part every time?

Failures that move between part shapes and between phases — one pick dies between waypoints, the next right after the grip — rule the part and the vision model out and put the vision-to-robot handshake under suspicion, because that is the only path every one of those cycles shares.

A model or lighting fault clusters: the same shiny variant, the same pose, always at the vision request, always carrying a no-part or low-score result. A handshake fault scatters. A stop between waypoints happens before any image exists, so no amount of model work can prevent it, and a failure right after a pick points at what the camera saw or what the robot received — not at the part's shape.

So the first move is negative: stop retraining, stop re-teaching pick points, stop quarantining part families. None of those actions touches the chain every failing cycle shares — request, image, result, grip command. The phase log in the next section decides which link in that chain breaks, and the fix is always cheaper than another training run.

How do you log which phase of the cycle the failure hits?

Tag every cycle with its phase and a millisecond timestamp from one clock — request sent, moving, gripping — and ten failures will usually show you whether the fault follows a phase, a moment in the shift or nothing at all.

Four fields are enough: cycle number, phase tag, robot clock in milliseconds, vision loop time in milliseconds. Save the camera image with every handshake result, pass or fail — a hardware-triggered grab loop in Python makes that automatic — so every failure arrives with its own evidence.

The table below is an illustrative log from a cell showing the fault:

Cycle 313 died with no reply at all: that is the handshake or the link. Cycle 314 returned a fast, healthy reply whose image could never hold a pickable part: that is trigger timing. Two failures, two different causes, and the phase column told you before anyone touched the cell.

Sort your own failures the same way. Faults bound to one phase point at triggers and stale results. Faults with no reply at all point at timeouts and links. Faults clustered late in the shift point at heat or load growth. Faults scattered across everything point at the physical layer: cable, connector, power.

When failures are too rare to chase live, take the loop to a bench — the camera stands hold one fixed pose while you run hundreds of cycles with the log live. The LS-MNT-0040 slide-table stand keeps that pose with a 700 mm column and a 400 × 450 mm base, so geometry cannot drift between attempts.

What does a failure between waypoints point to?

A stop between waypoints means the robot asked and nothing came back in time: the handshake timeout expired, the socket or link dropped, or the vision job was no longer running — check those three in that order before touching the image.

Work the causes in order:

1. Timeout under the worst case. The programmed timeout is shorter than the slowest vision loop. Log a few hundred loop times, compare the maximum with the timeout, and if the timeout loses, the timing budget two sections down sets it right.
2. Request collision. A new request arrives while the previous job is still running and queues behind it. Compare request and reply timestamps per cycle and look for overlaps; fix with busy and ready flags in the handshake.
3. Link or socket drop. Clean timestamps but no reply — the cable section takes over from here.
4. Job not running. The vision process died or the job unloaded after an earlier fault. Check process uptime after every fault, and make the first cycle after any fault start with a full re-handshake — the re-handshake at cycle start guide covers the sequence.

If the counters are clean and the timeout has headroom, read the socket state itself: a robot socket send string that is not received is usually stale connection state, not bandwidth.

One more read of the log separates two faults that look alike. A no-reply timeout is a handshake fault. A reply of "no part found" means the loop is healthy and the restart handling needs work — restarting the cycle after a no-part result keeps a healthy loop from being blamed for a broken one.

What does a failure right after a pick point to?

A failure right after a pick usually means the camera was triggered while the scene was still moving or changing: the image catches the gripper, motion blur or a part mid-transfer, so the locate finds nothing and the loop reports an error that timing created.

Two variants dominate, and the saved fault image tells them apart on sight.

Trigger overlap. The pulse fired while the gripper or the part was still in the field. The image shows blur or occlusion, the locate finds nothing, and the fault is motion's, not the part's. Trigger from the robot's in-position output instead of a timer, give the axes 50-100 ms to settle, and keep the exposure near 1-2 ms: a 1 ms exposure freezes the residual vibration that a 15 ms exposure smears into grey.

Stale result. The image is clean but the reply carries the previous cycle's measurement, which validation rightly rejects. Compare the result timestamp with the request timestamp on every cycle; any reply older than its request is a stale serve. The fix is a fresh measurement for every pick, and the stale vision result guide lays out the change.

Both faults hide in sequencing. When the request overlaps processing and motion, the request timing guide maps the overlap, and sequenced exposure trigger timing shows how one extra pulse per cycle produces the same signature.

How long does the vision step actually take?

Budget the loop as exposure plus transfer plus locate plus reply: for a 2 MP GigE camera the transfer alone is about 20 ms, so a healthy loop lands near 50 ms and a robot timeout of 100 ms gives the 2× headroom you want.

Worked example: a 2 MP GigE camera, mono8, 1920 × 1200 pixels. Image size: 1920 × 1200 = 2,304,000 bytes = 18.4 Mbit. Transfer: 18.4 Mbit ÷ 900 Mbit/s ≈ 20 ms. Add exposure 1.5 ms, locate 25 ms, reply 2 ms: 1.5 + 20 + 25 + 2 ≈ 50 ms. A 100 ms robot timeout carries 2× headroom, which is exactly what you want.

Same cell, busy scene: the locate walks a cluttered frame and takes 95 ms instead of 25 ms. The loop becomes 1.5 + 20 + 95 + 2 ≈ 120 ms, the 100 ms timeout expires, and the robot faults — only on the busy cycles, which reads as random and is fully explained by the log.

The decision comes from the measurement, not the average: log 200 loop times, take the maximum, and set the timeout to at least twice it — 240 ms here, so program 250 ms. Cut the worst case itself with a fixed ROI over the pick zone: an 800 × 600 window is 480,000 pixels, about a fifth of this frame, and pixel-bound locate work falls with the pixel count. The formula estimates the transfer; the measurement on your hardware is the number that sets the timeout.

Give each camera its own LAN port so transfers never queue behind one another. The vision controllers exist for exactly this — one port per camera, built-in light channels on most models, filtered by port count, memory and processor when you browse the category.

Should the robot trigger the camera or send a software command?

Fire the trigger from a robot output: a hardware pulse reaches the camera in microseconds with no operating system in the path, while a software request can sit in a queue for tens of milliseconds — exactly the jitter that turns a marginal timeout into an intermittent fault.

Trace both paths once and the difference is plain. Software: robot socket, operating system scheduler, camera SDK, exposure start — every hop adds milliseconds, so the same request lands 5 ms after the ask on one cycle and 40 ms after it on the next. Hardware: robot output to the camera trigger line, microseconds, and it fires only when the axes report in position. A software command is acceptable when the cycle carries half a second of slack and nothing is moving at the trigger moment; tighter cycles want the pulse.

Fire the light from the same event. A strobe inside a 1-2 ms exposure removes ambient flicker and motion smear in one step and makes every image in your log comparable.

Triggering, strobing and networking live on the vision machine, so pick one that owns that I/O directly. The MV-VC3301H-128G66 carries an Intel Core i3 with 8 GB of memory, at least five LAN ports so every camera keeps its own port, and four built-in light channels that fire the strobe from the same event as the exposure.

The vision system configurator walks the build camera by camera and shows the trigger wiring at the end.

How do you rule the Ethernet cable in or out?

Rule the cable out with counters and a swap, not with a guess: read the interface's error and link-reset counters around the failing cycles, wiggle the cable at both ends while someone watches those counters, then run a few hundred cycles on a drag-chain-rated replacement.

Run it as a sequence:

1. Read the counters. Note the CRC/error and link-reset counters, run the cell, and compare around the failing cycles. Errors at fault times are a physical-layer conviction.
2. Wiggle test. With the counters live, flex the cable at each connector and along its whole route. Any counter movement convicts that spot.
3. Swap. Run a few hundred cycles on a cable built for motion. A cable that has already worked in a drag chain keeps its habits even after re-routing, so the swap only counts with a motion-rated replacement.
4. Routing. Keep data cables away from motor power lines — a 100 mm gap is a fair rule of thumb on a machine frame — and cross at right angles where the paths must meet.
5. Heat. A connector hot to the touch under load is failing; thermal cable hotspot diagnosis explains why the hot spot is rarely where the fault is.

If the counters stay clean through all five, the physical layer is innocent and intermittent camera acquisition failure troubleshooting picks up the capture chain from there. The cables for machine vision are rated by motion type, and the cable step of the configurator matches cable to camera and controller.

Replace, do not repair. The LS-CBL-0199 is a 3 m high-flex GigE cable with screw locks, built to live in a drag chain; the LS-CBL-0117 is a 15 m RJ45 cable with a locking stud for longer static runs.

When is an intermittent pick error not a timing fault?

Three signatures say a fault is not timing: a pick that lands the same millimetres off every time is calibration, a failure on one part family with a clean image is model or lighting, and a fault on the first cycle after idle or alarm is leftover state.

Calibration. The pick lands the same millimetres off on every cycle, fault or no fault: the vision offset between camera frame and robot frame is wrong, and no timeout will move it. Fix it where the offset is configured — setting the vision offset and work object rotation for pick accuracy walks one robot language through it — and if the parts are round and symmetric, vision-guided robot repeatability with round symmetric parts explains the extra wander they add.

One part family only. Clean image, healthy loop, failures confined to one variant: a model or lighting gap, usually glare on a shiny finish or a feature the model never saw. The phase log proves it, because every failure carries the same part ID; fix the light or retrain, and the log confirms the fix within a shift.

First cycle after idle or an alarm. Leftover settings and half-closed connections produce exactly one failure per run. Start every run from a known state; the flaky test on leftover settings guide shows the reset sequence.

The phase log decides between them: an offset that repeats is calibration, a family that repeats is lighting or model, a first-cycle repeat is state — and everything scattered in between is the handshake, which sends you back to the timeout budget.

How do you prove the fix before the next shift runs?

Prove a fix with a counted run: 200 consecutive cycles at production rate with the phase log live and zero faults in the phase you fixed — and a full shift on top of that when the fault used to appear several times per shift.

Run the confirmation as a checklist:

1. Phase log live with no blind spots: every cycle tagged, every image saved.
2. 200 consecutive cycles at production rate, zero faults in the phase you fixed.
3. Worst-case loop time re-measured, timeout still at twice it.
4. Exactly one trigger pulse per cycle, counted at the camera input.
5. Interface error counters still at zero after the full run.
6. One forced robot fault, then a re-handshake and a clean pick on the next cycle.

Two clean runs in a row close the fault; a single return in the same phase means the fix addressed a symptom, so re-read the log instead of raising the timeout again.

Keep the log running after the fix too; the first recurrence then arrives as a timestamp instead of a mystery. If a log refuses to converge, bring it to us through the contact page — a page of timestamps usually decides it — and the robot vision systems topic page gathers the rest of these fault guides.