Camera image timestamps frozen or jumping: fix the clock source

The symptom looks impossible: the image stream runs smoothly at full frame rate, while header.stamp on every message stalls, then jumps back or forward by 5-10 seconds. TF throws extrapolation errors, bag replay drifts, and two-camera sync drops pairs, yet the camera itself seems healthy. If you integrate depth cameras or area scan cameras into ROS for robot guidance or inspection, treat this as a time-domain problem, not a sensor fault: the stamp is a copy of one clock, and that clock was never mapped into the host's. Below is what the header stamp actually reads, the three domains a driver can choose, the arithmetic for a 30 fps stream that steps by 7 s, and the echo checks that confirm the fix in a minute.

What the header stamp actually reads

header.stamp is written by the driver, not measured by your application code. The driver can fill it from three honest sources: the camera's own counter, the host clock at the moment the frame arrives, or the camera counter mapped into host time by a global-time service inside the driver. Pixels and stamps travel different paths — pixels across the link, the stamp computed at publish — and nothing forces the two to agree.

A camera counter is just a number the firmware increments per frame. Until something maps it onto the host's clock, it counts device time, and a stamp copied straight from it can sit seconds away from rostime. When the offset estimate anchors badly — at stream start, after the host clock stepped, after a cold boot — copied stamps stall or bounce while frames keep flowing. Counters are also finite: what a stamp does when the device counter wraps around is its own failure mode, covered in camera timestamp rollover and monotonic time mapping.

Why a smooth picture proves nothing

RViz and every other viewer draw a frame when it arrives. The render path never reads the stamp, so a 30 fps preview looks identical with healthy time and with broken time. The failures appear downstream, in the consumers that read time: TF lookups, because TF keeps a 10-second history by default and an offset error of 5-10 s brushes the edge of that cache; message_filters, which silently drops pairs whose stamps disagree; bag playback, which rebuilds its timeline from stamps; and any two-camera comparison. A stamp that runs backwards is worse than a slow one: every consumer assumes time advances, so a backwards step is not smoothed over, it is an error.

Flickering or torn depth in the preview is a different fault again — that is the pixel path misbehaving, not the clock, and fixing flickering depth images in RViz covers that diagnosis. Smooth pixels plus failing TF is the signature that sends you to the stamp.

The three time domains, compared

Decide once which clock owns header.stamp. The three options differ in what they cost and what they buy:

Arrival stamping is the zero-set-up option, but the stamp inherits every queue on the link; the arrival behaviour of a GigE camera is its own diagnosis, covered in GigE Vision frame arrival jitter. Global-time mapping gives host-usable stamps at a few milliseconds of residual, which is why guidance drivers default to it — and why the failure confuses people: the pixels stay perfect while the mapping misfires. The same domain choice decides camera-and-IMU alignment once frame rates drop; camera and IMU timestamp clock domains at lower frame rates works that pairing.

How to tell which clock your driver uses

1. Echo the stamp field on one image topic: rostopic echo /camera/color/image_raw/header/stamp. That path is the common default namespace; use your driver's image topic. rqt_topic shows the same field without a terminal.
2. Watch host time beside it: rostime in a test node, or date +%s.%N on the capture host.
3. Compute deltas between consecutive stamps. A delta near the frame period — 33.3 ms at 30 fps — means mapped device time, healthy. A delta of zero across a run of frames means a frozen copy. A delta of seconds is the bounce in question.
4. Track the offset, stamp minus host time at receipt. Stable within milliseconds: the map is anchored. Drifting or stepping: the estimator re-anchors on every stream start, and every restart restarts the risk.

Keep the echo running across a restart: a fault that steps once a minute needs a minute of watching, not ten seconds.

Worked example: a 30 fps color stream that steps by 7 s

The case: a depth camera publishes color and depth at 30 fps, 1280 × 720, 8 bits per channel, over USB 3.0. The stream previews smoothly while stamps stall, then step back or forward by 5-10 s; take 7 s as the step to analyse. Bases for the inputs: the 30 fps rate, 1280 × 720 resolution and USB 3.0 link are illustrative defaults for this class of stream; the 5-10 s steps and the smooth preview are the observed symptom.

The rule: a mapped device stamp advances one frame period per frame. An arrival stamp advances one frame period per frame but lags the exposure by frame bytes ÷ link payload rate. A broken map shows deltas of zero, whole multiples of the frame period, or steps of seconds.

The arithmetic:

• Frame period: 1 ÷ 30 fps = 0.0333 s = 33.3 ms.
• Frame payload: 1280 × 720 × 3 B = 2,764,800 B ≈ 2.8 MB.
• Arrival lag: 2,764,800 B ÷ 400 MB/s — USB 3.0's 5 Gb/s line rate less encoding and protocol overhead — = 6.9 ms, about 8 ms with driver overhead.
• Healthy band: 33.3 ms ± 3 ms, so 30-36 ms per frame.
• One 7 s step: 7,000 ms ÷ 33.3 ms = 210 frame periods hidden in a single stamp jump.

Result: while the stamp sat still, the scene kept moving; 210 frames of motion folded into one step. A 5-10 s offset error also brushes TF's 10-second default history, which is when extrapolation errors begin.

Decision, with its threshold: motion work needs stamp error well inside the frame period — budget about 5 ms here, one-seventh of 33.3 ms. The arrival stamp costs ~8 ms before jitter, so it misses the budget for this stream; the broken map misses it by three orders of magnitude. Run the mapped device domain and confirm deltas sit in the 30-36 ms band. For logging static scenes, ordering is the only requirement and the ~8 ms lag is acceptable and free.

Choosing a domain and setting it

Make the host clock sane first: run an NTP service on the capture machine and keep it out of suspend while streaming. A host clock step writes the same symptom as a camera fault, and no driver can stamp against a clock that jumps. The signature reaches beyond vision: a PLC real-time clock that jumps ahead after a power loss tells time just as badly until it is re-synchronized.

Set the domain where the driver exposes it: most depth-camera drivers carry a global-time or timestamp-mode parameter among the launch options. The name changes from driver to driver; the function is the toggle between arrival stamping and mapping device time into the host domain. For anything that moves, leave it on the mapped setting, then prove it with the delta check above.

On GigE area scan cameras the hardware helps: GigE Vision 2.0 commonly carries IEEE 1588 (PTP) support alongside a GenICam timestamp register, so cameras and a PTP-capable switch settle on one clock. The camera finder filters area scan cameras by interface, so you can shortlist, then read each data sheet for PTP. USB3 Vision devices expose a device counter with no standardized host sync, so the driver's mapping is the whole bridge.

When milliseconds still are not enough — fast pick cycles, event ordering against the PLC — stop asking stamps to carry it and gate exposure in hardware instead: external flash sync using exposure time as your shutter shows the pattern. Depth-plus-IMU rigs align by interpolating across stamps; IMU and depth image timestamp interpolation covers that arithmetic.

Checks before you trust the stamps

• 100-frame delta run: mean within a millisecond or two of the frame period, spread of a few milliseconds. Catches freezing and stepping at once.
• Offset watch: stamp minus host time at receipt, stable within milliseconds over several minutes. Catches a slowly drifting estimator.
• TF lookup over a recorded minute: no extrapolation errors. Proves the consumers, not just the echo.
• Bag round trip: record one minute, play it back against the /clock topic. If playback runs at the wrong speed, camera recording faster than real time covers the timing errors behind it.
• Cold-boot repeat: deltas still in band after a full power cycle. A check that passed once proves little with this fault.
• Cross-camera delta: every camera in one domain, so the stamp gap for a shared event equals the known sync offset, not seconds. On a multi-camera GigE host the bandwidth budget sets the arrival lag first; two GigE cameras on one host works that budget.

When the usual answer is wrong

• It recovered by itself: nothing recovered. A re-anchored estimator or a host clock step removed the symptom while the domain stayed undecided. Keep the check in the start-up routine.
• Blaming the camera: the sensor delivered every pixel on time. The stamp is driver work; a camera swap changes nothing.
• Buying a faster camera to fix it: frame rate changes the frame period — 16.7 ms at 60 fps instead of 33.3 ms — not the domain. The bounce scales down and stays wrong.
• Ignoring exposure convention: most drivers stamp the start of integration. At a 1 ms exposure the bias is negligible; at 10 ms every event comparison is biased by 10 ms. Check the convention once and account for it.
• Expecting microsecond precision from mapped stamps: the mapping carries a residual of a few milliseconds. Microsecond alignment is hardware trigger territory, not a stamping fix.

The decision, in one pass

• Motion, guidance, tracking: mapped device domain. Deltas at the frame period ± a few milliseconds — the 30-36 ms band in the worked example. Set the driver parameter, run the delta check after every stream start.
• Static scenes, logging: arrival stamping is acceptable. Budget the lag — about 8 ms for the 2.8 MB example frame, about 110 ms for 12 MB on GigE — and accept that it cannot be repaired afterwards.
• Multi-camera: one domain for all, via PTP on GigE Vision or a shared trigger, then the same delta checks across cameras.
• Already-recorded data with broken stamps: re-stamp in frame order for viewing only, and retire it from timing analysis.

If you are selecting hardware with these time features, start from 3D sensors, narrow the field with the camera configurator step, and skim the timing guides on the systems topic page. To have a stamp trace looked over, send the camera, the driver setting and the delta output through the contact page.