CPU time vs wall-clock time in vision pipeline benchmarking

You benchmark a frame-differencing pipeline on an embedded vision controller. A stopwatch says the run takes 30 seconds; the timing code, built on the C clock() function, reports about 15. The pipeline did not get faster — the two numbers measure different things. This explainer covers what CPU time and wall-clock time each mean, why vision pipelines diverge between them most of all, how to measure both in C, C++ and Python, and how to turn the measurements into frames-per-second numbers you can trust when you compare a CPU implementation against a GPU one, or when you size a vision controller for a camera line.

What clock() actually measures

The C standard defines clock() as returning the processor time used by the program, divided into CLOCKS_PER_SEC units. Processor time is the accumulation of the intervals during which the operating system actually scheduled your process onto a CPU core. It is a resource-use counter, like kilowatt-hours on a meter, not a chronometer.

Everything your program does while not running on a core is invisible to clock():

• Sleeping, explicitly or waiting on a condition variable
• Blocking on file and disk I/O, including reading video from storage
• Waiting on a hardware video decoder or a camera's frame delivery
• Work delegated to the GPU, a DSP or another process

The simplest proof needs no camera at all:

A program that sleeps for ten seconds reports close to zero from clock(). A program that runs flat-out on one core with no blocking reports the full elapsed time. Real vision pipelines sit between the two, which is exactly why a 30-second run can report half of that.

Why vision pipelines diverge the most

Video processing is unusually rich in non-CPU time, so the gap between CPU time and wall-clock time is larger than in a typical compute kernel:

• Decode. Reading an MP4 file involves demuxing, and on many platforms part of the decode work happens in hardware blocks or in a separate media process. The CPU waits.
• Display. imshow and waitKey hand frames to a windowing system and block until it is done with them.
• I/O stalls. Storage reads, memory-controller contention with camera acquisition, and network frame arrival from a GigE camera all consume wall-clock time without billing CPU time to your process.
• GPU work. When you move the blur and threshold to CUDA, clock() around the CPU side shrinks further, because the CPU now spends its time waiting on the device. A naive CPU-versus-GPU comparison built on clock() will overstate the GPU speedup or, if the calls are asynchronous, hide it entirely.
• **Multitasking." Even pure CPU code loses time when the scheduler preempts your process for other work; the preempted intervals count as wall-clock time but not CPU time.

Measuring wall-clock time correctly

Use a monotonic clock. Monotonic means it only moves forward and is unaffected by system time corrections, so a time adjustment during a long benchmark cannot produce a negative interval.

In C or C++ on Linux:

gettimeofday() also works and is what most legacy examples use, but it reads the real-time clock, which an administrator or NTP can step backward. For benchmarks, prefer CLOCK_MONOTONIC; for per-frame timestamps that must survive across reboots, use CLOCK_MONOTONIC_RAW or the system real-time clock depending on what you correlate against. On Windows, QueryPerformanceCounter is the equivalent monotonic source; in Python, time.perf_counter() is monotonic by contract.

For a deeper treatment of timing measurements on control hardware, this PLC benchmarking method for CPU, I/O and scan time covers the same distinction for controller scan budgets.

A worked example: the 1999-frame run

Take the frame-differencing case from the opening scenario. The stopwatch says 30 s for 1999 frames; clock() says about 15 s.

Wall-clock throughput:

The pipeline spends 7.5 ms per frame of real CPU work and another 7.5 ms per frame blocked — on decode, on the display path in imshow, on storage. The diagnosis follows directly: either remove the blocking (decode to raw frames in advance, write results instead of displaying them) or overlap it (double-buffer, a decode thread feeding a processing thread). The two numbers together tell you which fix applies; either number alone does not.

If you need to know how many pixels a camera delivers and at what resolution to match that 66 fps budget, the camera resolution calculator turns a field of view and a defect size into a pixel count.

Turning timestamps into trustworthy FPS

A single start-to-stop average hides jitter, and jitter is what drops parts on a conveyor. Instrument per frame:

1. Record a CLOCK_MONOTONIC timestamp with each frame as it enters processing.
2. Compute per-frame intervals as consecutive timestamp differences.
3. Report the mean, the 99th percentile and the maximum interval, plus a count of dropped or late frames.
4. Repeat the run at least three times and compare medians, not single runs.

A pipeline averaging 66 fps whose worst interval is 200 ms still misses a part passing every 40 ms. Related symptoms show up in articles we already publish: if frames arrive at uneven intervals from the camera itself, the diagnosis in GigE Vision frame arrival jitter: isolate camera and host timing applies; if the host, not the camera, is starving the pipeline, start with camera acquisition high CPU usage: replace busy waiting. When color conversion limits the frame rate, see RGB camera frame rate limits: pixel format and exposure.

Choosing hardware with both numbers

CPU time tells you how much processor a pipeline needs; wall-clock time tells you whether a given processor delivers the frame rate. Both matter when you size a system:

• A smart camera such as the LS-SMC-0063 runs the whole pipeline on-device, so its embedded processor must cover the CPU-time budget at your target frame rate with margin for the on-board lighting and communication tasks.
• A PC-based camera like the MV-CS200-10UM or the LS-CMR-0166 shifts the CPU-time budget onto a host, so the host must be sized for the sum of all cameras, not one.
• A vision controller such as the MV-VC3301H-128G66 also services camera ports, light controller channels and PLC traffic, so reserve headroom: a utilization ratio consistently above about 0.7 per core at peak leaves too little for the acquisition and I/O your benchmark did not model.

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

You can assemble a camera-plus-controller combination in the system configurator, or browse the area scan cameras finder filtered by interface and resolution.

Checks before you trust a benchmark

• Pin the run to one core (for example with taskset) and repeat it; variance across runs points to interference, not algorithm cost.
• Keep display out of the measurement, or measure with and without imshow separately so you can quantify its share of the wall-clock time.
• Warm up before timing: first-frame costs include file opens, codec initialization and page cache faults that a steady-state run never pays again.
• Use the monotonic clock for intervals and the CPU clock for utilization, and label the two clearly in your results so nobody compares the wrong columns.
• Benchmark on the target hardware, with the real camera traffic or media load present; a quiet developer workstation flatters both numbers.

When the CPU-time number is the right one

There are cases where you genuinely want processor time. Sizing a processor for N cameras is a CPU-time budget: N times the per-frame CPU cost must fit into the cores available. Checking whether your OpenCV build parallelizes is a CPU-time question: if wall-clock time barely drops with four cores but CPU time stays flat, the kernel is single-threaded. Licensing a pipeline by compute is a CPU-time question. The mistake is only in reading clock() as a chronometer. Report both numbers, each labeled, and every one of these decisions gets easier.

A short checklist to carry into the next benchmark:

• Monotonic timestamps per frame, mean and worst interval reported
• CPU time recorded alongside, utilization computed as the ratio
• Display and decode either excluded or measured separately
• Multiple runs, medians compared
• Same hardware, same background load, as the production system

For help matching a measured budget to specific cameras or a controller, the contact page reaches our applications team, and volume pricing is available by quote.