Build a software auto exposure algorithm: the control law

Camera-internal auto exposure tunes itself for a pleasant picture: an unknown target, values that drift with the scene, and on some stereo depth modules no way to read back what the color sensor is doing while AE runs. A production cell needs the opposite — an exposure you chose, can log, and can reproduce after a restart. You can build that in software. Grab frames, measure the mean brightness of a fixed region, and steer the camera's ExposureTime and Gain with a clamped ratio law. This page gives you the control law, the step clamps, the exposure cap derived from your own belt speed, and a worked example that converges in two exposure changes. The steps use standard GenICam features, so the same loop runs on GigE and USB3 cameras alike.

Why the camera's own AE works against you

Built-in auto exposure solves a photography problem: hold the scene near mid-gray so the picture looks right. That carries three costs on a line. The target is hidden — you cannot read what brightness the camera is aiming at, and on some stereo depth modules the applied color-sensor exposure is not reported at all while AE runs. The measurement moves — when the part fills more of the frame, AE re-balances and your defect thresholds drift with it. Convergence is slow: a patent study of embedded AE chipsets documents at least 18 frames to converge under common lighting, with some scenes over 30. The fix is to turn every auto mode off — exposure, gain, iris, white balance — and run a loop you own. The disable-every-auto case is laid out in camera brightness drifts over time: turn every auto mode off, and the same reasoning covers automatic iris, exposure and gain control.

The control law: one ratio, two clamps

For a linear sensor at fixed gain, mean brightness is proportional to exposure time. That single fact is the whole algorithm: to move the measured mean from B to the target T, multiply exposure by T / B. On an ideal linear pipeline one step lands exactly on target; the clamps exist because real sensors deviate a few percent and every measurement carries noise.

Defaults that hold across inspection work: T = 90 on an 8-bit scale, following the 0.35 × full-scale target of a published histogram-based AE study (0.35 × 255 ≈ 89); a deadband of 0.95 to 1.05 on r; an exponential average weighting each new measurement at 0.3. With the factor-2 clamp, exposure changes from a cold start run close to log2 of the initial brightness error — a ×8 error costs three changes, a ×64 error costs six. For a working implementation against raw streams, pair this law with the bounded-loop code for a raw camera; the derivation above is what that code assumes.

What you need

Four things, all standard on PC-attached area scan cameras:

• Settable exposure and gain. The GenICam features ExposureTime (float, unit µs) and Gain (float, unit dB) on any GigE Vision or USB3 Vision camera, writable from your grab loop.
• Early frame access. Raw or mono 8-bit data, or luma from the color pipeline, so you can average pixels yourself.
• A fixed measurement region. The part area in image coordinates, agreed with the downstream inspection.
• Two numbers from your line. The fastest the part moves past the sensor, in mm/s, and the blur you tolerate, in pixels.

Any camera exposing those two GenICam features runs this loop. Driven over GigE with a global shutter, this 2.5 MP body stands in for the moving-belt bench:

With motion out of the picture the exposure cap loosens and resolution can rise; this 20 MP USB3 camera covers the static bench:

The camera finder filters by interface, shutter and resolution when either bench needs a different sensor, and the configurator walks sensor, lens and light together.

Steps 1 to 3: measure, read back, step the exposure

1. Fix the measurement region and the target before the loop runs. Measure the part area only — a fixed ROI in image coordinates, never one recomputed from detection results, which feeds the loop its own output and destabilizes it (why detection-driven AE ROI destabilizes). Set T = 90 on 8-bit data unless the inspection says otherwise: dark-field and edge work sits at 40–60 because specular highlights drag a mean upward, and finding dark parts on a light background runs at 120–150 (choosing the brightness target). On a shiny part use the median rather than the mean. On 10-bit streams, unpack to 16-bit planes before averaging and scale the target the same way: 0.35 × 1023 ≈ 358 (the unpack order for 10-bit RGB data).

2. Read ExposureTime and Gain back from the camera every cycle. Steer from the applied values, not from your last write — the applied value is what the sensor used, and frame metadata pairs it with the image it produced (reading exposure time from frame metadata). Log exposure, gain and measured mean for every frame from the first one; that log is the evidence when a cell misbehaves months later.

3. Multiply exposure by target ÷ measured, clamped to a factor of 2, then re-measure. The worked run, target T = 90, starting at ExposureTime = 1000 µs and Gain = 0 dB:

Two exposure changes across three measured frames. The linear model predicted a mean of 80 after the first step (40 × 2.0) and the sensor returned 78 — a 2.5% deviation the clamp absorbs.

Steps 4 to 6: cap exposure, fall back to gain, settle

4. Cap ExposureTime at the motion-blur budget before the loop starts. Blur in pixels equals part speed on the sensor times exposure: blur_px = v_px_per_s × E_seconds, so E_cap = blur_budget_px ÷ v_px_per_s. Worked numbers: a belt at 100 mm/s under a camera whose 4000 pixels cover 100 mm of belt gives a pixel pitch of 100 mm ÷ 4000 px = 0.025 mm/px, so the part crosses 100 ÷ 0.025 = 4000 px/s. A 1-pixel budget sets E_cap = 1 ÷ 4000 s = 250 µs. Cap at the smallest of that, one frame period (33 ms at 30 fps — in production the blur budget almost always binds first) and the sensor's maximum exposure. If 250 µs leaves the image too dark, the answer is a strobed light, not gain (how strobe pulse width freezes motion blur). The cap logic itself is the subject of capping the maximum auto exposure time, and soft edges in saved images are the symptom that a cap is missing (diagnosing motion blur).

5. Only after exposure reaches its cap, raise Gain — 6 dB at a time. Gain amplifies noise along with signal, so exposure always spends first. With E held at 250 µs, the bench measured B = 11 at Gain = 0 dB. The required linear factor is 90 ÷ 11 = 8.2, which is 20 × log10(8.2) = 18.3 dB. Applied in 6 dB steps with a re-measurement between each: 6 dB, 12 dB, 18 dB. At 18 dB the linear factor is 10^(18/20) = 7.94, so the mean lands at 11 × 7.94 ≈ 87 — inside the 5% deadband (|90 ÷ 87 − 1| = 3.4%) and the loop stops. Keep the final gain at the minimum that holds the target.

6. Settle, hold, stop stepping. Smooth the measurement with the exponential average, measure only on settled frames, and hold whenever r sits inside 0.95 to 1.05. Under a hardware trigger take at most one step per part and expect a few parts of settle time at changeover (a trigger-paced grab loop in Python).

Verify it worked

Three checks, in order. Convergence: start the loop from a dark frame, an overexposed frame and a normal one; expect two to four exposure changes typically, and never more than log2 of the initial error. Image quality: shoot a gray step target after the loop settles and confirm every step still separates — the full procedure is in measuring camera dynamic range with a gray step target. Rate: run at production trigger rate and confirm no frames drop while the loop writes; per-frame logging of applied exposure and gain is the proof the values actually changed (verifying exposure and gain in per-frame metadata).

Common mistakes

• Averaging the full frame. Empty conveyor swamps the part and the loop "corrects" exposure for background. Measure the part ROI.
• Target above 160. No headroom remains; highlights clip and take defect signatures with them. Stay near 90 unless the inspection needs otherwise.
• Gain before exposure. You get noisy frames at exposures that exposure time would have fixed cleanly.
• Steering from your last write. Write-read mismatches happen; the value read back from the camera is the truth.
• Stepping on unsettled frames. The loop chases its own transient and hunts around the target.
• Leaving camera-internal AE on next to your loop. Two controllers fight over the same sensor, and yours loses.

When to switch the loop off

A live AE loop belongs at set-up and changeover, not during production. In a triggered, strobed cell, converge once, lock ExposureTime and Gain, and log them with every part — a real ambient change then shows up as drift in the log you can act on (controlling ambient light variation in pick and place vision). When one pass needs two exposures — bright for geometry, dark for surface detail — alternate them per frame with the camera's sequencer instead of steering live (alternating exposure and gain per frame). If a camera exposes no software exposure control at all, the workarounds are collected in no software exposure control: what to do instead. Smart cameras and vision sensors run their lighting logic locked at teach; this custom loop is the tool for PC-attached cameras — the smart camera category shows where that line sits.

To put the loop on a real cell: converge it once at set-up, log exposure, gain and mean brightness with every part, and treat a slow drift in that log as a maintenance signal rather than a tuning problem. For a second opinion on belt speed, pixel pitch or trigger wiring, the contact page reaches the applications team, and more camera-side setup reading sits on the cameras topic page.