Bilateral filter skin smoothing without the flat look

A beauty filter that runs bilateral blur and then lifts brightness flattens faces into wax: spots fade, but so do pores, eyelashes and jawline edges. The failure is not the algorithm, it is the settings and what gets blended. This guide gives the two-sigma arithmetic that drives edge-preserving smoothing, a worked example at 1080p30, and the frequency-separation pipeline that removes blotches while keeping texture. It is written for developers implementing smoothing in shaders on GPU video, and the same numbers apply when the goal is denoising an inspection image without softening the defect edges a detector needs.

What a bilateral filter actually weights

A Gaussian blur weights each neighbour only by distance: w = exp(−d² / (2σs²)), so nearby pixels dominate and detail spreads outward. The bilateral filter adds a second factor that falls with the brightness difference ΔI between the centre pixel and the sample. A pixel counts only if it is both close and similar; a strong edge fails the similarity test, keeps almost no weight and stays sharp. That second factor is the whole idea, and the bilateral filter article carries the formal definition. The weighted average replaces the centre pixel:

Two settings do all the work: the spatial sigma σs in pixels and the range sigma σr in gray levels on the 8-bit scale. Applying the filter several times in a row compounds the smoothing, so pass count is a strength control, not a separate algorithm.

Why nine samples and a big range sigma flatten the face

A three-by-three kernel samples nine pixels, reaching one pixel from the centre. A freckle three pixels wide puts clean skin about two pixels away, outside the window, so at a sensible σr the kernel averages freckle with freckle and nothing changes. The tempting fix is to raise σr until the spot fades. That removes the brightness test for every pixel in the frame, and every edge starts averaging.

The weights show it. Take a blemish edge of ΔI = 12 gray levels:

• σr = 4: exp(−144 / 32) = exp(−4.5) = 0.011 — the edge blocks the average and survives.
• σr = 20: exp(−144 / 800) = exp(−0.18) = 0.84 — the edge contributes almost normally and dissolves.

Eyelashes and nostril lines are far stronger than 12 levels, so they go first. The image reads as soft and low resolution because the high-frequency band has been deleted, not because anything was scaled up.

The brightness lift in the same chain does not help: a spot twelve levels darker is still twelve levels darker on a brighter base, because contrast, not absolute level, is what reads as a blemish.

Setting the two sigmas from measurements

Measure three values on representative footage before touching sliders:

1. Sensor noise σn: shoot twenty frames of a flat, evenly lit patch and compute the per-pixel standard deviation across frames. In the running example σn = 2.5 levels.
2. Blemish contrast ΔIb: spot-pixel level minus the median of the surrounding skin. Example: 12 levels.
3. Blemish radius: half the width in pixels. A three-pixel freckle has a radius of 1.5; round σs up to 2 px.

Then set σr = ΔIb / 2 and at least 1.5 × σn, and support of at least ±2σs on each axis. In the example: σr = 12 / 2 = 6 levels, checked against 1.5 × 2.5 = 3.75, so 6 stands; σs = 2 px; window −4 to +4 px, a 9×9 kernel of 81 samples. If the framework exposes the range sigma directly — one widely used GPU imaging library calls the property distanceNormalizationFactor — that single number is σr.

Check both behaviours at those numbers. Noise, ΔI = 2.5: exp(−6.25 / 72) = exp(−0.087) = 0.92, so noise still averages and the patch cleans up. Blemish edge, ΔI = 12: exp(−144 / 72) = exp(−2) = 0.135, so the edge mostly blocks the average and the spot fades without dragging the skin with it. Nine samples fits σs only up to about 1 px, which handles blemishes of two pixels or less; anything larger needs the wider window or a fast approximation.

Frequency separation: remove blotches, keep pores

Even with correct sigmas, a bilateral pass softens fine texture you wanted to keep. Frequency separation splits the frame so the filter only ever touches the low band:

σf separates blotch scale from texture scale. Start at σf = 6 px for 1080p and scale it with resolution: a 4K frame of the same framing has twice the linear pixel count, so σf = 12 px there. Blotches wider than about 2σf land in Ilow; pores and fine wrinkles live in Ihigh.

β is the texture dial. β = 1 re-adds every bit of detail: only the low band changed, so blotches fade while pores stay. β = 0.5 halves the contrast of the high band: an eight-level fine wrinkle reads at 8 × 0.5 = 4 levels. Values below 0.5 start to look processed.

m confines everything to skin. Build it in two stages: a face box from any face detector, then inside that box a chroma gate computed from the face's own statistics — take the median and the median absolute deviation of the Cb and Cr channels and accept pixels within ±3 MAD. Fixed color ranges accept beige walls and wooden desks; the face's own statistics do not.

Redness is the last piece, applied inside the mask on the Cr channel: shift each pixel toward the face median by a factor of 0.6-0.8. With a median of 145 and a spot pixel at 160: 145 + 0.7 × (160 − 145) = 155.5, rounded to 156. The spot keeps its spatial detail but loses most of its color pop.

Staying inside the frame budget

At 1920 × 1080 and 30 fps the pipeline must finish in 33.3 ms per frame, and the pixel rate is 2,073,600 × 30 = 62.2 Mpx/s. The kernel sizes above price out as follows: nine samples cost 62.2 Mpx/s × 9 = 560 M texture fetches/s, and the 81-sample kernel costs 62.2 Mpx/s × 81 = 5.0 G fetches/s. At 60 fps the budget halves to 16.7 ms and both figures double.

Three levers bring it back. Compute the mask on a quarter-resolution frame — 480 × 270 = 129,600 px, sixteen times less mask work — and upsample the result. Replace the large bilateral kernel with a guided filter: it approximates edge-preserving smoothing with box filters whose cost per pixel is independent of σs, which is why it is the standard video substitute. Keep every intermediate in GPU memory end to end; one CPU round trip per stage is usually what tips a working pipeline over budget, and the GPU tensor feeding guide shows the pattern. When frames still drop, time each pass separately with the low frame rate diagnosis approach before changing the algorithm. Where the scene is steady, averaging a few frames before the filter cuts noise at zero shader cost — the cross-frame noise filtering method applies the same idea to edges.

The same numbers in machine vision

Swap the goal and the same arithmetic serves inspection: average away sensor noise while the defect edge blocks the filter, exactly as the blemish edge did. The rule flips sign — set σr at half the defect-edge contrast, so a 30-level scratch edge with σr = 15 keeps a weight of exp(−900 / 450) = exp(−2) = 0.135 and survives denoising. One line matters more here than in retouching: never smooth the feature you measure. Edge-based matching lives on those gradients, as the edge-based pattern matching guide shows for unstable lighting.

For on-device deployment, this pipeline runs on smart cameras — inspection on the device, no PC. The LS-SMC-0017 pairs an 8.8 MP global-shutter sensor with a C-mount on an X86 platform, so a full custom pipeline runs at the camera. The LS-SMC-0063 is rated at 60 fps at 1.6 MP with a built-in 16 mm lens. Both fit the frame-budget check from the previous section, and the camera finder filters the range by resolution and interface.

When rule-based smoothing is not enough

Bilateral filtering plus frequency separation covers most retouching targets, but it cannot move attributes that are not band-limited — face shape, permanent shadows, color casts from mixed lighting. The learned route trains a network on before-and-after retouched pairs and reproduces the whole mapping in one pass; the retouching model training guide covers dataset construction. Keep the mask and the band split from this guide even then: a network that sees the full frame invents detail where it should not.

Acceptance checks before you ship

Run these five checks on recorded footage, not on the live feed:

• Texture retention: compute the standard deviation of the high band inside the mask before and after. At β = 0.5 the ratio must land at 0.45-0.55; anything below 0.4 means a stage is smoothing the detail band too.
• Edge safety: outside the mask the high-band ratio must be 1.00 ± 0.02. Test with eyelashes, glasses frames and beard edges in frame.
• Blemish fading: inject a 12-level, three-pixel spot on flat skin; after the pipeline it must read at 4 levels or less.
• Noise: the flat-patch standard deviation must drop from 2.5 to 2.0 levels or below.
• Timing: total pipeline latency at or under 33.3 ms at 1080p30 and 16.7 ms at 60 fps, measured per pass.

Fail one and the cause is almost always a row in the table under the second section. Everything above is settings and blending order, not a new filter. If the destination is an inspection station rather than a retouching pass, the configurator walks camera choice by resolution and interface:

The applications guides hub collects the neighbouring techniques, or contact us with your footage parameters and frame budget for a parts check.