System designHow-to

Night vision object detection training: adapt your data first

Fine-tune night vision object detection on infrared frames from your own scene: capture plan, splits, preprocessing, training settings and verification.

LumaSpec Vision11 min read Share on LinkedIn

On this page 11 sections
  1. 01Why night metrics collapse: a domain gap, not a tuning bug
  2. 02Confirm the gap before you retrain
  3. 03Capture a fine-tuning set from the deployment scene
    1. IR-illuminated scenes: mono silicon or SWIR
    2. Lights-off with no illuminator: thermal
  4. 04Size the object in pixels before you trust the camera
  5. 05Label and split so the metric stays honest
  6. 06Preprocess identically at training and at inference
  7. 07Fine-tune settings that move the metric
  8. 08Verify on a night the model has never seen
  9. 09Common mistakes that cost a week
  10. 10Before you retrain: the short checklist
  11. 11Questions

A detector that scores well on daylight footage often drops close to zero when the lights go off and the IR illuminator takes over. The cause is usually not the training code. Infrared frames are a different domain: monochrome, noisier, with a bright hotspot around the illuminator and none of the colour cues your classes relied on. This guide shows you how to close that gap: diagnose it in half an hour, capture a small fine-tuning set from the actual scene, split and preprocess it so the metric stays honest, fine-tune the existing model, and verify the gain on a night the model has never seen. The hardware side matters too, so the capture section names cameras that fit both IR-illuminated and fully lights-off scenes.

Why night metrics collapse: a domain gap, not a tuning bug

Daylight training frames carry colour, even illumination and low sensor noise. Your night frames carry none of that: the sensor runs at higher gain, the illuminator paints a hotspot at the centre of the frame, glass and eyes reflect it back, and every object is described by shape and texture alone. The features the network learned from colour gradients simply do not fire.

The thermal person-detection literature states the two working directions plainly: preprocessing shifts the image toward the daylight domain, and fine-tuning shifts the detector toward the infrared domain. Both help. Fine-tuning on real IR frames is the one that holds up in production, because no filter can invent colour the sensor never captured.

Brightness and contrast augmentation changes global statistics only. It cannot synthesise the hotspot gradient, the gain-noise spectrum or the missing colour channel, which is why augmentation-only attempts leave the metric flat.

Confirm the gap before you retrain

Run the daylight model on 30 or so IR frames from the deployment camera before touching the training set. The failure pattern tells you which fix applies:

One more check before training: confirm the frames are true infrared captures rather than a software night mode. Many integrated cameras fake night vision with gain and a colour tint, and the technical guide on selecting true IR night vision cameras walks through the checks that separate the two.

Capture a fine-tuning set from the deployment scene

  1. Fix the geometry. Camera position, lens and illuminator placement must match production exactly; tape the mounts so nothing drifts between sessions.
  2. Match the exposure mode. If deployment runs fixed exposure and gain, capture with those same values. If deployment runs auto, capture in auto too — training frames must look like deployment frames, not like the best possible frames.
  3. Capture across at least three sessions. Spread them over different nights so ambient light, temperature and illuminator warm-up vary, and record the exposure and gain values per session.
  4. Include negative frames. Give 10-20% of the set no target objects at all, so the model learns that hotspot reflections on glass and walls are background. The background images in object detection training article covers why unlabelled negatives belong in the set.
  5. Check the frames before labeling: mean brightness within your deployment range, visible histogram spread, and no accidentally daylight-lit frames.
  6. Budget about 150 labeled frames to start — 100 train, 25 validation, 25 test. The sizing logic in How many images to train an object detection model applies: if validation mAP still climbs after the first fine-tune, add 50 and repeat.

IR-illuminated scenes: mono silicon or SWIR

Most machine-vision illuminators work in the near infrared, around 850-940 nm. A monochrome sensor with no IR-cut filter picks these up directly, which is why a monochrome camera with an optical filter wins on contrast in night scenes. Where your classes differ by material rather than shape — wet versus dry, printed versus unprinted — a short-wave sensor adds contrast silicon cannot see, and the dual bandpass filter approach keeps one camera usable day and night.

The LS-IRC-0029 is a 0.3 MP mono SWIR camera with 15 μm × 15 μm pixels on a 1/1.2" sensor, global shutter, C-mount and GigE.

Infrared camerasShort-wave infrared area scan camera, 0.3 MP, GigE, Mono, 15μm x 15μm pixels (LS-IRC-0029)LS-IRC-0029
Sensor format
1/1.2"
Lens mount
C-mount
Interface
GigE
Resolution
Up to 3 MP
$5,999.00 · quotedView and quote

Lights-off with no illuminator: thermal

A thermal camera needs no light at all: an LWIR sensor images the heat objects emit, so lights-off operation comes with no hotspot and no illuminator to mount. The trade is lower resolution and image statistics that look nothing like daylight; camera sensor technologies from CMOS to microbolometer explains why.

The LS-IRC-0044 is a 640 × 512 LWIR thermal camera, 8-14 μm, with an 88.5° × 73.2° field of view, streaming 50 Hz over GigE. Where you want SWIR material contrast with more pixels, the MV-CI013-GS-NN is a 1.3 MP SWIR camera on a 1/2" sensor, fanless, C-mount, GigE.

Infrared cameras640 x 512 thermal camera, 8-14 μm, 50 Hz, GigE, FOV 88.5° × 73.2°, 9-24 V DC (LS-IRC-0044)LS-IRC-0044
Lens mount
C-mount
Interface
GigE
$5,885.00 · quotedView and quote
Infrared camerasHikrobot MV-CI013-GS-NN 1.3 MP SWIR Camera, IMX990, GigE, fanlessMV-CI013-GS-NN
Sensor format
1/2"
Lens mount
C-mount
Price on requestView and quote

Both families sit in the shop ranges: the infrared cameras category covers thermal and SWIR models, and the area scan cameras category covers mono silicon ones. The camera finder filters both by spectrum, resolution and interface.

CamerasInfrared cameras

Thermal imaging cameras

58 models · from $995.00

Size the object in pixels before you trust the camera

For a camera with a horizontal field of view FOV_h and N_h pixels across, at working distance d:

image width (m) = 2 × d × tan(FOV_h / 2) mm per pixel = 1000 × image width / N_h object width in pixels = object width (mm) / (mm per pixel)

Worked example with the thermal camera from the capture section. The LS-IRC-0044 has a 640 × 512 sensor and an 88.5° × 73.2° field of view.

At d = 3 m, the horizontal image width is 2 × 3 × tan(44.25°) = 5.84 m, so one pixel spans 5845 / 640 = 9.1 mm. A person 0.45 m wide therefore spans 450 / 9.1 ≈ 49 pixels. Vertically, 73.2° across 512 pixels at the same distance gives 2 × 3 × tan(36.6°) = 4.46 m, or 8.7 mm per pixel, so a 1.7 m person is about 1700 / 8.7 ≈ 195 pixels tall.

That matters because the first detection head in a YOLO-family network downsamples the image with a stride of 8: objects below roughly 10 pixels never fill a full grid cell, leaving the detector almost nothing to anchor on. At 49 × 195 pixels the person has comfortable margin, and halving the distance doubles both numbers. If your objects are smaller, run the check the other way — from object size to the camera pixels you need — with the camera resolution calculator.

Camera resolution for the smallest feature
Units
Pixels needed
–
Sensor size
– MP
Smallest sensor that fits
–
Pixel on the part
– mm
Pixels across the feature, that sensor
–

Pixels needed = field / feature × pixels per feature on each axis (field incl. margin). The lens must resolve the pixels too: Nyquist = 1000 / (2 × pixel µm) lp/mm.

Find cameras with enough pixels Open the full calculator

Label and split so the metric stays honest

Label on the frames exactly as the deployment pipeline will deliver them — same bit depth, same channel count, no pre-enhanced copies. If you label equalized frames and deploy raw ones, the gap returns on the first night.

Split by session, not by frame. A random per-frame split puts near-duplicate frames from the same night into both training and validation, and validation mAP inflates without telling you anything.

A concrete split for the 150-frame set: sessions one and two give the 100 training frames; session three gives 25 validation frames for early stopping and 25 test frames you touch once, at the end. Keep the 10-20% negative share in every split.

Preprocess identically at training and at inference

Three preprocessing decisions dominate night detection, and all three must run the same way in the training pipeline and the deployment process:

Channel count. Pre-trained backbones expect three channels, so replicate the mono IR frame into three. The grayscale image, 3-channel model input article shows the input shapes and the pitfalls.

Bit depth. Thermal streams arrive as 16-bit values while most trainers expect 8-bit. Map 16 to 8 bits once — clip at the 1st and 99th percentile of the training set, scale linearly to 0-255, hard-code those bounds — and never let a viewer's auto-contrast decide the mapping. The high bitdepth grayscale export article covers the export side.

Contrast. Global histogram equalization stretches the intensity histogram across the full range — the histogram equalization reference has the math — but it amplifies gain noise in dark frames. CLAHE limits the stretch per tile; OpenCV's defaults (clip limit 2.0, 8 × 8 tiles) are a reasonable start, and the two-method comparison in grayscale quantization for low-contrast images helps you choose. Run the same transform at inference, or do not run it at all.

Augmentation comes last: brightness (the V in HSV) jitter, contrast jitter and Gaussian noise add tolerance at the margins. They supplement the real IR frames; they never replace them.

Fine-tune settings that move the metric

  1. Start from your daylight weights or the standard COCO pretrain. Most backbone features — edges, textures, body shapes — survive the domain change.
  2. Drop the learning rate about tenfold relative to from-scratch training: if you trained at 0.01, fine-tune near 0.001, so the early steps do not destroy the pretrained features.
  3. Keep the input size at the deployment value. The common default of 640 matches a 640 × 512 sensor almost one-to-one; if objects land below about 20 pixels in the frame, the imgsz selection from your smallest object logic tells you when to raise it.
  4. Freeze the backbone for the first run when the IR set is small; unfreeze and continue if validation mAP stalls below your daylight baseline.
  5. Train 50-100 epochs with early stopping on validation mAP. Beyond that, the same 100 frames overfit — validation, not training loss, is the number to watch.
  6. If daylight accuracy collapses after the fine-tune, mix daylight replay back in — about one daylight frame per three IR frames as a starting mix — or run two models, one per shift.

Verify on a night the model has never seen

Evaluate once on the held-out session and once live, lights off, with objects on the routes they actually take. Report IR mAP at IoU 0.5 next to your daylight baseline, so the remaining gap is a number rather than a feeling.

Read the failures before adding data:

  • Precision good, recall low: the model knows the classes but not every pose, distance or occlusion — capture 50 more frames covering the missed cases.
  • Both low and boxes shifted: a training-versus-inference preprocessing mismatch is the usual suspect; compare one frame through each pipeline pixel by pixel.
  • False positives clustered at the hotspot: add negative frames with reflections from new angles.
  • Daylight accuracy fell: catastrophic forgetting — apply the replay mix from the previous section.

The fine-tuned model should recover most of the daylight-to-night gap, and the point where validation mAP stops climbing is the number that ends data collection. The systems topic page collects the rest of the training-side guides.

Common mistakes that cost a week

  • Same-night random split. The most common inflated metric; cut by session instead.
  • Illuminator moved after capture. Any mount change re-paints the hotspot; re-capture 30 frames and fine-tune again rather than assuming transfer.
  • CLAHE or equalization at inference only. The training frames were raw and the deployment frames are not.
  • 16-bit thermal frames fed straight into an 8-bit trainer. Values clip or quantize wrong; fix the mapping first.
  • Auto exposure on at capture but locked at deployment, or the reverse. Match the mode, not the ideal image.
  • More epochs instead of more frames. Once validation mAP has plateaued, the set is the constraint; add 50 frames from a fresh session.

Before you retrain: the short checklist

  • Geometry taped and matching production: camera, lens, illuminator.
  • Exposure and gain mode matched between capture and deployment, values recorded per session.
  • Frames from three or more sessions, about 150 labeled, with the 10-20% negative share included.
  • Split cut by session: 100 train, 25 validation, 25 test untouched until the end.
  • Preprocessing chain — channels, bit-depth mapping, contrast — saved as code shared by training and inference.
  • Daylight baseline mAP and night baseline mAP written down before the first fine-tune.
  • One held-out session reserved for the final live test.

If the deployment camera itself is still open, start the camera configurator from sensor and spectrum — mono SWIR for illuminated scenes, LWIR thermal for scenes that must run fully dark — or reach the team through the contact page with your object sizes and working distance.

Questions

How many infrared images do I need before fine-tuning pays off?

Start with about 150 labeled frames from the deployment scene — 100 for training, 25 for validation, 25 for test — captured over three or more sessions. Fine-tune, then watch validation mAP. If it is still climbing, add 50 frames from a fresh session and repeat. The plateau, not the frame count, tells you when to stop collecting.

Do grayscale IR frames need to be converted to RGB for training?

Replicate the single channel into three. Pre-trained backbones were trained on three-channel images, so replication keeps those weights useful. True single-channel input means changing the first convolution and starting closer to scratch, which a 100-frame infrared set cannot support.

Can brightness and contrast augmentation alone adapt a daylight model to IR night frames?

No. Jitter changes global brightness, but the infrared domain differs structurally: monochrome input, gain noise, an illuminator hot spot and its reflections. Augmentation helps as a supplement once real IR frames are in the set; on its own it leaves most of the gap open.

Will a thermal camera detect objects at night with no IR illuminator?

Yes. An LWIR thermal camera images emitted heat, so it needs no light at all and creates no hot spot. The trade is texture and resolution: a typical 640 × 512 device resolves about 9 mm per pixel at 3 m, so run the pixels-on-target check for your object sizes and distance before choosing it.

Should histogram equalization run before training or only at inference?

Decide it before training and keep it fixed. Whatever contrast mapping the model saw in training must be the one the deployment pipeline applies frame by frame. If you label equalized frames and then deploy raw ones, the input distribution changes and the metric drop returns on the first night.