3D visionHow-to

High quality training images from structured light 3D cameras

Set 2D exposure, gain and capture mode on a structured light 3D camera so training image quality matches inference, with worked numbers and a checklist.

LumaSpec Vision11 min read Share on LinkedIn

On this page 13 sections
  1. 01The decision: match inference, vary the world
  2. 02Set exposure from the histogram, not the preview
  3. 03Worked example: expose to the fringes, not the median
  4. 04Gain: keep it low, treat changes like exposure changes
  5. 05One capture mode for training and inference
  6. 06Vary parts, poses and finishes — not camera parameters
  7. 07Keep the scene still during each capture
  8. 08Before the first frame: logging and streams
  9. 09Capture procedure
  10. 10Cameras for this workflow
  11. 11When the usual advice is wrong
  12. 12Checks before you collect
  13. 13Questions

Training images that do not look like the frames your camera produces on the line waste every hour you spent labeling them. On a structured light 3D camera the problem has a specific shape: the 2D texture you train on is your exposure and gain settings laid over a projected fringe or speckle pattern, so image quality is a set of acquisition parameters you can fix and reproduce, not a matter of taste. This guide gives you the settings to lock, a worked exposure calculation with the arithmetic shown, the variation to build into parts and poses, and the checks to run before you collect a full dataset. It is for anyone about to shoot a defect or classification dataset from a 3D camera who wants to shoot it once.

The decision: match inference, vary the world

Lock five things and never change them between training and prediction: the camera, the optics, the working distance, the capture mode, and the exposure and gain values. A model trained on 2D texture learns features at a particular scale and brightness distribution; change any of the five and every feature shifts underneath it. Everything else — part batch, surface finish, tilt within fixture tolerance, the on/off state of the plant lights — is where variation belongs.

For quality, work to these numbers on an 8-bit texture image: brightest fringe peaks at or below 250 DN, median in the 100-170 DN band, and no clipped pixels anywhere you will label. A feature you need detected should span at least 3-5 pixels; below about 3 pixels most detectors cannot fire reliably. Start from the camera's 2D stream defaults for exposure, iris and gain, then tune exposure with the method in the next section.

Set exposure from the histogram, not the preview

The live preview rescales and lies; the histogram does not. Capture a representative part at fixed gain and read two numbers from the histogram: the median, and the value of the brightest fringe or speckle peaks. Because sensor response is linear in exposure time while no pixel saturates, you can scale both numbers to a new exposure:

t_new = t_old × DN_target / DN_measured

Apply it twice: once for the exposure the median wants, once for the exposure the peaks will tolerate, and take the smaller of the two. Then capture again and confirm on the new histogram — ambient light, a fresh shiny label or a different finish can move the peaks more than the arithmetic suggests.

Worked example: expose to the fringes, not the median

Say a frame of a grey housing, captured at t = 2.0 ms and the lowest gain, shows a histogram median of 80 DN and fringe peaks at 200 DN.

Chasing the median: you want 160 DN, so t_new = 2.0 ms × 160 / 80 = 4.0 ms. The peaks would rise to 200 × 4.0 / 2.0 = 400 DN and clip hard at 255. Rejected.

Capping at the peaks: t_max = 2.0 ms × 250 / 200 = 2.5 ms. At 2.5 ms the peaks land at 200 × 2.5 / 2.0 = 250 DN, leaving 5 DN of headroom, and the median lands at 80 × 2.5 / 2.0 = 100 DN.

Decision: run 2.5 ms, fixed, and accept the 100 DN median. Normalization removes the absolute level from a model's input; what it cannot remove is detail lost to saturation, and 100 DN sits far above the noise floor of a shaded station. If you want brighter mid-tones, cut ambient light or move the viewing angle off the specular streak instead of adding exposure. For the wider balance between the three controls, see exposure, gain and image quality settings, and why manual exposure beats auto plus gain when a preview looks darker after you switch auto off.

Gain: keep it low, treat changes like exposure changes

Leave gain at its lowest setting unless the histogram proves exposure alone cannot bring the peaks near 250 DN. Gain multiplies signal and noise together, so it lifts the noise floor and pushes peaks into clipping sooner than exposure does. If you raise it, re-shoot the histogram and apply the same peak cap — and if the deployed system will run that gain, every training frame must run it too.

The infrared texture needs its own histogram pass, because a sensor's response to the projected IR wavelength differs from its response to visible light; how CMOS cameras capture images in low light with IR explains the sensor-side behavior. The same manual IR gain tuning used for near-range depth applies to the IR texture you will train on.

One capture mode for training and inference

Structured-light cameras offer capture modes tuned for different part finishes — longer or stronger pattern sequences for dark or shiny surfaces, faster sets for matte ones. The mode changes the projected pattern set and therefore changes the texture the camera returns. A dataset shot in one mode with a deployment running another is a silent mismatch: the model trains on fringe textures that inference never produces. Pick the mode that survives your worst realistic surface, lock it, and if you ever switch, treat the data as a new dataset version. The per-surface trade-offs are set out in capture modes for reflective versus matte parts.

Vary parts, poses and finishes — not camera parameters

Diversity comes from the world, in this order:

  • Poses: every tilt and rotation the fixture allows. If the fixture tolerates ±10° of tilt, the dataset must contain that range, not just the nominal pose.
  • Finishes: matte, glossy, oiled and anodized versions of the same part. If a finish appears at a later process step, source early samples and shoot them now.
  • Ambient: shoot with the plant lights in the state production will run. If lighting varies across shifts, capture that variation rather than engineering it away only on the bench; the ambient light control methods from pick-and-place vision transfer directly to a dataset bench.
  • Negatives: empty fixture, a neighbouring part intruding into frame, parts at the edge of tolerance.

How many frames per class is a separate question — how many training images per class deep learning needs covers the numbers — but the rule at capture time is short: one new variation beats ten copies of an easy frame. Organize the tagged image library from the first session, because a dataset you cannot search is a dataset you will re-shoot.

Before collecting, check your smallest real feature against the sensor:

px_per_mm = image_width_px / FOV_mm defect_px = defect_mm × px_per_mm

Say the camera returns 1920 px across a 300 mm field of view: 1920 / 300 = 6.4 px/mm. A 0.5 mm scratch spans 0.5 × 6.4 = 3.2 px, which is labelable but marginal; a 1.0 mm defect spans 1.0 × 6.4 = 6.4 px and trains comfortably. If the worst real defect computes below 3 px, move the camera closer or step up sensor resolution before collecting — relabeling cannot add pixels that were never captured.

Keep the scene still during each capture

Structured light builds depth from several projected frames in quick succession, and motion during that sequence smears the fringes. The trap: the 2D texture can look sharp while the depth is already wrong, or the reverse. Fixture the part or gate capture on a stillness signal, and check a known dimension in the returned point cloud for each new part style. The mounting side of this is covered in why stillness during capture decides image quality.

Before the first frame: logging and streams

Store a sidecar record beside every image: exposure, gain, capture mode, working distance, light state and firmware version. Six fields turn a mysterious accuracy drop three months from now into a five-minute diff. Save the camera's native streams losslessly — color texture plus depth or infrared — and keep the deployment preprocessing in a versioned script; deciding which streams your pipeline consumes is easier with both saved than with a re-shoot.

Capture procedure

  1. Fix camera, optics, working distance and capture mode to the deployment plan, and write them down as the frozen set.
  2. Set gain to its lowest value.
  3. Place a representative part and capture one frame at the default exposure.
  4. Read the median and the fringe-peak values from the histogram.
  5. Compute the peak-capped exposure: t_max = t_old × 250 / DN_peak. Set it.
  6. Bring ambient to the production state, re-shoot the histogram, and confirm the peaks still sit at or below 250 DN.
  7. Shoot 20-30 frames across the pose, finish and ambient variation list.
  8. Check labeled features span 3 px or more and show no motion smear; check a known dimension in the point cloud.
  9. Capture one part five times and compare the histograms: medians within ±5 DN mean the set-up is stable enough to freeze.
  10. Freeze the parameters, confirm the sidecar records are writing, and only then collect the full dataset.

Cameras for this workflow

If you are still choosing hardware, match the depth technology to the part first; all three technologies below feed training data, but they differ in what the 2D stream contains:

Specification MV-DPS200P-02 MV-DLS300P-04 MV-DT01WMU
Near field of view 180 mm × 160 mm @ 250 mm 280 mm × 220 mm @ 400 mm –
Far field of view 420 mm × 320 mm @ 500 mm 620 mm × 440 mm @ 800 mm –
Clearance distance (CD) 250 mm 400 mm –
Measurement range (MR) 250 mm 400 mm –
Repeatability (Z-axis) 0.02 mm @ 300 mm 0.03 mm @ 600 mm ±(10+0.5%depth1)mm
VDI/VDE accuracy 0.03 mm @ 300 mm 0.05 mm @ 600 mm –
Resolution 3200 × 1944 @ RGB image; 1632 × 1264 @ depth map 3200 × 1944 @ RGB image; 1632 × 1080 @ depth map Depth map: 640 × 480 @ 30 fps; RGB image: 1280 × 1024 @ 30 fps
Data type Raw image (mono + color), Depth map, RGB-D image Raw image (mono + color), Depth map, RGB-D image Intensity image, Color image, Depth map
Acquisition time 0.2 ~ 0.7 s 0.4~0.9 s –
Image output latency 0.5 ~ 1.2 s 0.8~1.5 s 133 ms
Dimension 126 mm × 90 mm × 50 mm 236.3 mm × 79.8 mm × 98.5 mm 91 mm × 26 mm × 31.2 mm
Weight 0.74 kg Approx. 1.3 kg Approx. 70 g
Price $8,829.00 · quoted $16,579.00 · quoted Price on request

Values as printed in each product's data sheet; open a part for its drawing and full specifications.

A projected structured-light camera of this kind returns the aligned 2D texture and depth together, so the training stream and the deployment stream stay identical by construction:

3D sensorsHikrobot MV-DPS200P-02 Projected structured-light 3D cameraMV-DPS200P-02
Type
Stereo / structured light
$8,829.00 · quotedView and quote

For large parts scanned line by line, a laser galvanometer stereo camera of this kind gives dense depth over a larger volume:

3D sensorsHikrobot MV-DLS300P-04 Laser galvanometer stereo cameraMV-DLS300P-04
Type
Stereo / structured light
$16,579.00 · quotedView and quote

Where depth comes from time of flight rather than a projected pattern, the same set-up freeze applies to the IR stream:

3D sensorsHikrobot MV-DT01WMU ToF sensing cameraMV-DT01WMU
Type
Depth / ToF camera
Price on requestView and quote

Browse the full range in the 3D sensors category, filter by technology in the camera finder, or pair camera, mount and cabling in the system configurator.

When the usual advice is wrong

Deployment will run auto exposure. Fixed-value training is then the wrong plan — but so is capturing one auto-exposure benchmark. Read the range the auto exposure actually visits on your parts over a full shift, shoot the dataset across that same range, and label accordingly. Better still, fix exposure at deployment and remove the problem.

The model consumes depth, not texture. If the network classifies height maps or point clouds, the histogram rule does not apply; your quality targets are depth coverage, noise and trueness instead. The freeze-everything discipline still holds.

Parts move through the field of view. Blur is speed times exposure:

blur_mm = speed_mm_s × exposure_s

At a 100 mm/s belt and 5 ms exposure, blur = 100 × 0.005 = 0.5 mm — enough to erase a 0.5 mm defect completely. Use a stop gate so structured-light capture always sees a still part, or if the camera offers a single-shot mode, budget blur the same way; why encoder-triggered capture drops images at high RPM covers the trigger-side timing pitfalls that come with moving parts.

Checks before you collect

  • Fringe peaks at or below 250 DN and zero clipped pixels in labeled regions, on every finish in the set.
  • Median inside the 100-170 DN band — or consciously lower, as in the worked example.
  • Capture mode, exposure, gain and working distance written down and matching the deployment plan.
  • Smallest labeled feature at least 3 px; no motion smear in any frame.
  • Point-cloud dimension check passes on the largest and the smallest part.
  • Five repeat captures of one part land within ±5 DN of each other in the median.
  • Ambient in the production state, including the variation across shifts.
  • Sidecar metadata present and correct on every frame.

Run the list, freeze the five items, and the dataset you shoot this week is the dataset your model will see next quarter. If you want a second opinion on the capture set-up before committing the collection time, contact us — expert support is free with every order — and the 3D topic page collects the rest of the depth-camera fixes.

Questions

Can I shoot training images with auto exposure enabled?

Only if the deployed system will also run auto exposure. Auto exposure re-aims brightness for every part, so grayscale statistics move from frame to frame and your model learns to tolerate a drift it will then face anyway. If inference runs fixed exposure — the usual choice for structured light — shoot training frames at that same fixed value from the first session onward.

What file format should I save captured training frames in?

Save a lossless format such as PNG or the camera's raw output for anything you will label: fringe and speckle patterns carry strong regular texture that lossy compression softens, frame to frame differently. Keep native bit depth too — 8-bit texture, 16-bit depth or infrared — rather than converting early. Export the deployment pipeline's exact preprocessing as a versioned script so the training set rebuilds identically.

How dark can a training image be before it hurts the model?

Darker than a mid-gray median is fine while the features you label stay clear of clipping and of the noise floor. A median near 100 DN on an 8-bit scale with fringe peaks at 250 is a healthy training frame: normalization removes the absolute level, and no detail is saturated or crushed. What hurts is clipping or a noise floor swallowing small features, not darkness itself.

Should I train on the projected fringe images or the color texture image?

Train on the stream your deployed pipeline will consume, and decide that before collecting. Most structured-light setups feed the color 2D texture to the detector and use depth separately; a few classify defects directly in the infrared fringe images. If the choice is still open, capture both streams in the first session, label against your deployment decision, and keep the other stream as free extra training data.