GPU lens undistortion mapping: coordinate and sampling checks

A GPU correction that bends straight edges farther outward usually has a mapping problem, not a need for stronger coefficients. Calibration values, output pixels and texture coordinates occupy different coordinate systems. Mixing them can produce a plausible image with incorrect geometry.

For an inspection pipeline, establish one explicit coordinate contract before changing the shader. The checks below separate model direction, normalization, vertical orientation and filtering errors. You will build a destination-to-source map, verify its arithmetic, and then compare image sampling against a trusted reference.

Match the symptom to the first test

Start with the cheapest diagnostic: examine the source coordinates calculated for selected output pixels. An image comparison mixes geometry, interpolation and border handling together.

These are diagnostic priorities, not measured failure rates. For a broader calibration fault tree, use checks when undistortion looks worse after calibration. Here, keep the investigation focused on the shader's coordinate and sampling contract.

Freeze the reference contract

Before changing code, record the exact inputs shared by the reference implementation and GPU path:

1. Source image dimensions, orientation and any active crop.
2. Source focal parameters and principal point, expressed in pixels.
3. Distortion model, coefficient order and mapping direction.
4. Output image dimensions and output camera matrix.
5. Any rectification rotation, interpolation mode and border rule.

Use the same saved frame for both paths. Disable automatic image resizing or display correction that could hide differences. Confirm that the input is genuinely distorted; correcting an already corrected image applies an unwanted second transformation.

Do not assume that matching output dimensions means matching output intrinsics. One implementation may retain the original framing while another selects a different focal parameter or principal point to reduce empty borders.

The guide to interpreting distortion coefficients for an unrectified image explains the model-identification question. Establish that identity before transferring coefficients into a shader uniform array.

Build a destination-to-source map

A fragment shader normally gathers a source sample for each output pixel. It does not move each source pixel to a corrected destination. That distinction explains why the forward distortion equation belongs inside an undistortion sampler.

For the equations below, image coordinates start at the upper-left pixel center, with horizontal coordinates increasing rightward and vertical coordinates increasing downward. Assume zero skew and no rectification rotation.

First convert an ideal output pixel into normalized camera coordinates using the output intrinsics:

The focal parameters here are measured in pixels, not physical lens focal length. The normalized coordinates are dimensionless. Never evaluate the polynomial directly on pixel coordinates or texture coordinates running from zero to one.

For a common radial-plus-tangential model:

These equations convert an ideal ray into its distorted source location. That is the location the output fragment must sample.

Some calibrations instead use a rational radial factor:

Use that denominator only when the calibration specifies this model and coefficient meaning. Array length alone does not identify the model. Reject nonfinite coordinates or an invalid denominator rather than allowing invalid samples to propagate.

If rectification includes a rotation, transform the output ray into the source camera frame and divide by its depth component before applying distortion. A nonidentity rotation cannot simply be omitted while retaining the same reference output.

Check one pixel with visible arithmetic

Use a synthetic case to verify the shader independently of a calibration file. This example uses identical source and output intrinsics, no rotation and radial distortion only.

Assume an image measuring 640 × 480 pixels. Set both focal parameters to 500 pixels and the principal point to coordinates (320, 240). These values are chosen for easy arithmetic, not as a calibration recommendation.

For output pixel (520, 340), with k1 = -0.2 and every other coefficient zero:

The output pixel therefore samples source position (512, 336). Do not negate the coefficient merely because the map moves inward: this is a source lookup, not a displacement applied to the displayed source pixel.

For normalized texture coordinates whose origin matches the image convention:

The half-pixel terms address texture sampling, not lens distortion. If your computed map fails this test, correct the coordinate arithmetic before investigating interpolation.

Resolve vertical orientation at the boundaries

Do not assume that a texture's vertical origin follows the calibration image. Upload code, rendering frameworks and display stages may each apply a flip.

Keep the distortion equations in the calibration's image convention. Convert the output fragment position into that convention before normalization, then convert the calculated source coordinate into the sampler's convention afterward.

For a full-frame viewport starting at zero, suppose fragment coordinates describe pixel centers and increase upward. The conversion to upper-left image coordinates is:

If the source sampler also places its visual bottom at vertical texture coordinate zero, use:

These conversions are conditional, not universal shader requirements. If a framework already provides upper-left image coordinates or flips uploaded textures, adding another flip creates the error again. Subtract any viewport offset before applying the fragment conversion.

Use a test image with distinct markers in all four corners and an off-center cross. A symmetric grid can conceal a vertical reversal.

A center-only test is insufficient because many coordinate mistakes have little effect near the principal point.

Account for pixel centers, resizing and cropping

A persistent half-pixel shift can remain after the distortion geometry is correct. Decide whether each interface represents pixel centers as integer coordinates or as half-integer coordinates.

Under the convention used here, image pixel centers are integers. A normalized sampler addresses source pixel center u_src through (u_src + 0.5) / width. Passing u_src / width shifts the lookup away from that center.

Resizing also changes the intrinsics. For an edge-aligned resize using half-pixel center correspondence, followed by a crop:

Here, crop offsets are measured in the resized image. Other resize conventions require corresponding principal-point formulas; do not mix their coordinate rules.

For a crop without resizing, subtract its left and top offsets from the principal point. Keep the focal parameters unchanged. Sensor binning, camera-side scaling and undocumented acquisition crops require confirmation of their actual pixel correspondence.

Distortion coefficients generally remain unchanged under a coordinate-consistent resize or crop of the same image. The intrinsics change because pixel positions change.

If the remaining difference is only the visible framing, consult black borders after undistortion and output cropping before altering the distortion model.

Match filtering, borders and numerical precision

Once source-coordinate maps agree, compare the sampling stages. Selecting bilinear filtering is necessary for a bilinear reference, but it does not establish complete equivalence.

Check these independently:

• Texture access: Integer pixel-fetch operations do not automatically perform bilinear interpolation. Confirm which sampling operation the shader actually calls.
• Mip level: During validation, sample the original image level rather than an automatically selected lower-resolution level.
• Border handling: Edge clamping repeats boundary pixels. A constant-border reference may instead blend border values with valid neighboring pixels.
• Image representation: Match channel order, numeric range, transfer function and any automatic color conversion.
• Precision: Use adequate coordinate precision, especially for large images and polynomial evaluation near the perimeter.

For bilinear sampling, an out-of-range neighbor may affect the answer even when another contributing neighbor remains inside the image. Replacing every partly outside sample with a constant value will not reproduce all constant-border implementations. Match the reference's per-neighbor behavior if edge equivalence matters.

Small intensity differences can remain because implementations quantize interpolation weights or round output values differently. Establish an application-specific tolerance rather than demanding identical bytes without examining the cause.

If correction also reduces image scale, ordinary bilinear filtering may not sufficiently suppress aliasing. The guide to antialiasing during image warps and downsampling covers that separate sampling problem.

Verify geometry before judging image differences

Use a staged acceptance test so each failure points to a limited set of causes.

1. Identity test: Set distortion to zero and use matching source and output intrinsics, dimensions and orientation. The map should reproduce the input coordinate grid.
2. Radial-only test: Run the worked example and inspect locations at different distances from the principal point.
3. Tangential test: Use a synthetic calibration with a nonzero tangential term. Compare asymmetric positions in each quadrant.
4. Map comparison: Compare floating-point source-coordinate pairs against the reference throughout the valid region.
5. Sampling comparison: With matching maps, compare intensities using identical filtering and border settings.
6. Task validation: Confirm the corrected image still meets the inspection's dimensional or localization requirement.

A diagnostic map-error metric is:

Report both the maximum and a distribution statistic, such as the 95th percentile. A low average can conceal a large perimeter error.

For example, you might investigate map differences above 0.05 pixel during development. That is a diagnostic threshold, not a promised measurement accuracy. Set final limits from the inspection's error budget.

Use barrel distortion correction accuracy checks for independent geometric validation. Once correctness is established, real-time undistortion pipeline planning addresses map reuse and processing throughput.

Check acquisition hardware before replacing parts

A coordinate mismatch is normally a software-contract issue. Replacing a camera or lens will not fix incorrect normalization, sampling direction or vertical orientation.

When planning new acquisition hardware, start with area scan cameras and the camera finder. Confirm that the delivered frame's dimensions, crop and orientation are available to the processing pipeline.

The LS-CMR-0166 is a USB3 area scan camera with a rolling shutter. Treat it as an acquisition candidate, not as a statement about GPU processing compatibility.

For optics, review machine vision lenses through the lens finder. Select coverage, working distance and image quality for the inspection first; shader correction cannot recover detail that was never resolved.

The LS-FFL-0567 is a C-mount machine vision lens. Verify the intended camera's mount and sensor coverage rather than assuming these sourcing examples form a confirmed pair.

Changing focus, lens settings or acquisition geometry can invalidate the existing calibration. Record the accepted configuration and recalibrate when required by its stability and measurement checks.

Close the issue with a reproducible checklist

Before accepting the implementation, retain a saved frame, calibration record and expected coordinate samples alongside the shader version.

• The source image is raw with respect to the correction being applied.
• Source and output intrinsics are identified separately.
• Coefficient order, model direction and optional rotation match the reference.
• Distortion operates on normalized camera coordinates.
• Output fragments gather samples from calculated source positions.
• Image origins, vertical direction and pixel centers are explicit.
• Resize and crop operations are reflected in the intrinsics.
• Filtering, mip selection, borders and image representation are controlled.
• Map agreement and inspection accuracy have separate acceptance checks.

If zero distortion fails the identity test, stay with coordinate handling. If maps agree but images differ, investigate sampling. If both agree but measurements fail, revisit calibration quality and the inspection error budget.

For a new station, start the camera configuration after defining the required image geometry. For an existing station, contact application support with the coordinate contract, calibration model and representative map differences so the next investigation starts with evidence.