Sensor resolution vs video resolution scaling in cameras

A small video frame should show the same scene as a large one, just with coarser pixels. Too often it does not: the camera fills the requested size by cropping, and the field of view quietly shrinks. The cause is usually a pipeline that sets a video size without setting the sensor size behind it. This guide separates the three sizes in the chain — sensor output, ISP output and video output — gives the arithmetic that tells you which one you are actually getting, and shows the two-stage setting that keeps the whole scene on embedded ISP cameras and on standard area scan cameras.

What a resolution mode actually is

A resolution mode is a validated pairing of a sensor readout size, a processing path and an output size. Firmware authors validate each mode against ISP clocks, memory buffers and the demosaic pipeline, then publish the list. The list is short. A 1.0 MP-class sensor may advertise a native 1280 × 800 mode and one or two smaller ones, with nothing in between; any other size you request has to be produced from a listed mode by scaling, cropping, or both.

Two habits follow. First, enumerate what the camera advertises before you build the pipeline around it, rather than assuming a size exists because the arithmetic allows it — the same trap catches people on frame rate modes that are not advertised. Second, read the label for what it is: 720p or 800p names the output size, not the method, and the same label can mean scale on one camera and crop on another. The USB camera locked at 4K article is the sibling case, where a high-resolution mode blocks the whole pipeline until the sizes agree.

Sensor output, ISP output and video output

Three sizes sit in the chain, and mixing them up is what produces surprise crops:

• Sensor output — the pixel rectangle read from the array. This is the native size, 1280 × 800 in the running example, and it sets the maximum scene the camera can ever capture. The inch designators on data sheets, such as 1/1.8 or 1/2.9, name active-area classes rather than exact dimensions, as the image sensor format reference explains.
• ISP output — what the onboard processor returns after demosaicing and gain, and, on cameras that undistort wide lenses in hardware, after undistortion. The corrected rectangle is smaller than the raw capture; the trimmed border is the undistortion margin.
• Video output — the stream your code receives. The ISP reaches it either by scaling the whole frame (same scene, coarser pixels) or by cropping a centred region (same pixel scale, less scene).

Most SDKs expose two settings for this: one for the sensor size, one for the video size. Set only the video size and the driver picks a sensor mode for you, then fits your request inside it — and fitting inside without an explicit scale step usually means cropping. The failure runs the other way too: ask the video stage for more pixels than the sensor mode feeding it, and the pipeline refuses to start, because a video output cannot invent pixels.

The arithmetic of a cropped frame

Scaling and cropping deliver the same pixel count with different scenes, and one fraction describes the difference:

With s = 0.5, a cropped stream keeps half the linear field and one quarter of the scene area. On a flat target perpendicular to the optical axis, the kept width is exactly s times the full width. The angle form matters for wide lenses: a 74° horizontal field cropped at s = 0.5 keeps 2 · atan(0.5 · tan 37°) = 2 · atan(0.377) ≈ 41.3°.

Undistortion compounds the loss on ISP cameras. The margin is already gone from any corrected output, and a crop on top of it removes scene the raw sensor genuinely saw. Where that margin goes, and how the scaling parameter trades crop against edge distortion, is the subject of the undistortion scaling guide; its companion on black borders after undistortion covers the diagnostic side.

Worked example: 1280 × 800 sensor streamed at 640 × 400

Crop path — video size set alone, camera crops the undistorted frame:

• s = 640 ÷ 1280 = 0.5; scene area kept = 0.5² = 0.25
• Kept field angle: 2 · atan(0.5 · tan 37°) ≈ 41.3° of the original 74°
• Kept target width: 0.5 × 300 mm = 150 mm
• Pixel scale: 300 mm ÷ 1280 px = 0.234 mm/px — cropping does not change it
• Bytes: 640 × 400 × 60 = 15.36 MB/s ≈ 123 Mbit/s

Scale path — sensor native, video output scaled:

• Scene: all 300 mm and all 74°
• Pixel scale: 300 mm ÷ 640 px = 0.469 mm/px, twice as coarse; a 1 mm feature spans 1 ÷ 0.469 ≈ 2.1 px instead of 4.3 px
• Bytes: the same 15.36 MB/s ≈ 123 Mbit/s; the native stream costs 1280 × 800 × 60 = 61.44 MB/s ≈ 492 Mbit/s, four times more

Both small frames cost identical bytes, so bandwidth cannot choose between them — the scene does. The crop shows 150 mm of scene at 0.234 mm/px; the scale shows all 300 mm at 0.469 mm/px. If the part is 120 mm wide and always centred, the crop wins on detail. The moment the part can sit anywhere in the 300 mm field, the scale path is the only one that still sees it. That is the decision in one line: crop for fixed zones, scale for the whole scene.

Size the output from the task

Work backwards from the smallest feature you must resolve. The working rule in machine vision is three pixels across the smallest feature for reliable detection or measurement; one pixel cannot separate a feature from noise and two is marginal:

Example: a 120 mm field with 0.2 mm cracks needs 3 × (120 ÷ 0.2) = 3 × 600 = 1800 px across. A 640-wide stream fails outright and even 1280 falls short. Run the other direction and a 640-wide stream at three pixels per 0.2 mm feature covers 640 × 0.2 ÷ 3 ≈ 42.7 mm of field — enough only if the optics narrow the scene or the feature is bigger. If the sensor cannot deliver those pixels after scaling, change the lens or the sensor; the small pixel trade-off guide covers that sensor-side choice. The calculator below runs the same arithmetic from the pixels you have:

Binning, decimation and ROI on area scan cameras

Standard area scan cameras on GigE, USB3 or CoaXPress rarely resize to arbitrary sizes. Their sensor-level levers are:

• Binning — sums neighbouring pixels (2 × 2 or n × n) during readout. Both dimensions shrink, the field of view stays whole, and averaging lowers noise a little.
• Decimation (skipping) — drops rows or columns to reach the same output size with the whole field, but dropped pixels can alias periodic patterns such as mesh, textile or fine teeth; the Nyquist limit guide explains when that bites.
• ROI — reads only a sub-rectangle at native pixel scale. It cuts bytes without touching the detail inside the region, which makes it the right tool when the part always sits in a known zone; the ROI cropping guide walks the setup.

Real numbers on such a sensor: the LS-CMR-0166 reads 4000 × 3000 at 30 fps over USB3.

Where a camera offers 2 × 2 binning, that readout becomes 2000 × 1500 and bytes per frame fall by four; where it does not, the same arithmetic applies to a host-side scale or an ROI. The table sums up every path this guide has covered:

What each path costs in bandwidth and frame rate

Bytes per second = pixels × frame rate × bytes per pixel, so halving each dimension cuts bytes to one quarter at the same frame rate — equally true for scaling, binning, cropping and ROI. The link budget turns that into decisions. A GigE link carries roughly 118 MB/s of payload, so the worked example's native 61.44 MB/s stream fits with little to spare, while the 15.36 MB/s stream leaves room for a second camera on the same switch. USB3 carries roughly 400 MB/s and takes the native stream with headroom.

Frame rate has a hard ceiling at the sensor's readout time: a 640 × 400 output of a sensor that reads 1280 × 800 at 60 fps cannot exceed 60 fps. A smaller video output removes link and host bottlenecks sooner, which is why a scaled stream often sustains the sensor's maximum on a shared link where the full stream does not.

How to set it on the camera

Set the two sizes in this order:

1. Sensor size first, at native. In an embedded SDK this is the sensor-size setting (1280 × 800 in the running example); on an area scan camera it is width and height plus the binning or decimation controls. Native keeps the undistortion margin and the full scene available to everything downstream.
2. Video size second. The ISP scales the full frame to the target, 640 × 400, instead of cropping. If the SDK has no video-size setting, add a resize stage after the capture node — an image-manipulation node on the device or a host-side resize; the bytes saved are identical.
3. Undistort at sensor size, scale last. Keep the undistortion margin parameter (often called alpha) at its maximum so the corrected image keeps its border, then scale. Scaling or cropping first throws margin away permanently.

If you prototype in simulation first, the same split governs a simulated camera's pixel scale. On many board-level cameras the sensor, ISP and mode list all sit on the module, so the two settings are the difference between a pipeline that starts and one that crops by surprise. For a build where the ISP lives on the module, start from a 6 MP USB3 board camera like this:

Where the loop runs fast, a 3.1 MP global-shutter board camera reading 120 fps over USB3 keeps the sensor side off the bottleneck list:

When cropping is the right answer

Crop on purpose, never by accident. Three cases favour a cropped or ROI output: the part always sits in a fixed central zone and you want native detail there (ROI); the consumer of the stream needs a fixed frame regardless of scene, such as a display tile or a fixed-size model input (centre crop); mechanics permanently shadow the outer field, so there is nothing left to keep. Everywhere else, scale — the whole scene costs the same bytes, and the scene is the thing you cannot get back after the fact.

Checks before you commit

• List the modes the camera advertises and build on those, not on arithmetic alone.
• Compute s and s² for every planned size, and write down the scene fraction each mode keeps.
• Convert each path to mm per pixel (field ÷ video width) and confirm the smallest feature still spans three pixels.
• Check bytes per second (pixels × fps × bytes per pixel) against the link budget before wiring the pipeline.
• Undistort at sensor size with the margin at maximum; scale last.
• Prove the mode with a ruler at working distance: measure the visible width in each candidate mode, and the crop shows itself immediately.
• After any size change, recalibrate or rescale the intrinsics and rerun a golden-part image.

We keep the sensor and the stream as one decision in the camera finder; the camera configurator walks the pairing, and the cameras topic page collects the camera guides. If a mode list still does not add up, send it to us through the contact page and we will read it with you.