You ask an embedded camera for 648 × 486 and get back a buffer that is wider, taller, or refused outright. The sensor is usually innocent: the multiple-of-32 width and multiple-of-16 height limits live in the buffer descriptor, while the crop rectangle — the pixels you actually process — only has to be a multiple of 2. Underneath both, the sensor runs a short list of fixed binned modes, and the pipeline resamples whichever one it picks onto your request. Separate buffer, crop and sensor mode, and you can size a frame for your CPU budget, keep the field of view your task needs, and add no frame of latency. This explainer is for anyone writing image processing directly against a camera pipeline.
The buffer rule and the crop rule are different rules
A video port on this class of pipeline carries two rectangles. The buffer format demands a width that is a multiple of 32 and a height that is a multiple of 16. The crop rectangle beside it accepts any multiple of 2 per axis. A 648 × 366 picture is therefore legal: allocate 672 × 368 and set the crop to 648 × 366.
Round up, never down: 648 / 32 = 20.25, so the buffer takes 21 × 32 = 672 columns; 366 / 16 = 22.875, so it takes 23 × 16 = 368 rows. Your image sits at the top-left corner, and the pad pixels to the right and below are yours to ignore.
The padded row length is the stride, and it is the number you multiply by the row index — never the crop width. If a delivered buffer is taller than the picture you asked for, you are seeing this padding, and buffer height larger than you set traces where those lines come from. A related surprise, a snapshot that comes back 64 pixels short, is the same layer misbehaving: still image smaller than the ISP frame.
Why the hardware wants aligned rows
Rows start on alignments the DMA engine and the CPU's vector unit prefer; 16 or 32 bytes is the common preference on modern CPUs, which for 8-bit mono means 16 or 32 pixels. Microsoft's image stride documentation states the consequence directly: a stride can be larger than the visible row width times the bytes per pixel, and any transform that consumes the buffer must handle the mismatch. Even alignments also keep Bayer patterns and subsampled chroma periodic across a row, which is why the crop only asks for multiples of 2. The same discipline follows you onto acquisition hardware: buffers a frame grabber delivers carry a stride of their own.
The sensor underneath runs fixed modes
Your request never reaches the sensor verbatim. Take a representative 5 MP sensor with 2592 × 1944 pixels: at a 30 fps request it offers a 1920 × 1080 mode with a cropped field of view, a 1296 × 976 full-view mode that tops out at 42 fps, a 1296 × 730 full-view mode that reaches 49 fps, and full 2592 × 1944 only at 15 fps. The pipeline picks the mode whose ceiling covers your requested rate; requests below about 1280 × 960 land on a binned mode to hold the frame rate.
Binning combines adjacent pixels before readout — a quarter of the pixels, more signal per output pixel, and a faster readout. Skipping, by contrast, discards rows and columns to gain speed, at the cost of aliasing on fine texture. How these modes are exposed varies by camera family; how sensor resolution modes work covers the failure you see when only one of them is offered, and CSI short frames in binning mode shows the size bookkeeping a binned mode imposes on the receiver.
Whatever size you then ask for — 736 × 414, 640 × 360, 648 × 486 — the selected mode is resampled onto it with a filtered scaler. Filtered resampling reads cleaner than nearest-neighbour skipping. The frame rate is part of the same request, and a requested rate of 0 tells the pipeline to choose from an allowed range, which lets auto-exposure lengthen the integration time.
Worked example: 648 × 366 from a 2592 × 1944 sensor
The goal: process the fewest pixels, keep the full field of view, sit near 16:9, run 30 fps, and use hardware binning.
- Quarter resolution is 2592 / 4 = 648 wide and 1944 / 4 = 486 tall. That keeps the sensor's 4:3 shape, which wastes rows on sky and bumper.
- The full-view 16:9 mode is 1296 × 730. Half of it is 648 × 365, and 365 is odd, so the crop becomes 648 × 366.
- The buffer rounds up to 672 × 368.
- Pixels you process: 648 × 366 = 237,168 per frame, which at 30 fps is about 7.1 million pixels per second.
- Memory you allocate: with 8-bit mono the stride is 672 bytes, so one buffer is 672 × 368 = 247,296 bytes; 10,128 bytes of that, 4.1 percent, is padding.
- Encoding: 648 is not a multiple of 16 (648 / 16 = 40.5), so the encoder pads to 656 wide in 16 × 16 macroblocks and marks the pad for removal at decode. The height, 368, already divides by 16.
Sizing from the other direction — from feature size and tolerance to a pixel count — is what the camera resolution calculator does; if your smallest feature needs more than 237,168 pixels can resolve, no amount of buffer engineering will fix that.
The sensible requests compared
| Request | Buffer allocated | Mode used | Field of view | Pixels processed |
|---|---|---|---|---|
| 640 × 360 | 640 × 368 | 1296 × 730 binned | Full | 230,400 |
| 648 × 366 | 672 × 368 | 1296 × 730 binned | Full | 237,168 |
| 640 × 384 | 640 × 384 | 1296 × 976 binned | Full, taller | 245,760 |
| 648 × 486 | 672 × 496 | 1296 × 976 binned | Full, 4:3 | 314,928 |
| 1920 × 1080 | 1920 × 1088 | 1920 × 1080 native | Cropped | 2,073,600 |
The decisions fall out of the table. Choose 640 × 360 for the smallest CPU load: its width lands on the alignment with no padding, and the aspect is 16:9. Choose 648 × 366 for the cleanest rescale — exactly 2:1 from the mode's width, 1.995:1 from its height. Choose 640 × 384 when you want more road than sky and can carry 3.6 percent more pixels. Choose 648 × 486 when the processing needs the full 4:3 frame. Choose 1920 × 1080 only when per-pixel detail matters more than width: the mode crops the field to about 74 percent of the sensor's width and 56 percent of its height, and it carries 8.8 times the pixels of 648 × 366.
What the rescale costs in latency
A scaler stage runs as part of the same pipeline that captures the next frame, so the resample adds a fraction of a frame, not a frame. At 30 fps one frame is 33.3 ms, and the resample of a quarter-megapixel image is a sub-millisecond stage on hardware of this class; your cadence stays one frame per request. The latency that hurts comes from your own queue — when acquisition outruns processing, the fix is on the processing side, as acquisition faster than processing sets out.
When the usual answer is wrong
- The camera does not capture your exact request. Everything between the mode and your crop is resampled. For work that must be pixel-true — metrology, sub-pixel edge fitting — run a mode without rescaling and set binning and ROI so the arithmetic is exact; camera ROI offsets that reset to zero shows the ordering that keeps those settings sticky. What resolution buys you at the pixel level is covered in small pixel camera selection.
- A smaller request is not automatically a better signal. A 648 × 366 frame served from a binned mode averaged four sensor pixels before readout; the same size served by rescaling a full readout did not. When a binned mode exists at the aspect you want, prefer it. Quad Bayer sensors: binning, remosaic and full resolution shows how far a sensor can go, combining pixels then reconstructing colour.
- A non-multiple-of-16 encode size is not illegal, but do not assume every player crops the pad. Encode one clip at 648 wide and check playback for black edges before you ship.
- Mode changes are not free. Switching sensor modes restarts the stream, so decide the mode at start-up and change it deliberately; switching sensor streaming modes safely covers the crash path when you get it wrong.
Checks before you commit
- Read the stride and the buffer height from the delivered frame metadata; address pixel (x, y) as base + y × stride + x × bytes_per_pixel.
- Shoot a grid or ruler and confirm the aspect and field of view you expected — the quickest test that the mode you assumed is the mode in use.
- Log frame timestamps for a few seconds and compare the achieved rate against the mode ceilings; 49 versus 42 versus 15 fps separates the modes quickly.
- Multiply crop width × crop height × fps and compare it with your measured CPU load, so headroom decisions rest on the pixels you process, not the buffer you allocate.
- Encode a clip at your chosen size and confirm the player removes the macroblock padding.
Cameras to build this on
The interesting mode lists sit on small board-level cameras, where the sensor does the binning and your code does the cropping. The LS-BCM-0003 is a 3.1 MP USB3 board-level camera with a 1/1.8" global-shutter sensor at 120 fps. The MV-CB060-10UM-B is a 6 MP USB3 board camera with a 1/1.8" IMX178 sensor, supplied as a bare board. The LS-CMR-0166 is a 12 MP USB3 camera at 30 fps with 4000 × 3000 resolution. One quarter of that output is 1000 × 750, which needs a 1024 × 752 buffer — the same two-rectangle trick, one sensor size up.
- Sensor format
- 1/1.8"
- Lens mount
- C-mount
- Interface
- USB3
- Resolution
- 5 MP
- Sensor format
- 1/1.8"
- Interface
- USB3
- Resolution
- 6-10 MP
- Shutter
- Rolling
- Sensor format
- 1/1.7"
- Interface
- USB3
- Resolution
- 12 MP
- Shutter
- Rolling
Browse board-level cameras for embedded builds, area scan cameras for the full sensor-size range, or filter directly in the camera finder. The configurator walks the same choice step by step, and if the mode list for your part number decides the design, ask us through the contact page.
Questions
Does my requested resolution have to be divisible by 16?
No. Only the buffer dimensions carry the alignment rule: width a multiple of 32, height a multiple of 16. The crop region can be any multiple of 2, so 648 × 366 is a valid picture inside a 672 × 368 buffer. Macroblock encoders such as H.264 do their own padding to multiples of 16 and mark it for removal at decode, so odd encode sizes are legal too.
Will the camera drop columns if my width is not a sensor mode width?
No. The pipeline selects a mode that meets your frame-rate request and resamples the whole mode frame onto your request. Asking for 640 wide from a 1296-wide mode is a 2.025:1 filtered downscale, not a dropped column. You only lose field of view when the mode itself crops — the 1920 × 1080 mode on a 4:3 sensor keeps about 74 percent of the width.
Does the rescale add a frame of latency?
No. The scaler runs as a pipeline stage while the next frame integrates, so your cadence stays one frame per request — 33.3 ms at 30 fps. The resample of a quarter-megapixel frame is a sub-millisecond operation on hardware of this class; the latency that hurts your loop is the queue in front of your own processing, not the scaler.
Why did the buffer come back bigger than the image I asked for?
Padding. The driver rounds the allocation up to the alignment rules — 648 becomes 672 columns, 366 becomes 368 rows — and may append extra lines for the ISP. Read the stride and height from the frame metadata rather than recomputing them from your request, and index rows with the delivered stride.
Is a hardware-binned mode better than a rescaled small request?
For signal-to-noise, yes. Binning combines charge from adjacent pixels before readout, so signal adds while noise adds more slowly; a rescale averages pixels that have already been read out with their noise included. When a binned mode exists at the aspect you want, use it, and use the rescale only for exact sizes the mode list does not offer.
How do I tell which sensor mode actually served my request?
Log frame timestamps for a few seconds and compare the achieved rate with the mode ceilings: on the example sensor, 49 fps points to the 1296 × 730 mode, 42 fps to 1296 × 976, and 15 fps to full resolution. To force a mode, ask for a frame rate only that mode can reach — requests are honored as long as they stay within the sensor's limits.


