Sensor transpose dropped lines: fix a 90 degree rotated feed

A 90 degree transpose is supposed to be a free orientation change on an embedded board camera: one call, and the feed matches the way the sensor is mounted. Instead the image arrives with a strip of pixels cut from the left edge and re-attached on the right, while the frame counter keeps ticking. That wrap is the signature of the transposed copy path outrunning the board's memory bandwidth, not a defective sensor. This guide is for engineers running microcontroller-class vision firmware on board-level camera modules. It shows how to confirm the diagnosis in ten minutes, cap the frame rate to stop the dropped lines, compose the same orientation from flips that stay on the fast path, and take the copy off the CPU for good.

What the failure looks like

The wrap is the tell. An electrically failing sensor produces black gaps, noise or a frozen stream; a bandwidth-starved transpose produces intact pixels in the wrong slots, because the frame builder packs every block it receives into the next expected line slot. Content that misses its slot lands in a later one, so the missing strip reappears displaced across the frame. Which edge it lands on follows the copy order; on the board behind this guide, pixels cut from the left side turned up attached to the right.

Two checks, one minute each, separate bandwidth from damage:

• Step the frame rate down in 5 fps increments. Wrap that thins and then disappears points at the copy path, and the fixes below apply.
• Freeze the scene on a printed pattern. A static scene that still wraps frame after frame rules out motion.

If the wrap survives a heavy rate cap, suspect the sensor or its link instead, and start from the board camera evaluation checklist for inspection systems.

Why a 90 degree transpose outruns the memory bus

An un-transposed copy walks memory front to back, so every fetched cache line is used in full. A transposed copy writes across the grain: consecutive output pixels sit one full row apart in the input, 320 bytes apart on this feed.

For the reported feed, 320 × 320 pixels at 1 byte per pixel of grayscale:

• frame_bytes = 320 × 320 × 1 B = 102,400 B
• copy_traffic at 45 fps = 2 × 102,400 × 45 = 9,216,000 B/s ≈ 9.2 MB/s of read-plus-write
• t_frame at 45 fps = 1 / 45 = 22.2 ms per frame

The un-transposed path on the same board sustains that traffic because its access is sequential. Column-wise access is not: it consumes one byte from each 32-byte cache line and wants the neighbouring bytes one column later. A 102,400 B image against a 16-32 KB data cache is 3-6× the cache, so a line is evicted before its other 31 bytes are needed, and read traffic grows by at least the image-to-cache ratio — 3× with a 32 KB cache, 6× with 16 KB. Add the cycles the copy loop itself burns and the line callback misses its deadlines; the driver drops the lines it cannot place, and those displaced lines are your wrap.

A second margin-killer hides in the clock assumption. The board here runs a 400 MHz core while 480 MHz is the figure people budget against: (480 - 400) / 480 = 17% of the copy budget gone before the first pixel moves.

Fix 1 — cap the frame rate first

Set the cap before transpose engages, so the copy engine never sees a line pace it cannot hold. On the stack behind this failure the calls read:

sensor.set_framesize(sensor.B320X320)   # 320 × 320 square ROI
sensor.set_framerate(15)                # cap first, transpose second
sensor.set_transpose(True)              # 90 degree sensor-side path

The comments are ours; the calls and their order are the stack's. The number comes straight from the arithmetic above: at 15 fps the frame period stretches from 22.2 ms to 1000 / 15 = 66.7 ms, three times the time per frame at unchanged per-frame traffic. The reported board went from wrapped-every-frame to stable at 15 fps, and held a stable 20 fps once the 320 × 320 sensor timing was tuned.

Commit to a set point with this procedure:

1. Run the transposed feed untuned and note the rate where wrap starts: 45 fps here.
2. Cap at a third to a half of that figure: 15 fps, confirmed clean on the reported board.
3. Step up in 5 fps increments until the wrap returns, back off one step, and keep 20% margin. The production band lands at 15-20 fps.

Two follow-ons to budget. Exposure cannot exceed the frame period, so a 15 fps cap sets a 66.7 ms exposure ceiling; why camera exposure time has a maximum limit and how to exceed it covers the mechanics. And if the capped value is not among the modes the sensor advertises, the driver may ignore it — camera frame rate mode missing: verify advertised modes shows how to enumerate what is really offered.

Fix 2 — compose the orientation from flips

A transpose reflects the image across its main diagonal, so it swaps handedness; it is not a rotation. Blob detection, counting and occupancy logic rarely care. Text, labels, threads and any asymmetric part read mirrored, and an operator watching the feed sees it at once — decide which result you need before writing code.

The flips are near-free everywhere: they either toggle in the sensor's orientation register or run as row-wise copies the memory system likes. A 180 degree mount therefore never needs the transposed path at all.

For a true 90 degrees, move the transform to after capture:

sensor.set_transpose(False)
img = sensor.snapshot()                                  # native orientation, full rate
img.replace(hmirror=False, vflip=True, transpose=True)   # same orientation, after capture

The transposed copy still happens, but it now competes for host loop time instead of racing the sensor's line clock. A late frame is slow; it is not corrupt. Trading a data-integrity failure for a throughput cost is nearly always the right side of the trade — price the new per-frame cost the way acquisition faster than processing: fix the frame rate gap does.

Fix 3 — take the copy off the CPU path

Rotate the camera instead of the image. If transpose exists only to match a 90 degree mount, the cheapest processor is a bracket: a mechanically rotated module streams natively in the orientation you wanted, and the copy disappears. Check two side effects before re-mounting. The field of view swaps axes, so a 4:3 sensor laid sideways covers a 3:4 scene. A rolling shutter's readout direction also turns with the camera, so horizontal motion smear becomes vertical. For 180 degree mounts, mount upside down and finish with the sensor's free flip register.

Shrink the frame the copy moves. Copy load scales with pixel count. Halving both dimensions quarters it: 160 × 160 × 1 B = 25,600 B, a quarter of 102,400 B, so the same callback holds four times the frame rate. 2 × 2 binning quarters the bytes inside the sensor before the copy ever runs — 4x4 binning vs 2x2: sensor readout decides your frame rate explains how readout modes set the pace.

Update the driver. The permanent fix lives below your code: a line callback that hands both un-transposed and transposed copies to a memory DMA engine (MDMA in this firmware family's release notes) removes the deadline problem instead of managing it. If your board's changelog lists that work, the update beats any cap.

Which fix fits your case

On a host PC the failure largely evaporates. USB3 and GigE board cameras deliver frames into memory with orders of magnitude more bandwidth, and a rotation applied in host memory works on complete frames, so lines cannot drop. What remains is link saturation: stacking a transform on an already-saturated link starves frame rate, which is a budgeting question, not a corruption one. The same arithmetic governs the embedded side — USB hub drops camera frame rate: budget the bandwidth first walks the hub case, and MIPI camera lane bandwidth: what limits frame rate covers the camera-to-processor link.

When several fixes apply, take the zero-per-frame ones first: mechanical rotation and driver updates, then flips and post-capture transforms, and only then a standing rate cap, which costs frame rate for the life of the machine.

Verify before you ship

A wrap that appears once a minute is easy to miss in a two-second check, so verify against pixels and time:

• Point the camera at a printed pattern with numbered rows and columns. A dropped line shows as a displaced strip, and the numbers name the lines the callback lost.
• Hold the scene static and difference consecutive frames; any non-zero row marks a drop. Count the mismatches instead of eyeballing them.
• Soak at the production rate for at least ten minutes. Deadline misses cluster under interrupt storms and thermal transients, not in the first second.
• Fix the orientation before you calibrate. A rotation permutes the whole intrinsics matrix, so any calibration taken in the native orientation is invalid for the shipped one — recapture the calibration targets only once the mount, cap and flips are final.

Buying headroom instead of managing a copy

When you are choosing the module rather than fighting one, the fix is headroom: a camera whose rate budget survives a host-side transform, and a host with bandwidth to spare. The board-level cameras range splits into USB3 parts for host-side transforms and GigE parts for longer runs; the camera finder filters both by resolution, shutter and mount.

The LS-BCM-0003 streams 120 fps at 3.1 MP over USB3 with a global shutter, so a post-capture rotation still leaves frame budget at inspection rates. The LS-BCM-0020 tops out at 18 fps at 6.3 MP, and the binocular LS-BCM-0043 runs 1280 x 1024 over USB3 with a global shutter. Embedded Linux hosts that prefer GigE are covered by the MV-CB016-10GM-M-W, a 1.6 MP monochrome global-shutter module with an M12 mount.

Match the module's top rate to your loop including the transform, then shortlist with the board level camera selection for embedded vision machines guide.

Still unsure whether your loop survives its orientation handling? Send the frame size, rate and host through the contact page and we will size the copy budget with you, or browse the systems guides for the neighbouring failure modes.

The ten-minute triage list

1. Re-run the feed with transpose off at the same frame rate; a clean image shifts suspicion to the copy path.
2. Enable transpose and step the rate down in 5 fps increments until the wrap disappears; that rate is your failing maximum.
3. Cap at a third to a half of it, confirm a clean stream on a numbered pattern, and soak for ten minutes.
4. Decide chirality: algorithms tolerate the bare transpose; operators and text need the transpose-plus-flip composition or a mechanical rotation.
5. Check the changelog for a memory DMA transposed path before tuning anything else; the update may retire the cap.
6. Recompute copy_traffic = 2 × frame_bytes × fps after any change of frame size, pixel format or binning.