V4L2 sensor modes from a kernel module without device tree entries

You are writing a sensor driver for a MIPI CSI-2 host, and the board description has turned into a pile of per-mode entries: one node per resolution, one per frame rate, each repeating numbers the driver could hold for itself. There is a cleaner split. The device tree records what is wired to the board — I2C address, data lanes, supplies, endpoint. The sensor's modes are behaviour, and on a plain V4L2 host they can live entirely in the kernel module: one static mode table, a load-time parameter that picks an entry, and no device tree entries for the modes. This guide builds that table, sizes the MIPI timing behind it, wires it into the V4L2 hooks the receiver calls, and verifies the result.

Decide first: driver-owned modes or device tree entries

The kernel's own device tree usage documentation describes the device tree as a data structure and language for describing hardware. Frame size, frame rate and blanking are not wiring; they are how the driver drives the part, so they belong in code. Lane count, I2C address, regulators and the endpoint's data-lanes property are wiring, so they stay in the device tree. For a wider map of which layer owns which job on an embedded board, the embedded Linux development reference on kernel, device tree and drivers covers it.

Commit to the split with one test: if your capture application negotiates with plain V4L2 calls — formats, frame intervals, streaming — you can delete every per-mode device tree entry today. The sensor then advertises modes from the driver's table, and nothing on the host misses the removed nodes. If instead your SoC's ISP or acquisition middleware keeps its own mode list in the device tree, leave those entries: strip them and the middleware finds no camera, however healthy the sensor is. And trim only the mode entries, never the sensor node itself — a driver whose node vanished fails differently, and the camera driver and device tree mismatch diagnosis shows what that looks like.

What you need

• The kernel's V4L2 common functions and data structures documentation (driver API section of the kernel documentation): the types and helpers the pad-level hooks exchange.
• The sensor datasheet's register map, specifically the pixel-clock (PLL) registers and the frame-length and line-length registers. Names and addresses are sensor-specific; your table carries the values and the driver writes them.
• The receiver's supported lane rate band, from its datasheet or platform manual.
• The standard V4L2 command-line utilities (the v4l-utils package) for verification.

Build the mode table

One row per streaming configuration. Width and height are what the application sees. line_length and frame_length include the blanking you author around the active area. pixel_rate is the sensor pixel clock you assign to that row. lanes and bpp describe the wire, and mbus_code is the media bus code you report upstream. The frame rate is derived and never stored:

Keeping fps derived is what makes the table unable to drift from the registers. The rows themselves look like this — my own illustrative layout, typed for a RAW10 part:

struct sensor_mode {
    u32 width, height;    /* active pixels, active lines             */
    u32 line_length;      /* total pixels per line, blanking in      */
    u32 frame_length;     /* total lines per frame, blanking in      */
    u32 pixel_rate;       /* sensor pixel clock for this row, Hz     */
    u32 lanes;            /* CSI-2 data lanes                        */
    u32 bpp;              /* bits per pixel on the wire              */
    u32 mbus_code;        /* media bus code reported to the host     */
    const u32 (*regs)[2]; /* register/address sequence for this row  */
};

static const struct sensor_mode mode_table[] = {
    /* width height  ll    fl  pixel_rate  lanes bpp code               */
    { 1920, 1080, 2200, 1150,  75900000,   2,  10, MEDIA_BUS_FMT_SRGGB10_1X10 },
    { 1920, 1080, 2200, 1150, 151800000,   2,  10, MEDIA_BUS_FMT_SRGGB10_1X10 },
    { 1280,  720, 1400,  750,  63000000,   2,  10, MEDIA_BUS_FMT_SRGGB10_1X10 },
};

Two rows can share a size and differ only in pixel_rate — that is how one sensor delivers 1080p30 and 1080p60. The size enumeration reports the pair once; the interval enumeration reports 1/30 s and 1/60 s from the two rows. Every row also points at its register sequence, because a mode is not just numbers: it is the PLL, blanking and MIPI clock settings that make those numbers real. Respect the sensor's minimum blanking from its datasheet; blanking below the minimum breaks the sensor's internal timing.

Worked example: MIPI lane rate for a 1920 × 1080 RAW10 mode at 60 fps

The inputs

• Active area 1920 × 1080 px with RAW10, so bpp = 10 bit/px — CSI-2 packs four 10-bit pixels into five bytes. This is the second row above, the typical 1080p RAW10 embedded case.
• line_length = 2200 px and frame_length = 1150 lines: the blanking you author, 280 px (14.6 %) around the active line and 70 lines (6.5 %) around the active frame.
• lanes = 2, the endpoint's data-lanes value on this board.
• A receiver band of 80–1500 Mbps per lane, the range typical CSI-2 D-PHY receivers accept; D-PHY v1.2 parts extend to 2.5 Gbps.

The rule

The frame rate sets the pixel clock, the pixel clock and bits per pixel set the total payload, and the lane count divides it. The per-lane result must sit inside the receiver's band with headroom:

The arithmetic

1. pixel_rate = 2200 px × 1150 lines × 60 1/s = 151,800,000 px/s.
2. Total payload = 151,800,000 px/s × 10 bit/px = 1,518,000,000 bit/s.
3. Per lane = 1,518,000,000 bit/s ÷ 2 = 759 Mbps per lane.
4. Overhead check: a full-line long packet adds a 4-byte header and a 2-byte CRC, 48 bits per line, about 0.2 % of the 22,000 payload bits per line. Blanking and packet spacing cost a little more; the margin below absorbs them.

The result and the decision

759 Mbps per lane — inside the 80–1500 Mbps band with roughly a factor of two to spare. The threshold that generalises: at 10 bit/px, two lanes carry any mode whose pixel rate stays at or below 2 × 1500 Mbps ÷ 10 bit = 300 Mpx/s. This row needs 151.8 Mpx/s, so it runs on two lanes. The same timing at 120 fps needs 303.6 Mpx/s — over the threshold — so that row moves to four lanes and lands back at 759 Mbps per lane. Divide the pixel rate by 300 Mpx/s at 10 bit and round the lane count up; that is the lanes column decided. The MIPI CSI-2 frame rate estimation guide runs the same arithmetic from the lanes side.

Wire the table into the V4L2 hooks

1. enum_mbus_code: return each distinct mbus_code in the table — one here, MEDIA_BUS_FMT_SRGGB10_1X10.
2. enum_frame_size: walk the table and return each distinct width × height pair, skipping repeats.
3. enum_frame_interval: for a requested size, return one v4l2_fract per row of that size — {1, 30} and {1, 60} for the two 1080p rows.
4. set_fmt: accept a size only if a row holds it; snap anything else to the nearest row and write the delivered format back, because applications trust the applied format, not the requested one.
5. s_frame_interval: match the requested rate to the nearest row and store its index; report back the rate that row yields.
6. get_mbus_config: report V4L2_MBUS_CSI2_DPHY and the row's lane count, so the receiver configures its lanes before the first packet arrives.
7. s_stream: write the selected row's register sequence — pixel clock first, then line_length and frame_length into the line-length and frame-length registers — and start the sensor last.

If the sensor has no I2C control path at all, steps 1–6 stand and step 7 shrinks to stream on and off; the V4L2 CSI sensor driver without I2C control guide covers those stubs.

Pick the mode at load time, not in the device tree

One parameter selects the row:

static ushort mode_index = 0;
module_param(mode_index, ushort, 0444);
MODULE_PARM_DESC(mode_index, "Row index into the driver's sensor mode table");

Validate it first thing in probe: mode_index >= ARRAY_SIZE(mode_table) fails with -EINVAL, and nothing downstream ever sees an uninitialised row. Parameters select rows; they never invent timing. A width, height and rate triple that matches no row must snap to a row or fail the probe, because a mode without its register sequence streams garbage. At 0444 the value is readable under the module's parameters directory in sysfs, so a running system shows which mode it loaded for. Passing mode_index=1 when the module loads selects the 1080p60 row of the example table.

Verify the modes

1. List what the driver advertises and compare against the table: v4l2-ctl --list-formats-ext should print 1280 × 720 at 60 fps and 1920 × 1080 at 30 and 60 fps, and nothing else.
2. Check the negotiated rate: v4l2-ctl --get-parm reports the frame interval the pipeline settled on.
3. Measure instead of trusting: grab 120 frames and timestamp them. 120 frames in 2.00 s is 60.0 fps. Within ±2 % of the table value is healthy; repeat for every row.
4. media-ctl -p prints the media graph. Every pad on the path should show the selected width, height and code; a pad fixed to one format hides table rows from applications, and the guide to verifying advertised camera modes shows where that negotiation goes wrong.

Common mistakes

• Advertised rate the registers cannot deliver. enum_frame_interval says 60 fps while pixel_rate / (line_length × frame_length) gives 30. Applications schedule on the advertised number.
• Rows with no register sequence. The sensor ends its real frame early, the receiver reports short frames and buffers get dropped — the CSI short-frame isolation guide tells you which side misreports.
• Lane count changed in the table but not in the endpoint. The receiver keeps the device tree's count; the probe fails or the stream stays dead. The two-lane versus four-lane probe failure guide walks it.
• Receiver line padding disagreeing with the active width. The classic symptom is solid green frames, and the pixel clock and padded line width guide traces it.
• Timing invented at the parameter level. A mode that exists only as module parameters, with no row behind it, streams nothing useful.

When not to move the modes into the driver

Two cases reverse the decision. If the SoC's ISP middleware keeps its own mode list in the device tree, driver-only modes leave it staring at nothing, so ship the entries it reads. And if nobody on the project wants to maintain register sequences, buy capture hardware that ships supported instead of writing the table: area scan cameras on GigE, USB3, 10GigE and CoaXPress come with their own capture stacks, and board-level cameras exist to build into a machine that already has a host.

For the embedded route, the board level camera selection guide narrows sensor and interface first. If this turns into a shopping decision rather than a driver problem, the camera finder filters by interface, resolution and shutter, the configurator starts at the camera step, and the vision systems topic collects the rest of the bring-up guides.

Checklist before you ship the driver

Table

• Every row has a register sequence you have streamed on hardware.
• For each row, pixel_rate / (line_length × frame_length) matches the advertised rate to better than 1 %.
• Blanking clears the sensor datasheet's minimum line and frame lengths.
• Duplicate sizes are separated by distinct frame intervals.

Device tree

• The node keeps compatible, reg, the endpoint with its data-lanes count, and supply wiring.
• Per-mode entries are removed only after you confirm nothing on the host reads them.

Receiver

• For every row, pixel_rate × bpp ÷ lanes sits inside the receiver band with headroom, as in the worked example.

Parameters and verification

• mode_index is validated in probe and readable under sysfs.
• --list-formats-ext matches the table exactly, measured frame rate lands within ±2 % per row, and media-ctl -p shows matching formats on every pad.