Cameras & sensorsHow-to

Camera debug tracing: check the driver, set the debugfs log level

A debugfs log-level write fails when the camera debug driver is not loaded. Check lsmod and the kernel config, enable the option and rerun the camera tracing.

LumaSpec Vision9 min read Share on LinkedIn

On this page 9 sections
  1. 01Why the write fails before it starts
  2. 02What you need
  3. 03Enable the trace, step by step
  4. 04Worked example: a level-2 write fails and the config has no camera debug line
  5. 05Read the trace once the level takes
  6. 06When the level takes but the trace stays coarse
  7. 07Leave the debug driver in every kernel you ship
  8. 08Does this apply to your camera?
  9. 09Questions

You follow the camera bring-up guide, run the one command that raises trace detail — echo 2 into a log-level node under /sys/kernel/debug — and the shell answers "No such file or directory". Every log afterwards stays too coarse to debug the pipeline. That failure almost never means tracing is unsupported on your board. It means the camera debug driver that creates the node never made it into the running kernel.

This guide takes you from the failed write to a working trace: confirm the debugfs mount, check the loaded modules, read the kernel config to see whether the driver was built in, shipped as a module or left out, then set the level your guide asks for. It applies to camera pipelines built into embedded Linux boards — the kind you build when you mount board-level cameras onto a carrier.

Why the write fails before it starts

Debugfs is a free-form tree. The /sys/kernel/debug directory holds only what loaded drivers choose to create; there is no fixed camera section. The log-level node appears the moment the debug driver that registers it is present in the kernel, and it is absent when that driver is not. Nothing announces the absence: boot is clean, the camera still captures, and the only symptom is a missing file plus logs with too little detail.

The capture driver and the debug driver are separate pieces, which is why the symptom misleads. Your capture path can be perfect while its debug companion is missing — the camera streams normally and only the trace granularity suffers.

Two faults produce the same error. Debugfs may not be mounted (unusual on vendor kernels, common on stock desktop ones), or the driver is missing. Settle the mount question first: mount | grep debugfs should print a line containing /sys/kernel/debug. No output means mount it — sudo mount -t debugfs debugfs /sys/kernel/debug — and retry the write.

What you need

  • Root access on the board. Watch a verbose trace from a serial console: a flooded SSH session can stall, a console keeps printing.
  • The kernel configuration of the running build: /proc/config.gz, or /boot/config-$(uname -r).
  • The camera bring-up documentation for your platform: it names the debug module, the debugfs path and the valid level values.
  • A kernel source tree matching the running version, in case the check ends in a rebuild.

Enable the trace, step by step

  1. Reproduce the failure and read the exact error. Write the level the way root actually performs it: echo 2 | sudo tee /sys/kernel/debug/camrtc/log-level. "No such file or directory" points at a missing mount or node; "Permission denied" means the redirect ran as your own user before sudo could act — the classic plain sudo echo 2 > /path trap.
  2. Confirm the mount: mount | grep debugfs. Nothing printed? Run sudo mount -t debugfs debugfs /sys/kernel/debug and retry.
  3. List the node's parent: ls /sys/kernel/debug/camrtc/. A missing directory means the driver is not in the kernel. A directory without your node means a driver or kernel version mismatch — compare against the guide.
  4. Search the loaded modules: lsmod | grep -i camera. A hit proves the module is loaded. No hit proves nothing: a driver compiled into the kernel never appears in lsmod.
  5. Read the kernel config: zcat /proc/config.gz | grep -i camera, or grep -i camera /boot/config-$(uname -r). Find the line matching the debug driver your guide names. =y means built into the kernel image. =m means a loadable module. No line means the option was never set.
  6. =m: load the module — modprobe with the exact module name from the guide — then repeat step 1.
  7. No line: enable the camera debug option in the kernel configuration, rebuild and flash exactly as the vendor's build guide prescribes, reboot, and repeat from step 3.
  8. Set the level and read it back: echo 2 | sudo tee /sys/kernel/debug/camrtc/log-level, then cat /sys/kernel/debug/camrtc/log-level prints 2. More detailed lines start arriving in the kernel log.

Worked example: a level-2 write fails and the config has no camera debug line

Inputs, and where each comes from:

  • The command and its error: echo 2 > /sys/kernel/debug/camrtc/log-level returns "No such file or directory". The platform's camera bring-up guide names this command in its debugging section, and its level 2 is the example value for a medium-detail pipeline trace.
  • The console fact: kernel dynamic-debug messages sit at severity 7, and a console prints them when its own log level is 8 — both numbers are the standard printk severities in the Linux kernel's own logging scheme.
  • The diagnostic facts: lsmod lists only modules; the kernel config records a driver as built in (=y), a loadable module (=m) or absent (no line).

The rule in words: a debugfs node exists exactly while the driver that registers it is in the running kernel. A missing node therefore means an unmounted debugfs or a missing driver, and the config file settles which.

The arithmetic:

  1. mount | grep debugfs prints 1 line — debugfs is mounted, so the mount is not the fault.
  2. ls /sys/kernel/debug/camrtc/ returns "No such file or directory" — 0 entries where the node should live.
  3. lsmod | grep -i camera returns 0 lines — not loaded as a module, which alone is no verdict.
  4. The config grep returns no line for the debug driver — the rebuild branch. (A =m line would have meant 1 command, no reboot; a =y line would have moved the search to binding and the device tree.)

Result: 0 modules and no config line put this board in the rebuild branch. Enable the option, rebuild, flash, reboot. The directory now lists the log-level node, echo 2 followed by cat reads back 2, and the trace gains the detail the guide promises.

Decision, with thresholds: treat the config grep as the verdict, not lsmod. =m means one command away. No line means one rebuild away. =y means stop checking the kernel and look at the boot log and device tree instead. Before budgeting the rebuild, confirm the guide documents tracing for your SoC variant at all — some families ship the debug driver on some variants only. For the console, the kernel's printk numbering puts dynamic debug at severity 7 and prints a message when message_level < console_loglevel, so sudo dmesg -n 8 puts the whole level-7 trace on the serial console.

console prints a message when message_level < console_loglevel dynamic debug messages: message_level = 7 dmesg -n 8 -> console_loglevel = 8 -> 7 < 8 -> full trace on the console

Read the trace once the level takes

Stream the kernel log with sudo dmesg -w, or journalctl -kf on systemd boards, and reproduce the fault while the console shows the level-7 lines. For a record you can attach to a ticket, snapshot the ring buffer right after the failure: dmesg > camera-trace.log.

When the level takes but the trace stays coarse

Three checks, in order:

  1. Wrong node. Platforms often expose several debug nodes side by side, and only the one the guide names controls trace verbosity. Writing a neighbouring node succeeds and changes nothing.
  2. A second level on the co-processor. Pipelines that run part of the camera stack on a co-processor keep a separate trace there, often with its own log-level node; the kernel-side level does not raise it. The guide lists both.
  3. Trace points compiled out. If the driver is built in and bound but prints nothing, the kernel may lack dynamic debug support — without it, debug-level prints do not exist whatever you write. ls /sys/kernel/debug/dynamic_debug/ answers that in a second.

Capture-side trouble — missing nodes, wrong formats — sits upstream of tracing. The next stop there is media-ctl pipeline setup: fix capture node failures on embedded SoCs.

Leave the debug driver in every kernel you ship

  • Enable the camera debug option in the baseline kernel configuration for production builds, so any field unit can be traced without a special flash.
  • Record the debug module name, the debugfs path and the valid level values next to each release's notes.
  • Add the echo-and-read-back check to bring-up sign-off: it takes seconds and catches a config regression before it costs a debug session.
  • Document that a =y driver never appears in lsmod, so the next engineer reads the config symbol instead of repeating the false "not loaded" conclusion.

Does this apply to your camera?

Debugfs log-level tracing belongs to camera pipelines built into embedded SoCs: MIPI CSI-2 modules and the camera boards you mount on a carrier. A standard machine-vision camera on a PC — GigE Vision or USB3 Vision — captures through a vendor user-space library on top of ordinary network and USB drivers, so there is no camera debugfs tree to enable.

The equivalent first checks for those cameras: the GigE transport layer that must load before any camera answers (ping works, zero cameras: GigE transport layer not loaded), udev permissions that leave a USB3 camera root-only (USB3 Vision camera works as root only: fix the udev rule), and the UVC driver a standard webcam needs enabled in the kernel (camera detected but no video: enable the UVC driver in your kernel).

More fixes of this kind sit on the systems topic page. If you are choosing a module for an embedded build and want a driver story you can support in the field, the camera finder filters by interface, mount and shutter, and the area scan cameras category is the shelf for factory-floor models. A build decision that needs a second opinion goes to the contact page, where it reaches our applications team.

Questions

Why does the shell report 'No such file or directory' for a debugfs log-level node?

Two causes cover almost every case: debugfs is not mounted, or the driver that creates the node is not in the kernel. Run `mount | grep debugfs` first; no output means mount it with `sudo mount -t debugfs debugfs /sys/kernel/debug`. If the mount is present but the node is not, go straight to the lsmod and kernel-config checks — the driver was left out of the build.

What number should I write to the camera debug log level?

The values are vendor-defined, not Linux-standard, so the bring-up guide names the level for the detail you want; a level of 2 is the usual example for a medium-detail pipeline trace. Kernel-side dynamic-debug messages sit at severity 7, so a console log level of 8, set with `dmesg -n 8`, is what displays them.

How do I tell a built-in camera driver from a loadable module?

Read the config for the running build: `zcat /proc/config.gz | grep -i camera` or `grep -i camera /boot/config-$(uname -r)`. A line ending in `=y` means compiled into the kernel image, and such a driver never appears in `lsmod`. `=m` means a module you can load with `modprobe`. No matching line means the option was never set and a rebuild is required.

Does this debugfs method apply to GigE or USB3 Vision cameras on a PC?

No. Those cameras capture through a vendor user-space library over ordinary network or USB drivers, so there is no camera debugfs tree to enable. Their first checks are the GenICam transport layer for GigE, udev permissions for USB3 and the UVC driver for standard USB webcams. The debugfs log-level method belongs to camera pipelines built into embedded SoCs.

The echo now succeeds but the trace detail has not changed — what next?

Check three things in order. Confirm you wrote the node the guide names for verbosity, since platforms often expose several debug nodes side by side. Look for a separate log-level node for the camera co-processor's own trace, which the kernel level does not raise. If the driver binds but prints nothing, dynamic debug support may be compiled out; check that /sys/kernel/debug/dynamic_debug/ exists.