USB camera into a ROS2 DetectNet pipeline: replace the rosbag input

A DetectNet quickstart launch proves the pipeline by replaying a rosbag: the same frames, the same detections, every run. Good for a smoke test, useless for a camera on your bench. The swap to a live USB camera is smaller than it looks. The detection graph does not know or care where its sensor_msgs/msg/Image messages come from, so you replace the bag player with a camera node that publishes the same message type on the same topic. This guide shows the edit with the usb_cam package on ROS2 Humble, the encoding and resolution checks that keep frames flowing, the bandwidth arithmetic for your USB link, and the checks that prove detections on live frames.

What you need

• A ROS2 Humble environment where the DetectNet quickstart launch already runs end to end. If the bag replay produces detections, the detection side is proven and only the input changes.
• A USB camera the kernel exposes as a V4L2 device: a standard webcam or a capture card. ls /dev/video* answers in one line.
• The usb_cam package, installed in the same container or environment where you run the launch: sudo apt install ros-humble-usb-cam
• v4l-utils, for the v4l2-ctl tool that lists a camera's formats and frame rates.
• Edit access to the quickstart launch file, or a package of your own to compose a new launch.

What the bag player does, and what replaces it

A launch file is a recipe that starts nodes. The quickstart recipe does two jobs: it starts the detection graph, and it starts a player that republishes recorded sensor_msgs/msg/Image messages onto the topic the graph subscribes to. The graph consumes whatever arrives on that topic and nothing else.

That makes the task narrow: remove the player, start a publisher of the same message type on the same topic, with an encoding the graph accepts. Find the topic first, while the bag still plays. ros2 topic list shows the active names; ros2 topic info on the image topic prints its type, which should read sensor_msgs/msg/Image. Note the exact name — the camera node must publish it character for character.

Install the camera node and find the device

1. Install the usb_cam package in the detection environment: sudo apt install ros-humble-usb-cam.
2. Plug the camera into a port on the machine itself and list the devices with ls /dev/video*. Recent kernels list several nodes per camera — one captures frames, the others carry metadata — so confirm which node offers formats.
3. Read the formats with v4l2-ctl -d /dev/video0 --list-formats-ext. Note the uncompressed formats, GREY and YUYV on most webcams, and the size and frame-rate pairs on offer. Write down the largest uncompressed size at the rate you want.
4. Start the camera node on its own with ros2 run before touching the launch. Point its device and format settings at what step 3 showed, then run ros2 topic hz on its output topic: it should report the configured rate.

If no device appears at step 2, the USB link checks in camera has power but not detected: USB link checks apply to webcams the same way. Swap the cable and the port before suspecting the camera.

Swap the player for the camera node

1. Open the launch file you actually execute, the one named in your ros2 launch command, and search it for the player: the node entry whose package or name references the bag.
2. Comment out that entry, then check the file's includes too. Some quickstarts start the player from an included launch file rather than the top-level one.
3. Add a node entry for the usb_cam package in the same file. Set its device, resolution, pixel format and frame rate from your v4l2-ctl notes, and remap its output topic to the exact name you recorded from ros2 topic list.
4. Launch the same command as before. Nothing on the detection side changes: same graph, same topic, live frames.

The cleaner structure for anything you will keep: build your own package in your workspace's src directory and compose the detection nodes and the camera node in your own launch file. The quickstart stays untouched as the reference, and your launch becomes the one you maintain.

If a camera setting you pass never takes effect, the usual cause is another launch argument winning over your parameter. ROS launch overrides your camera config walks through that failure mode.

Match the encoding and resolution

Read the bag's encoding before you stop it. With the quickstart still playing, run ros2 topic echo on the image topic with the --field encoding and --once flags. The printed string is exactly what the graph has been consuming.

Configure the camera for an 8-bit format: mono8 for a mono sensor, or a bayer_rggb8-style encoding for a color sensor read raw. The camera node converts each V4L2 format to a ROS encoding, so read what it publishes with echo rather than trusting the camera's spec sheet. If mono8 arrives where you expected Bayer, that is the fallback covered in camera image encoding falls back to mono8: request Bayer instead; request the Bayer encoding explicitly instead of accepting the substitution.

The network's input has a fixed size and DetectNet-style preprocessing scales each frame to it, so the bag's resolution does not bind you. Start at a 1080-class resolution and judge from the detections. Raise it only when the smallest object of interest lands too small after scaling, because every extra pixel costs USB bandwidth and CPU at the rates the worked example below puts numbers on.

Last, tame auto exposure. A webcam hunting exposure between frames changes brightness and frame timing together. Once the camera points at a fixed scene, lock exposure and gain, then re-check the frame rate, because long exposures cap the achievable rate. Runtime camera parameter control shows how to adjust exposure and white balance without restarting the node.

Worked example: a 1920 × 1080 USB camera at 30 fps over USB 3.0

The inputs, and where each comes from:

• 1920 × 1080 at 30 fps: a typical UVC webcam's top uncompressed offer. Read your camera's real numbers with v4l2-ctl --list-formats-ext.
• Bytes per pixel, from the ROS encoding definitions: mono8 and the 8-bit Bayer formats carry 1 B/px; yuv422 carries 2 B/px.
• Port limits: USB 3.0 signals at 5 Gbit/s and moves about 400 MB/s in practice after protocol overhead; USB 2.0 signals at 480 Mbit/s and manages about 40 MB/s.

The rule in words: multiply width by height by frame rate by bytes per pixel, then compare with the port's practical ceiling.

The arithmetic:

• Pixels per frame: 1920 × 1080 = 2,073,600 px
• Pixel rate: 2,073,600 px × 30 fps = 62,208,000 px/s
• mono8: 62,208,000 px/s × 1 B = 62,208,000 B/s ≈ 62.2 MB/s ≈ 498 Mbit/s
• yuv422: 62,208,000 px/s × 2 B = 124,416,000 B/s ≈ 124.4 MB/s ≈ 995 Mbit/s
• mono8 at 15 fps: 2,073,600 px × 15 fps × 1 B = 31,104,000 B/s ≈ 31.1 MB/s

The result, and the decisions. Both raw encodings fit under USB 3.0's roughly 400 MB/s ceiling, so this stream is not bandwidth-limited on a real 3.0 port. Keep yuv422 only if the graph consumes it; dropping to the 1-byte format halves the stream, 124.4 to 62.2 MB/s, for nothing lost. On a USB 2.0 port the same mono8 stream needs 498 Mbit/s against a 480 Mbit/s line rate and cannot pass uncompressed at 30 fps. The workable threshold there is about 40 MB/s: 1080p mono8 meets it at 15 fps (31.1 MB/s) and misses it at 30. Decision: on a 3.0 port with an 8-bit format, run the full rate; on a 2.0 port, halve the frame rate or the resolution before blaming the model.

Two such mono8 streams at 30 fps need about 124 MB/s between them, inside one USB 3.0 controller's budget. The acquisition plan for that case is the multi-camera ROS launch guide.

Verify the pipeline end to end

1. ros2 topic hz on the camera topic reports the configured rate, within a frame or two.
2. One echoed message shows the encoding, width and height you set.
3. Time the detection output with ros2 topic hz as well; that number, not the camera rate, is your pipeline rate. If the camera outruns the graph, set the camera's frame rate to the measured graph rate. When the rate sags for reasons you cannot see, the ROS2 image publishing frame-rate guide shows how to find which stage is holding it down.
4. Point the camera at the real parts and compare detections against the bag's. The bag was your regression set; record a new one from the live camera and keep it as the next regression set.

The rosbag is still running: why your edit did nothing

In order of likelihood:

1. You edited the source file, but ros2 launch executes the installed copy. Rebuild with colcon build, source the install space again, or run the launch by the full path to the file you edited.
2. The player starts from an included launch file. Search the includes for the bag entry the same way you searched the top-level file.
3. An earlier launch is still alive. ros2 node list shows the player's node still registered. Stop the old process, then launch again.

The clean sign of a working edit: the bag's topic stops publishing within seconds of launch, and ros2 topic hz on it reports nothing.

More guides of this kind — frame rates, encodings, launch fights — sit on the systems topic page.

When a webcam is not enough

A webcam proves the graph. A production cell asks for more: a global shutter that does not smear parts moving through the frame, a C-mount lens chosen for your field of view, hardware trigger input to freeze parts in step with the line, and a housing with a lockable connector. Industrial USB3 Vision cameras bring all of that. They publish through a GenICam producer rather than the UVC class, so instead of usb_cam you wrap the camera's SDK in a small ROS2 publisher node, or use a vendor ROS2 driver where one exists, and remap it onto the same image topic. The detection graph stays exactly as it is.

The USB3 models below span 6 MP mono, 12 MP global shutter, and 12 MP at 30 fps:

Pick the global-shutter model for parts in motion; the rolling-shutter parts suit static scenes and slower lines. Compare the field in the camera finder, or work backwards from the part in the camera configurator. If the deployment is simpler without a full PC, smart cameras run their inspection on the device itself: you trade your own graph for a boxed, supported package, while area scan cameras stay the choice when the graph must remain yours.

The decision, per case

• Bench prototype, static or slow scene: run the webcam through usb_cam at the graph's measured rate. A 1080-class mono8 stream on a USB 3.0 port carries with margin, per the worked example.
• Parts in motion, trigger sync, or an unattended cell: keep the graph, swap the camera side to a USB3 Vision global-shutter model behind a thin publisher node.
• No PC wanted on the machine: put the inspection on a smart camera and keep ROS2 for the stations that need it.

For the second case, the MV-CH120-11UC is that class of part: 12 MP, global shutter, USB3.

If you want the camera side checked against the graph you already run before you order, the contact page is the route.