IP camera stream into image processing: snapshot URL vs raw TCP

A network camera on the machine line is an easy way to get pixels into your processing code — until you open a raw TCP socket and start staring at bytes that are not frames. The camera already serves its video through a documented HTTP interface: one complete JPEG per snapshot request, or a continuous MJPEG stream. Your job is to read that URL, decode the JPEG into a pixel buffer, and size the loop so fetch and decode fit the frame budget. This guide gives the decision table, the loop to copy, the bandwidth arithmetic for a 640 × 480 snapshot at 15 fps, and the point where a network camera stops being enough and a GigE Vision camera earns its place.

The decision: read the documented URL, never parse the socket

An IP camera that shows a web page is an HTTP server, and its video is already packaged for you. The socket layer in your operating system assembles the transport; the HTTP interface marks where each image begins and ends. The only real choice is which documented URL your code reads.

For image processing, the snapshot URL and the MJPEG stream URL are the working answers. RTSP earns its place for viewing and recording. The raw socket earns almost none; the sections below show why, and what to do instead.

What you need

• The camera on the machine network, its address fixed by a DHCP reservation or a static address.
• Its manual or the manufacturer's product page, which names the snapshot, stream and RTSP paths for that model.
• A JPEG decoder in your processing environment — every common language has one built in or one call away.
• A timer loop with timestamps, so you can prove the rate you are actually getting.
• For RTSP only: a media framework such as FFmpeg or GStreamer that decodes H.264 into frames.

Find the camera's documented URL

1. Open the camera's web page in a browser at its address and run the live view once.
2. Read the manual or product page for the exact paths. Three shapes cover most cameras: http://<camera-ip>/jpg for a single snapshot, http://<camera-ip>/stream for a continuous MJPEG stream, and CGI endpoints of the form http://192.168.1.148/cgi-bin/view.cgi?action=snapshot. These are shapes, not universal paths — your camera's paths are its own.
3. Fetch each URL outside your code first: a snapshot URL should return a still image, a stream URL should update continuously.
4. If the camera requires credentials, most HTTP servers take them as URL parameters or basic authentication; the MJPEG stream URL with username and password recipe shows the working forms.

When RTSP is the only path the camera advertises and you would rather have MJPEG, the guide to finding the MJPEG CGI URL when RTSP fails walks the search.

The pull-decode-process loop

The loop is four operations plus one guard, repeated for every frame:

1. Fetch. Request the snapshot URL and confirm the response type is image/jpeg before trusting the bytes.
2. Check the markers. A JPEG starts with the bytes FF D8 and ends with FF D9; anything else means a broken or non-image response, so drop the frame and log it.
3. Decode. Hand the bytes to your JPEG decoder and get a pixel buffer. Reuse one buffer across frames instead of allocating a new image each time; the buffer reuse pattern is what keeps long loops from leaking.
4. Process. Run the inspection on the buffer, then store or discard the result.
5. Time it. Timestamp every completed frame. Target rate minus achieved rate is your first diagnostic.
6. Guard. Wrap fetch and decode in reconnect logic: on any error, close, wait 1-2 s, reopen. Cameras reboot and leases change; a loop that dies on the first dropped connection is the integration that gets pulled.

For continuous capture, request the stream URL once and split the response into frames at the JPEG markers instead of issuing one request per frame. The full recipes for grabbing images from an IP camera over an HTTP snapshot URL and for polling a snapshot URL on a timer carry the details; more acquisition patterns sit on the systems topic page.

Verify the set-up by leaving the loop for an hour with logging on: a steady achieved fps and no reconnect storm mean the loop holds. A decaying rate points at the camera's encoder, not at your code.

Worked example: sizing a snapshot loop for 640 × 480 JPEG at 15 fps

Inputs, and where each comes from:

• Frame size 50 KB per snapshot, a typical JPEG size for 640 × 480 colour at usable quality.
• Target rate 15 fps.
• Link: a dedicated 100 Mb/s Ethernet port to the camera. KB means 1,000 bytes here, and link rates are decimal bit/s.

The rule in words: average link traffic equals frame size times frame rate, and bytes become bits by a factor of 8.

The arithmetic, step by step: 50 KB × 15 fps = 750 KB/s = 750,000 B/s. Multiply by 8 bit/B and the average traffic is 6,000,000 bit/s, or 6 Mb/s. Each frame crosses the link as a 400 kb burst (50,000 B × 8 bit/B = 400,000 bit), which takes 4 ms to serialize on a 100 Mb/s port (400,000 bit ÷ 100,000,000 bit/s = 4 ms).

Result: 6 Mb/s average against 100 Mb/s of link — 6% utilization — and a frame budget of 66.7 ms per image.

Decision: the link is not the constraint; the frame budget is. Fetch plus decode must land inside 66.7 ms. A current JPEG decoder runs on the order of 100 MP/s per core, so this 0.31 MP frame (640 × 480 = 307,200 pixels) decodes in about 3 ms, and the fetch adds single-digit milliseconds — the budget holds with room to spare. Stay on the snapshot loop while fetch plus decode stays under about two-thirds of the budget, roughly 45 ms here. Above that, or when frames grow to 1080p-class sizes, move to the MJPEG stream URL and drop the per-request overhead.

Why raw TCP parsing never gets there

The plan — open a TCP socket, read everything, and pick the video out of the bytes — fails because there is nothing to pick. Your operating system's network stack handles Ethernet framing, IP routing, TCP acknowledges and retransmission on its own; by the time code reads the socket, it sees one ordered byte stream with no acknowledges in it. What the stream lacks is frame boundaries, and an application protocol defines those, not the network. HTTP is that protocol for a web camera: headers name the content, and each JPEG carries its own start and end markers. Without the vendor's format document, marker bytes alone cannot tell you where the camera's own headers stop and the image data begins.

This is the same reason a raw UDP camera is not detected by your vision SDK: SDKs speak documented protocols, not improvised byte streams. GigE Vision applies the idea to industrial cameras — a standard stream with documented packet sizes, timeouts and MTU behaviour, so a stalled grab has documented fixes (GigE Vision stream timeout). If a device genuinely speaks only an undocumented TCP protocol, the legitimate fix is the vendor's protocol specification, not packet surgery. Cameras sold as a USB module behind a USB-to-Ethernet adapter often publish no stream format at all; their documented access is the web page and its snapshot CGI, which is one more reason to read the HTTP URL.

RTSP: when continuous video beats discrete frames

When the camera serves H.264 over RTSP — the path its manual lists, usually beginning rtsp:// — do not decode it yourself. Pull the stream through FFmpeg or GStreamer, which handle the RTSP handshake, the RTP packetization and the codec, and hand your code decoded frames to buffer. That is the right path for live viewing, remote links and archived footage, and the same stream plays in an HMI (displaying an RTSP stream in an HMI).

For measurement, prefer the JPEG paths. H.264 is lossy and inter-frame: it encodes differences between frames, so compression artifacts are not contained in one image, and a block edge can sit a fraction of a pixel from the true edge. Snapshot JPEG and MJPEG compress each frame on its own, which keeps the error local and consistent enough to calibrate around. A system that measures should get its pixels from a camera built for measurement, not from a codec built for bandwidth.

When a network camera stops being enough

The snapshot and stream paths carry presence checks, label and code reading at modest rates, and station monitoring. Step up to an industrial camera when the job needs any of these: global shutter for moving parts, exposure set in microseconds, a calibrated photometric response, raw Bayer data you debayer yourself, or sustained rates beyond what the camera's encoder offers.

A GigE Vision camera runs over the same Ethernet cable, but its SDK hands your code complete frames with exposure, gain and trigger under your control — no URL hunting and no decode step, and you can check the raw stream directly (checking a raw Bayer stream). The LS-CMR-0053 streams 12 MP frames over GigE at 9 fps. Beyond it, the area scan cameras range goes from small mono sensors to 25 MP and more, and the camera finder filters by resolution, shutter and interface.

Run the loop on a vision controller

A desktop tower runs the loop; a fanless vision controller runs it for years inside the cell, with several network ports and its own light outputs. The MV-VC3301H-128G66 carries 5+ LAN ports, 3-4 USB 3 ports, 4 built-in light channels and 8 GB of memory, so the camera network stays separate from the office network. Where the camera powers over its data cable, the MV-VC5525-256G70-NN has PoE camera ports, so the cable that carries frames powers the camera too.

Common mistakes that stall the loop

• Fetching faster than the camera encodes. Snapshot endpoints are often rate-limited on the camera side; identical consecutive frames are the signature. Move to the stream URL instead of hammering the snapshot.
• Feeding bytes to algorithms. The fetched blob is a JPEG file, not pixels; decode it first — the distinction between image bytes and pixel data is the whole bug.
• No reconnect path. Every camera reboots eventually; a loop without close-wait-reopen dies the first time.
• Measuring through H.264. Codec edges drift; gauge on JPEG frames or on an industrial camera instead.
• It plays in the viewer but not in your code. The URL is almost always the difference — a missing parameter, the wrong port, or authentication the viewer adds silently. The same failure pattern is covered in resolving IP camera viewer URL and stream errors.

Decide by task and rate: the snapshot URL for discrete frames at VGA-class rates, the MJPEG stream URL for continuous capture, RTSP for viewing, an industrial GigE camera for measurement. Size the loop with the arithmetic above, pick the box that runs it from the vision controllers page, and when the case lands on an industrial sensor, the camera section above has the finder. To have the acquisition sized from your three numbers — frame size, target fps, measured decode time — send them through the contact page.