You have a page that plays a camera stream through opencv.js, and a recognition API that runs somewhere else. The missing link is the hand-off: getting one frame out of the capture loop and into a POST request. The loop itself looks fine — the trap is that cap.read(src) returns only true or false, so any variable assigned from it stays a boolean and the pixels never leave the Mat. The four steps below move the frame — Mat to canvas, canvas to JPEG string, string to your API — with the payload numbers and loop pacing that keep the page stable at a real frame rate.
cap.read fills the Mat and returns only a boolean
cap.read(src) follows the same convention as desktop OpenCV: it reads the next frame into src and returns the boolean from the grab — true when a frame arrived. So this line from a typical loop does nothing useful:
let Frame = cap.read(src); // Frame is true or false, never an image
The frame is already in src. There is no separate frame object to fetch: process src directly or hand it to the next step. That is why the debugger shows nothing meaningful on the return value — there is nothing there but true.
What you need
- A video element with a playing stream. From a webcam: call
navigator.mediaDevices.getUserMedia({video: true}), assign the returned stream tovideo.srcObject, thenvideo.play(). The browser prompts for permission and rejects with a documented error if the device is missing or denied. - The opencv.js module loaded, with the loop started only after it reports ready.
- A canvas element for encoding. It does not need to be large or prominent, but it should stay in the layout.
- An API endpoint that accepts a POST with an image field, on the same origin or with CORS headers set.
- A camera the browser can open as a video device — a webcam, or any camera whose driver registers it as a standard video device. Machine-vision cameras take a different path, covered at the end.
The grab loop below uses the same wiring as the live edge detection guide, with the POST added after the draw.
Step 1: grab one frame into the Mat
let video = document.getElementById('videoInput');
let src = new cv.Mat(video.videoHeight, video.videoWidth, cv.CV_8UC4);
let cap = new cv.VideoCapture(video);
let ok = cap.read(src); // ok: boolean
if (!ok) throw new Error('no frame arrived');
Build the Mat from video.videoWidth and video.videoHeight — the stream's real size — and keep the two matched; a Mat of a different size stretches the frame. Allocate src once before the loop and reuse it every iteration rather than creating a fresh Mat per frame.
Step 2: draw the Mat onto a canvas
let canvas = document.getElementById('canvasOutput');
cv.imshow(canvas, src);
cv.imshow accepts a canvas element or its id and paints the Mat onto it. If the API wants grayscale, convert first — and delete the conversion Mat when you are done:
let gray = new cv.Mat(video.videoHeight, video.videoWidth, cv.CV_8UC1);
cv.cvtColor(src, gray, cv.COLOR_RGBA2GRAY);
cv.imshow(canvas, gray);
gray.delete();
Step 3: encode the canvas as a JPEG string
let dataUrl = canvas.toDataURL('image/jpeg', 0.8);
// 'data:image/jpeg;base64,...'
canvas.toDataURL returns a data URL: a short prefix, then the image as base64 text. The first argument picks the format — 'image/png' is lossless but lands several times larger for photographic content — and the second sets JPEG quality from 0 to 1. For POST bodies, 0.6 to 0.8 is the working range. If you would rather send binary, canvas.toBlob hands you a Blob asynchronously and skips the base64 inflation; the sizing numbers below show what that saves.
Step 4: POST the frame to your API
JSON with the data URL in one field is the simplest contract — the server decodes the string and hands the JPEG to the recognizer:
async function postFrame(dataUrl) {
let res = await fetch('https://your-api.example/recognise', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ image: dataUrl })
});
if (!res.ok) throw new Error('POST failed with ' + res.status);
return await res.json();
}
The URL and field name are yours to define. A binary body cuts the payload by a quarter:
canvas.toBlob(function (blob) {
let form = new FormData();
form.append('image', blob, 'frame.jpg');
fetch('https://your-api.example/recognise', { method: 'POST', body: form });
}, 'image/jpeg', 0.8);
One frame per request keeps failure handling simple: each response answers exactly one frame.
Size and pace the POST loop
Two estimates drive the numbers:
Worked example: a 640 × 480 frame at JPEG quality 0.8, typical photographic content — the 30-80 KB band, take 50 KB for the arithmetic.
- Raw RGBA would be 640 × 480 × 4 = 1,228,800 bytes ≈ 1.2 MB, about 24 times the JPEG. That factor is why the canvas route beats posting the pixel array.
- Base64 inflation: 50,000 × 4 / 3 ≈ 66,700 characters ≈ 67 KB, plus the 23-character
data:image/jpeg;base64,prefix — call the POST body 67 KB. - Upload time: 67,000 bytes × 8 = 536,000 bits = 536 kbit. A 100 Mbit/s link moves 100 kbit per millisecond, so 536 / 100 ≈ 5.4 ms; a 10 Mbit/s link needs 536 / 10 ≈ 54 ms.
- Pacing: a 5 fps loop runs on a 200 ms period, so even the slow link leaves margin for encoding and the API round trip. At 30 fps the period is 33 ms and the same frame no longer fits the wire — throttle the POSTs, crop, or drop resolution.
Typical sizes at quality 0.8, scaled by pixel count from that anchor:
| Frame | Pixels | Typical JPEG | POST body (base64) |
|---|---|---|---|
| 320 × 240 | 0.08 MP | 8-20 KB | 11-27 KB |
| 640 × 480 | 0.3 MP | 30-80 KB | 40-107 KB |
| 1280 × 720 | 0.9 MP | 90-240 KB | 120-320 KB |
| 1920 × 1080 | 2 MP | 200-540 KB | 270-720 KB |
Content moves these bands, so log dataUrl.length once per session and size from your own scene, not the table.
For the loop itself, use the tutorial's measure-then-delay pattern:
const FPS = 5;
function processVideo() {
let begin = Date.now();
if (cap.read(src)) {
cv.imshow(canvas, src);
postFrame(canvas.toDataURL('image/jpeg', 0.8)).catch(function (e) {
console.warn('post failed:', e);
});
}
let delay = 1000 / FPS - (Date.now() - begin);
setTimeout(processVideo, Math.max(delay, 0));
}
postFrame is not awaited, so the API round trip overlaps the period — but if the API is slower than the loop, requests queue in the browser and latency climbs silently. The hidden buffering guide is the playbook for finding exactly that kind of queue. If you await the POST inside the loop instead, the round trip becomes your period — the same trade-off as capture call versus image ready.
Verify the hand-off
Three checks, in order, prove the chain end to end:
- The Mat has pixels:
okwas true andsrc.data.lengthequals width × height × 4 — 1,228,800 at 640 × 480. - The string is a JPEG:
dataUrl.startsWith('data:image/jpeg;base64,')is true, and the decoded size lands in the band for your frame size:
console.log(dataUrl.slice(0, 23), Math.round((dataUrl.length - 23) * 3 / 4));
// 'data:image/jpeg;base64,' and the JPEG size in bytes
- The API received what you sent: expect
res.status200, and have the endpoint echo back the decoded image size. When your 67 KB POST shows up as 50 KB of JPEG on the server, the hand-off is proven.
Common mistakes
-
Building the Mat before the stream has a size. If
getUserMediahas not resolved and playback has not started,videoWidthis 0 and the Mat is created empty; wait for the video element to report its dimensions. -
A tainted canvas. A video from another origin without CORS approval makes
toDataURLthrow a SecurityError; serve the stream same-origin or with CORS headers. - Posting the pixel array. 1.2 MB per 640 × 480 frame instead of roughly 50 KB of JPEG — 24 times the bandwidth for the same picture.
- Fire-and-forget without a catch. A failing POST surfaces as an unhandled rejection while the loop keeps drawing; the page looks alive and the API receives nothing.
-
A hidden canvas that stops painting. Several mobile browsers have had
cv.imshowfailures ondisplay:nonecanvases; keep the canvas in the layout and test on the browser you deploy to.
When the browser is the wrong client
A browser cannot open an RTSP or GigE Vision stream: media elements take device cameras through getUserMedia and HTTP streams the browser can decode, nothing else. Two patterns cover machine-vision cameras.
Grab on a PC, post from the PC. Run the camera's SDK in a small service, encode there, POST from there. The PC-side access options guide lays out how to reach the frames, and HTTP CGI snapshots cover cameras that serve a single frame over HTTP. For this pattern pick a GigE area scan camera from the camera finder — the MV-CU020-90GC, a 2.5 MP GigE camera with a global shutter, is a typical bench choice.
- Sensor format
- 1/1.7"
- Lens mount
- C-mount
- Interface
- GigE
- Resolution
- Up to 3 MP
Inspect on the camera. If this page is a step toward a production check, skip the frame upload entirely: a smart camera runs the inspection on the device and sends results, so only the verdict crosses the network. The LS-SMC-0063, for example, captures 60 fps over GigE with its integrated 16 mm lens and white light — the loop that decides pass or fail never leaves the camera.
- Interface
- GigE
- Resolution
- Up to 3 MP
Whichever half you are building, configure the camera side or reach us through the contact page with the camera and API details. More grab-loop patterns live on the systems topic page.
Questions
Why does cap.read return false right after the page loads?
The stream is not playing yet. Wait for getUserMedia to resolve, set video.srcObject, call video.play(), and create the cv.VideoCapture once the video element reports its real size (the loadedmetadata event). A false that appears later usually means the browser paused a hidden or background tab's stream — resume playback and the next read returns true.
Can the browser read an RTSP or GigE Vision camera URL directly?
No. Media elements in a browser take device cameras through getUserMedia or HTTP-based streams the browser can decode; RTSP and GigE Vision are outside that set. For a machine-vision camera, grab frames with the vendor SDK in a small service on a PC and POST from there, or pick a camera that serves snapshots over HTTP. A smart camera sidesteps the transport question by inspecting on the device.
Can I send PNG instead of JPEG?
Yes. canvas.toDataURL('image/png') encodes losslessly, but photographic frames land several times larger than the JPEG and there is no quality knob to turn down. For API uploads, JPEG between 0.6 and 0.8 is the working range; encode one frame both ways once, log dataUrl.length for each, and keep the smaller.
How do I shrink the frame if the API rejects large bodies?
Three levers, largest effect first: draw the Mat onto a smaller canvas before encoding (a 320 × 240 canvas carries a quarter of the pixels of 640 × 480), drop JPEG quality toward 0.6, and crop to the region the API actually inspects. Sending a Blob through toBlob and FormData also removes the base64 inflation entirely.
How many frames per second can I post?
Budget bandwidth first: payload × rate. A 67 KB body at 5 fps moves about 335 KB/s, roughly 2.7 Mbit/s — comfortable on a 100 Mbit/s link. The API's own processing time usually sets the real ceiling, so time one round trip and keep the loop period above it. Firing posts without awaiting is how hidden queues grow.

