Vector subscript out of range in OpenCV Java DNN: causes and fixes

Your OpenCV DNN pipeline in Java dies with a dialog that reads vector subscript out of range, the JVM exits without a Java stack trace, and the path in the dialog points into a Visual Studio folder your machine has never had. The message is a native assertion raised inside the OpenCV library, and it hides two separate defects: a result value your Java code discards, and a model file the DNN importer cannot digest. This guide gives you the one-line minMaxLoc fix, a four-step isolation procedure that names the failing native call within minutes, the arithmetic that checks your classification and detection indexing, and the deployment habits that keep the assertion off the production controller.

Why the message points at code you never wrote

vector subscript out of range is not a Java exception. It is a bounds check on a C++ std::vector inside the OpenCV native library, reached from your code through JNI. The process dies at the C level, so the JVM exits without building a stack trace, and the dialog names a vector header under the MSVC tools directory of a Visual Studio 2019 install. That path belongs to the machine that compiled the library, often a build server, which is why the folder does not exist on your PC. The header tells you which compiler checked the bounds, not where the defect is.

The check lives in debug builds of the native library. Release builds drop it, so the same defect asserts on one machine and passes silently on another. That fact explains most of the confusion around this error: the program "worked yesterday" because yesterday's machine carried a different native build, not because the defect was absent.

Rank the causes before you touch anything. First, a model file the DNN importer mishandles: the assertion fires during import or on the first forward(). Second, a debug-built native library that turns a latent out-of-bounds read in the importer into a hard stop. Third, your own indexing, which normally produces silent garbage rather than this dialog but still needs fixing. The typical failing program also loads the native library twice, System.loadLibrary(Core.NATIVE_LIBRARY_NAME) followed by System.loadLibrary("opencv_java470"), and on 4.7.0 both calls resolve the same file, so drop the second call and remove one variable.

Defect 1: the discarded minMaxLoc result

This defect does not raise the dialog, but it sits in nearly every program that reports the crash, and it corrupts every prediction the moment the crash is gone. Core.minMaxLoc(Mat) returns its answer; it does not write into a Point you construct. Calling it and ignoring the return leaves your Point maxLoc at its constructor default of (0, 0), so the class index computes to 0 and the confidence reads element (0, 0): the first class, whatever the network actually saw.

// broken: the result of minMaxLoc is discarded, maxLoc stays (0, 0)
Point maxLoc = new Point();          // x = 0.0, y = 0.0
Core.minMaxLoc(agePreds);            // return value dropped
int ageIdx = (int) maxLoc.x;         // always 0

// fixed: assign the returned MinMaxLocResult, then read its maxLoc
Core.MinMaxLocResult mm = Core.minMaxLoc(agePreds);
int ageIdx = (int) mm.maxLoc.x;      // the column that holds the maximum

Core.MinMaxLocResult carries four public fields: double minVal, double maxVal, Point minLoc and Point maxLoc. The fixed pattern assigns the result and reads the winning column from mm.maxLoc.x.

An index pinned to 0 belongs to a larger family of failures: a mapping that silently points at the wrong entry. The label index mismatch guide shows how to test a whole index-to-class mapping, not just the argmax line.

Defect 2: the assertion inside forward()

When the dialog survives the minMaxLoc fix, the staged prints show a consistent pattern: the blob builds, setInput returns, and the process dies at or just after forward(). The importer's own log lands right before it: the TensorFlow importer logs how many nodes it parsed, and old Caffe prototxt files log an upgrade notice about deprecated V1LayerParameter fields on the way in. Treat those lines as context, not as the fault; the importer parsed something and then the native side went out of bounds.

Two patterns recur in the wild. ONNX exports have asserted exactly there on 4.7.0 debug builds; one upstream report records a YOLOv7 ONNX file dying after net.forward() on 4.7.0 debug x64. Older Caffe networks tell the mirror story: a pose network that imported and ran on the 3.4.x branch asserted on 4.7.0. And an 8-bit ONNX re-export of a TensorFlow face detector has imported and run where the original TensorFlow file asserted. The model file and the library version are a pair; either side can break it.

The tell that separates binding from model: run the identical file in Python. Same model, same blobFromImage scale, size, mean and channel swap. If the reference asserts too, the model/importer pair is broken and no amount of Java rewriting will help.

Isolate the failing stage in four steps

1. Fix defect 1 first. It is one line, it is wrong in every case, and leaving it in place poisons the results as soon as the crash stops.
2. Print one line after every native call in the chain: after each readNet... call, after setInput, after forward(), after the first get(). The last line on the console before the dialog names the failing call. Keep the prints behind a flag you switch off at deployment.
3. Run the identical model file in Python or C++ with identical preprocessing. If the reference asserts, the model/importer pair is faulty and you move to step 4 with a format change. If the reference runs, compare the two native libraries: version and build type. A Python side that will not import at all is a different fault; the cv2 version mismatch fix covers that error.
4. Change one variable at a time. Re-export the network to ONNX and load it with Dnn.readNetFromONNX. Or run the OpenCV branch that imported the file before, the 3.4.x branch for Caffe networks built years ago. Or swap a debug native library for a release build of the same version. Preprocessing must move across unchanged when you switch formats, or confidences drift; the preprocessing mismatch guide gives the checklist.

Once a format imports everywhere, choose where it runs by speed and memory, not by habit: the ONNX runtime choice guide compares the options inside OpenCV DNN.

Worked example: the numbers behind both indexing bugs

Take the age branch. forward() returns the eight class probabilities as one flat block, and agePreds.reshape(1, 1) lays them out as a single row of eight columns. Say the row holds 0.02, 0.01, 0.03, 0.08, 0.31, 0.41, 0.11 and 0.03, a softmax output, so the values sum to 1.00. Core.minMaxLoc scans the row once and puts the maximum, 0.41, at column 5. The fixed code reads mm.maxLoc.x as 5.0 and casts it, so ageIdx is 5.

The broken line reports the youngest bracket at 0.02 confidence for every face, in every frame. That is why the fix lands before the crash hunt: left in place, you ship a classifier that cannot lose.

The detector branch hides the second bug. A single-shot detector returns its candidates as a 1 × 1 × N × 7 blob: N candidates of seven values each, being image index, class, confidence, then x1, y1, x2, y2 normalized to the input frame. With N = 200, detections.total() is 200 × 7 = 1400, and detections.reshape(1, (int) detections.total() / 7) gives a 200 × 7 matrix whose row i is candidate i. Reading (i, 2) straight off the unreshaped blob addresses a 2-D location that does not correspond to candidate i, so the confidence and the box are whichever values land there.

Convert one detection on a 640 × 480 frame to see the arithmetic end to end. Candidate i carries confidence 0.91 and a box of (0.52, 0.31, 0.78, 0.79). The (int) cast truncates: x1 = (int)(0.52 × 640) = 332, y1 = (int)(0.31 × 480) = 148, x2 = (int)(0.78 × 640) = 499, y2 = (int)(0.79 × 480) = 379. The rectangle is new Rect(332, 148, 499 - 332, 379 - 148) = new Rect(332, 148, 167, 231), that is 167 px wide and 231 px tall, and it passes the 0.7 threshold because 0.91 > 0.7.

Match the evidence to the fix

Work the table from the top; one row is your case, and the evidence, not a hunch, picks it.

Deploy the pair you validated

What you ship is a trio: native library version, build type and model file. Freeze all three, record them with the project, and put the same trio on the machine at the line. A controller running a different native build can reintroduce the assertion months later, after a routine rebuild; nothing in your repository changed, so nothing in the logs explains it.

On the line, that trio lives in a vision controller next to the cameras. The MV-VC5525-256G70-NN carries 32 GB of memory, eight built-in light channels and PoE camera ports, so cameras, lights and the pinned OpenCV runtime sit in one box.

When inference grows past one general controller, a dedicated inference terminal takes the model load. The MV-VAC3504X-128G01 is an AI inference terminal with five or more LAN ports and four built-in light channels, and the MV-VC4712-256G10-2TH pairs an Ultra 7 255H processor with 16 GB × 2 of memory, six built-in light channels and PoE camera ports.

Checks before you call the pipeline fixed

• One native library per machine: a single known version, release build, loaded once on the library path, with the second loadLibrary call gone.
• The model file on the controller is byte-identical to the one you validated; compare a hash, not a filename.
• net.empty() checked after every load call, with the model path in the log line when it returns true.
• Twenty reference images through both runtimes: class indices agree and probabilities agree to two decimals.
• Every list index guarded by the length of the list actually loaded; an eight-entry bracket list breaks the day a model with a different class count replaces it.
• Stage prints off, or behind a flag, at deployment.

When the usual answer is wrong

Three detours cost more than they save. Blaming the Java binding first wastes a day: the model file is the more common culprit, and one reference run settles it. Jumping straight to a newer OpenCV version can break a working stack elsewhere, calibration tools included; the OpenCV version conflicts guide lists what clashes and how to keep two projects apart. And rewriting working preprocessing to chase the crash changes inputs the network was trained for; preprocessing moves only when the format moves, and the outputs get re-validated.

Pin the trio for the life of the project. The SDK version pinning fix shows the mechanism: declared versions, one place, checked at startup instead of discovered at runtime.

If the pipeline now runs clean and the next step is putting it on the line, send the runtime and model pair through the contact page and we will spec the controller around it; free expert support comes with every order. More fixes in this family sit on the systems topic page.