Build a visual SLAM feature vocabulary for a new dataset

A monocular SLAM pipeline can track nearby frames successfully yet fail to recognize a previously visited place. Its visual vocabulary may represent the original training imagery poorly, or its preprocessing may differ from the vocabulary-building pipeline. This guide explains how to prepare representative frames, train a compatible vocabulary, and validate loop-closure retrieval. It also separates vocabulary training from map creation, so you can decide whether rebuilding is necessary before collecting more data.

1. Decide whether you actually need a new vocabulary

A visual vocabulary groups local image descriptors into visual words. Each image becomes a compact word histogram that supports fast retrieval of potentially matching places. The vocabulary does not contain camera poses, metric scale, or a complete reconstruction.

You do not automatically need a new vocabulary for every dataset. Reuse can work when the descriptor family, preprocessing, vocabulary format, and scene appearance remain compatible. Rebuild when compatibility fails or held-out evaluation shows inadequate coverage.

Begin by identifying what the supplied asset contains. A saved file might package cluster centers, descriptor settings, word weights, a retrieval database, or several of these together. Its filename alone does not establish its contents.

Place retrieval is not the same as estimating motion or generating depth. The distinction in visual odometry versus depth mapping helps define which part of the pipeline you are changing.

2. Freeze the image and descriptor contract

Before training, write down the exact transformations between an acquired image and its descriptors. Vocabulary training and runtime matching must follow the same contract.

Record at least:

• Image dimensions and any crop or resize operation.
• Color-to-grayscale conversion and intensity representation.
• Whether images are raw or rectified.
• Feature detector, descriptor family, descriptor length, and data type.
• Pyramid settings, detection thresholds, and feature-count limits.
• Descriptor distance metric and vocabulary implementation version.

Matching dimensions are not enough. Two descriptors can have equal lengths while encoding different information. Their visual words are not interchangeable.

For binary descriptors, use a vocabulary builder designed for their distance metric. Do not convert packed descriptor bytes into arbitrary floating-point coordinates and assume ordinary Euclidean clustering remains meaningful. For floating-point descriptors, use the normalization and clustering approach supported by the selected implementation.

Resizing also changes feature appearance. If the runtime pipeline downsamples images, apply the same filtering during training; antialiasing before image downsampling explains why this affects repeatable image detail.

3. Prepare representative frames without evaluation leakage

Train on the operating conditions you expect, not merely the easiest frames to process. Include textured surfaces, low-texture areas, turns, approach and departure views, and the illumination changes present in deployment.

Avoid filling the training set with consecutive, nearly identical images. A stationary interval can contribute thousands of frames while adding little appearance diversity. Sample by traveled distance when reliable motion information exists, or combine time spacing with an image-change criterion.

For example, retaining one image every 2 seconds from a 30-minute recording gives approximately:

That is a sampling example, not a recommended universal interval. Fast turns may require closer spacing; stationary sections may require none.

Split recordings before selecting individual frames. An illustrative five-traversal dataset could use three traversals for vocabulary training, one for tuning, and one for final testing. Randomly splitting adjacent frames would make evaluation easier than deployment.

Keep two forms of overlap distinct:

• Vocabulary-training overlap: training descriptors accidentally include held-out evaluation imagery.
• Place overlap: query and reference images intentionally show the same location, allowing retrieval to be evaluated.

Place overlap is necessary for positive loop examples. It does not require training the vocabulary on those exact images. If you also need generalization to unfamiliar routes, evaluate that separately.

Reject corrupt frames, but do not quietly remove every difficult image. Preserve a labeled challenge set containing motion blur, exposure changes, repeated structures, and moving objects. If playback timing looks suspicious, check recording timestamp and playback errors before treating them as vocabulary failures.

4. Extract and balance the training descriptors

Run the frozen extractor over the training split and inspect its output before clustering. Record feature counts per image and the fraction of images producing too few descriptors for useful matching.

A training pool should not be dominated by one highly textured surface. Limit descriptors per image, balance contributions across traversals, and inspect where retained features occur. A wall of repeated markings may produce many descriptors without offering distinctive place information.

Consider an illustrative training set with 2,000 selected frames and a cap of 500 descriptors per frame:

The storage estimate assumes a 32-byte binary descriptor. It excludes feature locations, indexing structures, clustering workspace, and temporary copies. Other descriptor formats require different budgets.

Useful checks include:

1. Display feature locations on representative images from every traversal.
2. Compare feature-count distributions between training and held-out frames.
3. Inspect descriptors from moving objects and repeated patterns.
4. Confirm that sampling produces identical outputs when rerun with the same seed.
5. Measure extraction time independently of vocabulary-training time.

Low feature counts across an entire sequence usually point toward image quality or extractor settings. Increasing vocabulary size cannot recover detail that was never detected.

5. Train, weight, and save the vocabulary

Use the vocabulary builder supported by your SLAM or retrieval implementation. Its output must match the runtime reader, including tree structure, descriptor representation, word identifiers, and scoring conventions.

Some builders use a hierarchical tree. With branching factor 10 and depth 4, a fully populated tree has a nominal maximum of:

Actual populated words can be fewer. Some implementations assign words differently, so confirm what their depth and branching parameters mean.

A larger vocabulary can distinguish appearances more finely, but it also increases training cost and can fragment matches between changed viewpoints. A smaller vocabulary can merge unrelated descriptors into common words. Compare a few configurations on the tuning split rather than choosing by vocabulary size alone.

One common image representation combines term frequency with inverse document frequency:

Here, N is the number of images used to fit the weights, and df is the number containing that word. Implementations may smooth these expressions or normalize vectors differently.

For 40 occurrences among 1,000 assigned descriptors, TF is 0.04. If the word appears in 10 of 1,000 weighting images, IDF is ln(100) ≈ 4.61. The unnormalized weight is therefore approximately 0.184.

Save the vocabulary with its extractor configuration, preprocessing description, training split identifier, random seed, scoring settings, and software version. Keep a checksum so deployment can verify the intended artifact rather than relying on filenames.

6. Use the same representation during loop matching

At runtime, extract descriptors with the saved settings, assign them to the saved vocabulary, and construct the image representation using compatible weighting and normalization.

The retrieval database then stores these representations for reference keyframes. It is separate from the vocabulary itself. Building a new map usually means populating a new database, not necessarily retraining the visual words.

Exclude the current frame and an appropriate neighborhood of recent frames from loop candidates. Otherwise, high retrieval scores may simply reflect consecutive views rather than a genuine revisit. Choose the exclusion window from motion and keyframe behavior, not a fixed duration copied from another application.

Use retrieval to propose candidates, then verify them geometrically:

1. Match compatible descriptors between query and candidate images.
2. Apply the correspondence filters supported by the pipeline.
3. Estimate a geometric model appropriate to the scene and camera assumptions.
4. Check inlier count, inlier distribution, and model residuals.
5. Require consistency with the mapping system before accepting a loop constraint.

Repeated windows or shelving can produce strong word-histogram similarity without identifying the correct place. A high appearance score is not sufficient evidence for a map correction.

Monocular maps can also have scale drift. Vocabulary training does not establish metric scale; the separate discussion of monocular reconstruction scale ambiguity addresses that problem.

7. Validate coverage and accepted loops separately

Measure vocabulary coverage through retrieval behavior on held-out recordings. Visual words generally receive assignments even for unfamiliar descriptors, so a high assignment rate alone does not demonstrate useful coverage.

Inspect word occupancy, unusually dominant words, quantization-distance distributions, and retrieval errors. A shift in these diagnostics suggests a mismatch, but none independently proves that the vocabulary is unusable.

Create labeled queries with known reference locations and a stated tolerance for what counts as the same place. Include negative queries that have no valid match in the database.

In this example, retrieval misses ten positive queries. Verification loses another four. The first group suggests retrieval or image-representation work; the second suggests correspondence or geometry investigation.

Do not present a clean small test as proof that incorrect loop closures cannot occur. Inspect every accepted wrong match, expand the negative set, and repeat testing after parameter changes. Tune on validation data, then freeze settings before evaluating the final test split.

8. Stabilize acquisition before blaming the vocabulary

Vocabulary quality depends on repeatable image detail. Exposure, motion blur, lens coverage, and focus can change the descriptors more than another clustering run does.

When selecting area scan cameras, consider shutter behavior, required image detail, and acquisition compatibility. Use the camera finder to compare acquisition options rather than selecting by pixel count alone.

The MV-CU020-90GC is a global-shutter, color, GigE camera. Treat it as an acquisition candidate, not a guarantee of vocabulary performance.

Select machine vision lenses for the required view, sensor coverage, focus range, and mounting arrangement. The lens finder supports that screening.

The LS-FFL-0567 is a 16 mm, C-mount lens with a 1.1" sensor format. Confirm the assembled field of view, focus, mechanical clearance, and usable image detail before ordering a camera-and-lens combination.

If acquisition settings change dynamically, log those changes with the images. The guide to applying camera settings while streaming explains acquisition concerns that should be resolved separately from descriptor training.

For an existing recorded dataset, evaluate its available imagery first. Buying different hardware does not repair missing detail in already captured frames.

9. Commission the artifact and know when not to retrain

Before deployment, run this short acceptance checklist:

• Confirm the vocabulary loads through the production reader without conversion surprises.
• Verify the extractor settings and preprocessing against the saved manifest.
• Reproduce a fixed set of descriptor assignments and retrieval rankings.
• Check that recent-frame exclusion prevents trivial local matches.
• Inspect geometrically accepted loops, including visually repetitive locations.
• Measure latency and memory at the intended database size.
• Archive the previous working artifact and document rollback behavior.

Do not retrain first when the actual problem is descriptor incompatibility, incorrect image handling, an empty retrieval database, or failed geometric verification. Likewise, a dataset with few revisits cannot demonstrate strong loop-closure recall simply by growing the vocabulary.

The practical decision is evidence-based: retain a compatible vocabulary when held-out retrieval and verification meet your requirements; rebuild when representative training improves those measurements without unacceptable runtime cost.

For a new capture system, start with the camera configuration step. If you already have recordings, bring representative successful and failed frame pairs, extractor settings, and validation measurements to applications support before changing hardware.