Halfway through the epoch the run dies: RuntimeError from DefaultCPUAllocator, 'you tried to allocate 35896320000 bytes', raised at torch.vstack the moment validation starts. Host RAM climbed steadily all epoch and never came back, and a smaller batch size only slows the climb. If you are training a memory-bank anomaly detection model - PaDiM or PatchCore, in a framework such as anomalib - on a production dataset of thousands of normal images, nothing is installed wrong. The model is doing what it was designed to do: keep a feature vector for every position of every training image, and that arithmetic has caught up with the host. This guide computes the bank before you train, shows why batch size and VRAM are red herrings, and gives three fixes with the threshold where each applies.
The decision, up front
PaDiM, PatchCore and their relatives store what they learn: features, in host RAM, one entry per training image. The cost is linear in image count and untouched by the GPU. Nothing leaks; the design is the leak.
Decide by the shape of the dataset:
- More than about 1,000 normal images per class. Cap what enters the bank: about 300 images for PaDiM, a 10% coreset for PatchCore. Below that count the cap costs almost nothing in accuracy; above it, the stacking peak passes most workstations.
- Defects that span a few pixels in the resized image. Tile instead of resizing down - after capping the count.
- A dataset that keeps growing, or many classes per station. Switch to a flow or distillation model with no bank at all.
- Batch size and VRAM. Leave them alone; neither is in the failure path.
Why the bank grows and never shrinks
A memory-bank model does not optimize anything. The backbone is frozen; each training step pushes the batch through it, moves the feature maps to host RAM with a .cpu() call, and appends them to a list. No gradients, no optimizer step, nothing freed. The loss = nan in the progress bar is the same story: the training step returns nothing to optimize and the framework logs the absence. It is not a fault.
The list is consumed exactly once, at the start of validation. torch.vstack builds one contiguous copy of every stored tensor in a single allocation, the per-position mean and covariance are computed, and only then is the stack released. That is why RAM climbs all epoch, spikes to roughly twice the bank at validation, and never dips in between.
C is the concatenated channel count of the selected layers; ResNet-18 layers 1-3 give 64 + 128 + 256 = 448. The grid sits at the stride-4 layer, where the three maps are aligned - also the resolution of the anomaly map before it is upsampled to full size.
Worked example: 480 px, ResNet-18, a 683-step epoch
The inputs, each taken from the failing set-up itself:
- images resized to 480 × 480 px, so the grid is 480 / 4 = 120 per side, 120 × 120 = 14,400 positions
- C = 448 channels from ResNet-18 layers 1-3
- 4 bytes per float32 value
- 683 steps per epoch at 16 images per batch, from the training log: about 10,900 training images
The rule is to multiply, and the arithmetic runs:
- One image: 14,400 × 448 × 4 = 25,804,800 bytes, about 25.8 MB.
- A capped bank of 300 images: 300 × 25.8 MB = 7.7 GB.
- The full epoch: 10,900 × 25.8 MB ≈ 280 GB.
- The failing log asked for 35,896,320,000 bytes - 35.9 GB in one contiguous block, about 1,400 images' worth (35.9 GB ÷ 25.8 MB ≈ 1,390) - while the list still held its own copy, so the host faced roughly 72 GB of demand at that moment.
The decision it leads to: at this resolution the full epoch is past a 64 GB workstation before validation ever runs, and more RAM only moves the wall, because the bank holds about 40 images per gigabyte - 64 GB buys about 2,500 images. Cap the bank at 300 images and the same host trains with headroom.
Match the symptom to the cause
| Symptom | What it proves | Fix |
|---|---|---|
DefaultCPUAllocator failure at validation, tens of GB requested |
the feature bank outgrew host RAM | cap the bank or switch models |
| RAM climbs all epoch, run slows into swap near the end | same cause, close to the limit | the same cap; the memory will not clear on its own |
| CUDA out of memory during the forward pass | backbone or image too large for VRAM | smaller image, tiling, shorter backbone |
| Precision-recall buffer warning on large validation sets | metric objects keep every prediction in RAM | trim validation size or logged metrics |
Only the third row is one a bigger GPU answers. For the fourth, aligning the task and its metrics keeps validation memory proportionate to what the metrics actually use.
Fix 1: cap what enters the bank
PaDiM needs far fewer images than you have. The public industrial-defect benchmark categories these models were tuned on carry 60-400 normal images each, and the per-position mean and covariance converge well before a thousand. Sample about 300 representative normals per class and route the surplus to validation, where extra images genuinely sharpen the decision threshold. If the pipeline must see every image, store a random subset as you stream: a reservoir sample of 300 keeps the bank flat however long the line runs.
Cutting dimensions works the same way. The PaDiM paper randomly keeps 550 of the 1,792 channels ResNet-50 offers across the same three layers; apply the idea to ResNet-18's 448 and keep 100:
300 images × 14,400 positions × 100 channels × 4 bytes = 1.73 GB
PatchCore's answer is the coreset. With ResNet-18's middle layers at the stride-8 grid, the patch store is about 5.5 MB per image at 480 px - roughly 60 GB for this epoch, of which a 10% coreset keeps about 6 GB with little loss at the operating point the paper reports.
Banks are per class, so the host cost multiplies: eight classes at 7.7 GB each is 62 GB before any stacking peak. That is why training across multiple product classes starts with a per-class cap, not a bigger machine. After any cap, re-check pixel-level AUROC on held-out defect images; a small drop is the price of a bank that fits.
Fix 2: tile when detail, not count, is the problem
Tiling splits each frame into tiles and runs the extractor per tile. It buys resolution, not memory: one 256 px tile carries a 64 × 64 grid, about 7.3 MB of features, and four overlapping tiles total roughly 29 MB - a little more than the whole 480 px frame, with every tile another entry in the bank. Applied to a too-large dataset, tiling makes the failure arrive sooner. The usual advice is wrong exactly here.
The rule: tile when a defect spans fewer than about 8 pixels in the resized image. At the stride-4 grid each embedding position covers a 4 × 4 patch, so a defect a few pixels wide touches only a handful of positions, and no bank size recovers detail that resizing removed. Cap the image count first, then tile what remains.
Fix 3: switch to a model with no bank
Flow-based and distillation models fit parameters instead of storing features, so training memory stays roughly constant as the dataset grows. Choose one when the dataset keeps growing, when classes multiply, or when retraining is frequent enough that bank size decides the workflow. Normalizing-flow methods also target the cold-start case - a line with defect-free images only - which is the normal starting position in production.
What you trade is a different tuning loop: a training run with real convergence, thresholds to re-tune, and the discipline that comes with an optimizer. Once the model is settled, ONNX inference from anomaly maps to pass/fail moves the decision to the line, where smart cameras run it next to the part. The MV-SC6050C-08M-XBN is a 5 MP GigE smart camera with an 8 mm focal length lens and a multi-color light source.
- Interface
- GigE
- Resolution
- 5 MP
- Sensor format
- 1/1.45"
- Shutter
- Global
What does not help, and why
- A smaller batch size. The bank is N × bytes per image; batch size is not in the formula. It only changes how many steps the climb takes.
-
A bigger GPU. The stored features live on the host after the
.cpu()move, andDefaultCPUAllocatornever touches VRAM. A 24 GB card does nothing for a 36 GB stack. - More dataloader workers, normalization tweaks, new seeds. A gigabyte or two of noise around a 36 GB problem.
- Saving every validation image. Image saving and logging add validation-time buffers on exactly the run that is short of RAM; switch both off while you size the bank.
To size the host honestly - bank, stacking peak, dataloader and metric buffers together - the CPU and GPU checks for a deep learning set-up list what to measure, and why small models use large GPU memory covers the VRAM-side twin of this failure. If the training host also runs a control task, put training inside a cgroup or container memory limit, so an over-allocation stops the training job instead of the line.
Keep the capture side worth storing
A bank is only as good as the frames in it. Three hundred sharp, consistently lit normals beat ten thousand mixed ones, because the covariance should describe part variation, not exposure drift. Lock the exposure, lock the gain, fix one geometry - then collect the cap.
The capture hardware decides how comparable the frames are. The MV-CH120-11UC is a 12 MP USB3 camera with a global shutter, a 1.1-inch sensor and a C mount.
- Sensor format
- 1.1"
- Lens mount
- C-mount
- Interface
- USB3
- Resolution
- 12 MP
For inline capture, the LS-CMR-0166 is a 12 MP USB3 area scan camera at 30 fps.
- Sensor format
- 1/1.7"
- Interface
- USB3
- Resolution
- 12 MP
- Shutter
- Rolling
With frames in hand, the custom dataset folder and mask set-up is the piece to get right before any training run; more pieces like this one sit on the systems topic page.
Checklist before the next run
- Compute the bank in bytes - N × (H/4) × (W/4) × C × 4 - before training starts, and write the number down.
- Cap the class at about 300 images; route the surplus to validation.
- Cap channels (100 of 448) or apply a 10% coreset when several classes share one host.
- Tile only when defects span under about 8 px in the resized image, and only after capping the count.
- Plan for a validation peak near 2 × bank, and keep it under two-thirds of host RAM.
- Expect
loss = nan; it is the design, not a fault. - Set a cgroup memory limit on any host that also serves the line.
- Re-check pixel AUROC on held-out defects after every cap.
Where to go next
Choose the camera that will produce the next dataset in area scan cameras or the camera finder, size the whole station in the system configurator, and if the bank math needs a second opinion, write to us through contact - free expert support comes with every order.
Questions
Does lowering the batch size reduce the memory bank?
No. The bank grows with the number of training images, not with the batch size; a smaller batch only splits the same total across more steps, so the crash arrives at validation a little later. Batch size does change GPU memory during the forward pass, which is why lowering it feels like it helps. Leave train_batch_size alone and cap the images or channels that enter the bank instead.
How many normal images does PaDiM actually need?
Between 60 and 400 per class covers the benchmark categories these models were tuned on, and about 300 is the working number. The bank statistics - mean and covariance at each position - converge well before a thousand images, so extra normals buy very little accuracy while costing 25.8 MB each at 480 px with ResNet-18. Put the surplus images into the validation set, where they do improve the decision threshold.
Why does the crash land at validation instead of during training?
Training only appends each batch's features to a list, so memory climbs smoothly and nothing looks wrong. Validation is the first place the list is consumed: torch.vstack builds one contiguous copy of every stored tensor in a single allocation, and the original list is still alive during the copy. Peak demand is therefore about twice the bank, which is why a run that trained for half an hour dies the moment validation starts.
Is the loss = nan during training a second problem?
No, it is expected. Memory-bank models have no optimizer; the training step only extracts and stores features, so the framework logs a missing loss and warns that training_step returned None. The same run prints a warning about metric buffers for large datasets - that one is real but separate: prediction-logging metrics such as precision-recall curves hold every validation score in RAM.
Will a bigger GPU or more VRAM fix the allocation error?
Not this one. The stored features are moved to host RAM with a .cpu() call, and DefaultCPUAllocator is a host-side allocator, so a 24 GB card changes nothing about a 36 GB stack. More host RAM postpones the failure at about 40 images per gigabyte. GPU out-of-memory during PatchCore training is a different failure with a different fix: a smaller coreset, a shorter backbone, or tiling.