Camera logging memory growth: diagnose and bound log storage

A camera application that consumes more memory each hour does not necessarily have a heap leak. Repeated messages can fill a memory-backed filesystem, while formatting, writing, and retry activity consume processing time. This workflow is for engineers diagnosing long-running acquisition stations. You will separate process memory from log storage, estimate the growth rate, and introduce storage limits without losing the first useful explanation of a camera fault.

1. Identify which memory is growing

Start by distinguishing three measurements: application memory, filesystem consumption, and memory available to the host. A single overall memory percentage cannot identify the owner.

On Linux, a tmpfs filesystem stores file contents in memory and may use swap when configured. Do not assume /tmp uses it: inspect the filesystem containing the actual log destination. Containers and symbolic links can make the apparent path misleading.

Process resident memory and filesystem memory are not always independent accounting categories, particularly with mapped files. Avoid adding overlapping counters together. If private application memory is the growing component, follow the separate workflow for image buffer ownership and camera memory leaks.

2. Capture a baseline before changing settings

Record enough information to compare the same workload before and after a change. Start with fixed acquisition settings and a repeatable trigger source.

1. Record application, camera firmware, SDK, and logging configuration versions. Obtain any replacement packages from the manufacturer or its authorized distributors.
2. Identify every log destination, including application files, service output, system logs, and remote forwarding.
3. Record process private memory, resident memory, host available memory, filesystem usage, and swap activity where available.
4. Count messages by event type, rather than recording only the total number of lines.
5. Record received frames, acquisition timeouts, incomplete frames, queue depth, and processing latency.
6. Sample these measurements every 60 seconds initially, using a monotonic clock for elapsed intervals.

Useful Linux inspection commands include:

findmnt -T /tmp
df -h /tmp
df -i /tmp

These examples inspect /tmp; use the actual log destination when investigating your station. The first identifies its filesystem, while the others show space and inode consumption. They do not establish that the camera application writes there.

Keep the measurement log bounded too. Recording extensive diagnostics can otherwise create a second version of the original problem. For comparable CPU measurements, distinguish CPU time from wall-clock time.

3. Calculate whether log volume explains the growth

Measure a representative stored record, including its timestamp, severity, metadata, and line ending. Use an average across many records because exception reports can be much larger than ordinary messages.

For an illustrative workload, assume 30 frames/s, two repeated messages per frame, and an average stored record of 240 bytes:

This is an estimate for one destination before compression or additional indexing. A service manager that captures the same output can create another stored copy.

Now compare the estimate with measured directory growth. Similar slopes support the log-storage explanation, but do not prove that every lost byte of available memory belongs to logs. Other workloads may change simultaneously.

A growing file does not inherently make each subsequent append more expensive. Increasing CPU demand can instead reflect retry storms, formatting, rotation, compression, memory pressure, or an expanding asynchronous queue. Measure those mechanisms separately rather than assuming file size explains everything.

4. Use controlled tests to isolate logging cost

Change one variable at a time. Keep frame settings, image processing, trigger timing, and application versions unchanged during the initial comparison.

First, run the normal configuration long enough to establish its memory and message-rate slopes. Next, use a documented logging setting to reduce one identified repetitive diagnostic. Preserve error reporting and compare equal observation windows after startup activity settles.

Interpret the comparison carefully:

• If file growth stops but private memory continues rising, investigate allocations or queues.
• If CPU drops while frame delivery remains unchanged, logging overhead was contributing to processing demand.
• If missed frames improve, repeat the test before treating the relationship as causal.
• If invalid images continue, investigate the underlying acquisition fault even when the console becomes quiet.

A second test can move logs to a bounded, non-memory-backed destination with adequate write performance. This helps distinguish memory-backed storage consumption from message formatting and write overhead. It is a diagnostic comparison, not a reason to accept unlimited disk growth.

If an acquisition thread remains busy without receiving frames, investigate busy waiting in camera acquisition. If failures occur specifically when recording begins, use the workflow for external-trigger logging crashes.

5. Bound storage, repetition, and queued work

Use three separate controls. File rotation alone does not bound an in-memory writer queue, and rate limiting alone does not guarantee a storage ceiling.

Set a total storage budget

Choose a per-file limit, retained-file count, and explicit total allowance. For example, five retained files limited to 20 MiB each represent a nominal 100 MiB allocation. Confirm whether the logger counts the active file within those five files.

Allow additional space for rotation, temporary files, metadata, and other log destinations. If a strict ceiling is required, use a supported filesystem or service-level limit and define what happens when writes fail. Do not rely on compression achieving a particular ratio.

Use logger-supported rotation and retention. Deleting an open log file may not release its storage until the writer closes it. Uncoordinated external rotation can also leave the application writing somewhere unexpected.

Limit repeated messages without losing transitions

Record the first occurrence immediately. Aggregate identical repetitions and periodically emit their count and duration. For example, emit a summary every 30 seconds while keeping the total repetition count accurate.

Distinguish different fault identities. An exposure warning, transport timeout, and storage failure should not share one suppression bucket. Record recovery as its own state transition.

Bound asynchronous logging

Limit the writer queue by bytes where supported, or conservatively account for variable message sizes. Define whether overload drops lower-priority records, aggregates repetitions, or blocks producers.

Avoid an unbounded queue that converts slow storage into a heap-growth problem. Also avoid blocking the acquisition callback indefinitely. Track dropped-log counts so that quiet output is not mistaken for healthy operation.

6. Test disconnection without creating an error storm

A missing camera can trigger repeated connection attempts, each generating several messages. Storage limits must continue working during this condition, not only during normal acquisition.

Perform disconnection tests on an isolated test station or during an approved maintenance procedure. Do not interrupt a production camera while the machine depends on its output.

A useful connection-state record contains:

• The transition from connected to unavailable.
• The initial error category and relevant diagnostic context.
• Retry count and elapsed outage time.
• Periodic summaries rather than identical reports for every attempt.
• The transition back to connected and the outcome of acquisition restart.

Use documented retry intervals or capped backoff where supported. Do not implement a tight reconnect loop simply to reduce recovery latency.

Treat storage-full behavior as a separate test. The logger should expose the write failure without recursively logging that failure into the same broken destination. If recovery itself stalls, continue with acquisition recovery faults under load.

7. Verify the fix over a representative run

Repeat the original workload with the storage and repetition controls enabled. A brief startup check cannot validate a problem that previously appeared after many hours.

For example, if symptoms previously appeared around 15 hours, a 24-hour run provides additional observation time. It is not a universal acceptance standard; choose duration around the application's operating cycle and failure history.

Include several rotation events, a controlled outage, and recovery. Check that total storage reaches a plateau and that private memory stabilizes after normal initialization. A bounded sawtooth pattern from rotation differs from indefinite upward growth.

Calculate frame-loss rate from a trustworthy expected count:

For externally triggered acquisition, use an independently counted trigger total when available. Requested frame rate is not proof that every trigger occurred or that the camera accepted it. Separate intentional shutdown intervals from unexplained losses.

Compare latency distributions as well as averages. Rotation or compression can create occasional delays that disappear inside a mean value. Retain the measurements and configuration needed to reproduce the acceptance run.

8. Check hardware requirements only after isolating the cause

Additional memory can postpone an unbounded-growth failure without correcting it. Faster storage can reduce write delays while leaving retention uncontrolled.

For an existing station, document camera interface, processing demand, log budget, and required diagnostic retention before replacing anything. The 3D sensors category and camera finder help identify sensing options; neither establishes software compatibility or logging behavior.

The MV-DT01WMU is a ToF sensing camera. Confirm supported acquisition software and diagnostic controls for the intended application.

The MV-SVC2506-16G60-128G is a vision controller with camera ports. Verify operating-system support, storage configuration, and application resource requirements before selection.

Compare vision controllers against the measured workload rather than assuming a larger system will eliminate the fault.

9. Close out with a bounded-logging checklist

Before returning the station to service, confirm the following:

• The actual log destination and filesystem type are recorded.
• Every local sink has a retention policy and storage budget.
• Repeated events retain their first occurrence, count, and recovery transition.
• The writer queue cannot grow indefinitely.
• Storage exhaustion has a defined, tested response.
• Frame delivery and latency remain acceptable through rotation and recovery.
• Private memory and total log storage stabilize during the extended test.
• Temporary diagnostic verbosity is disabled after evidence collection.

Keep the configuration alongside the acceptance measurements so future software updates can be checked against the same conditions. If you need help matching acquisition hardware to the measured workload, send the interface, frame demand, runtime measurements, and diagnostic-retention requirement through application support.