3D visionTroubleshooting

3D camera SDK logs filling embedded storage: set the log sink

SDK logs from a 3D camera filling an embedded controller's storage: why the JSON log level cannot stop them, and the fix for each SDK version.

LumaSpec Vision8 min read Share on LinkedIn

On this page 10 sections
  1. 01Symptom: the flash fills and the files are not yours
  2. 02The one-minute test: whose files are these?
  3. 03Cause 1: the SDK's logging core writes files by default
  4. 04Cause 2: the pinned version predates the stdout default
  5. 05Worked example: SDK 1.1.1, two 3D cameras, Python 3.10
  6. 06When the usual answer is wrong
  7. 07What else eats the same flash
  8. 08Checks before you close the ticket
  9. 09Choosing the next platform so logging stays yours
  10. 10Questions

You run a 3D vision application in a container on an embedded controller, and the root filesystem fills within days. Your own log file is a few megabytes, so the space is going somewhere else. It is going to the camera's client SDK, which writes its own log files from inside your container. This troubleshooting guide separates the two logging systems in every 3D camera deployment, gives the two environment variables that stop the file writes on a glog-backed SDK, and maps the fix to the version pinned in your image. The case worked through is a common deployment: two 3D cameras, Python 3.10, the client library pinned at 1.1.1.

Symptom: the flash fills and the files are not yours

df -h shows the root partition climbing toward 100% over a few days. The files doing the damage are not in your application directory. They sit in a temp directory, named after the SDK's internal modules, and every running process adds its own set.

The camera's own configuration, read back through the SDK's config interface, shows this block:

"log": { "components": {}, "level": "warn", "storage": "volatile" }

That block is the wrong place to work. It configures the firmware logs written on the camera itself. The files filling your controller are written by the client library in your container, and that library never reads the camera's log level.

The one-minute test: whose files are these?

Stop your application container for ten minutes and watch the log directory with du -sm. Growth halts while the app is down and resumes when it starts, so the writer is the SDK inside your container. The camera cannot write to the controller's filesystem at all — "storage": "volatile" keeps the device's own logs in its RAM.

Changing "level" to "none" cannot fix this, even if the device accepts the value. Device firmware logs and client library logs are two separate systems. Plant software has the same split, which is why SDK messages can go missing from the platform's own message log even when every setting looks right.

Cause 1: the SDK's logging core writes files by default

A 3D camera SDK is usually a C++ core with Python bindings on top, and through the 1.x line this family used glog as its backend. glog writes files to disk unless told otherwise, and it reads its settings from environment variables once, when the library initializes.

Severity levels are numbered: 0 INFO, 1 WARNING, 2 ERROR, 3 FATAL. Two variables stop the files:

  • GLOG_logtostderr=1 sends messages to stderr instead of creating files.
  • GLOG_minloglevel=3 suppresses everything below FATAL.

Put them in the image, not in a shell you open later:

ENV GLOG_logtostderr=1
ENV GLOG_minloglevel=3

Rebuild, restart, and the INFO-level flood stops at the source. Messages land on stderr, which the container runtime collects with the rest of your output.

If you want glog's files during commissioning, point GLOG_log_dir at a directory on a tmpfs mount. The flash stays clean, but the cost moves to RAM, so treat that as a debug setting rather than a fix.

Cause 2: the pinned version predates the stdout default

The deeper cause is the pin. In this SDK family's changelog, release 1.3.2 replaced glog with a pluggable logger in the C++ core, and release 1.4.1 brought the same hook to the Python bindings. From 1.3.3 the default destination is stdout, so a fresh deployment writes no log files at all.

Client library in the image Logging behavior What stops the files
1.1.1 and the line up to 1.3.2 glog, file sink by default GLOG_logtostderr=1 with GLOG_minloglevel=3
1.3.2 (C++) / 1.4.1 (Python) pluggable logger Register your own logger and route messages into your rotating log
1.3.3 and newer stdout by default Nothing: rebuild on the newer pin and no files appear

Decision: if the image pins 1.1.1, ship the two ENV lines today and schedule the rebuild. If you can rebuild now, skip the variables and move the pin to a 1.3.3-or-newer release.

Rebuild with the pin in one place so every build agrees — the approach Docker build SDK version mismatch: pin one library version everywhere works through. Keep the library and the camera firmware on aligned releases, the failure mode 3D camera header version mismatch: fix firmware and SDK alignment covers. Re-run your capture tests after the jump, because an update can break a Python vision script at the API layer.

Worked example: SDK 1.1.1, two 3D cameras, Python 3.10

Inputs, each with its source:

  • Client library pinned at 1.1.1 in the container image — the glog era of this SDK family.
  • Two 3D cameras streaming, application in Python 3.10, root filesystem on the controller's flash.
  • Log directory size read with du -sm at the start and again exactly 1 h later. Example readings from a flood of this kind, with 2,048 MB free on the example controller: 212 MB, then 252 MB — substitute your own pair.

Rule in words: the hourly delta times 24 gives daily growth, and free space divided by daily growth gives the time to a full partition.

growth_per_day_MB = (du_after_MB - du_before_MB) / hours × 24 days_to_full = free_MB / growth_per_day_MB

Arithmetic: (252 − 212) MB over 1 h is 40 MB per hour, so 40 × 24 = 960 MB per day. With 2,048 MB free, the partition fills in 2048 / 960 ≈ 2.1 days. That is a controller that survives the weekend and fails midweek.

Fix applied: the two ENV lines go into the Dockerfile, the image is rebuilt, the container restarted. The same measurement over 1 h now shows 0 MB of new files, so 0 × 24 = 0 MB per day. The pass threshold is zero: any nonzero delta after the fix means the variables are not reaching the process that writes the files.

Continuous small writes also spend the flash's write endurance, not only its capacity, which is why the fix belongs at the sink and not in a deletion schedule. For the soak test itself, keep the cameras' emitters aimed away from the bench and check each unit's laser class marking before you extend the run.

When the usual answer is wrong

  • On 1.3.3 and newer the glog variables are inert: the backend that read them is gone. Leave them or remove them, but debug elsewhere.
  • Files still appear on 1.1.1 after the rebuild: either another binary in the image links glog as well — the variables govern every glog process — or the variables were set after init. Read the program name out of the file name to find the writer.
  • rm on a file the process still holds open frees no space. Truncate the file in place with truncate -s 0, or restart the writer, and expect the writes to resume until the sink itself is fixed.

What else eats the same flash

Triage in one line, largest directory last:

du -sm /tmp /var/log /app 2>/dev/null | sort -n

Checks before you close the ticket

  • One hour after the rebuild, the log directory delta is 0 MB.
  • docker logs shows no SDK messages below FATAL.
  • df holds flat across a full shift, not just the first hour.
  • The camera's config still shows its "log" block untouched — "warn", "volatile" — because the device side never needed a change.
  • The new pin lives in the image, not in a manual pip install inside a running container.

Choosing the next platform so logging stays yours

Make the log sink part of camera selection, not a post-launch surprise. Two questions sort any platform: where does the client library write by default, and can the sink be replaced with your own logger? Both answers sit in the SDK's changelog.

Browse the 3D sensors range or the camera finder, then take the shortlist through the system configurator. Two structured-light examples from the range:

For projected-pattern stations, the MV-DPS200P-02, a projected structured-light 3D camera:

3D sensorsHikrobot MV-DPS200P-02 Projected structured-light 3D cameraMV-DPS200P-02
Type
Stereo / structured light
$8,829.00 · quotedView and quote

For binocular stations, the MV-DLS300P-04, a laser galvanometer stereo camera:

3D sensorsHikrobot MV-DLS300P-04 Laser galvanometer stereo cameraMV-DLS300P-04
Type
Stereo / structured light
$16,579.00 · quotedView and quote

If the station is already built and you want a second opinion on the logging set-up, send the du readings and your image pin through the contact page.

Questions

Does setting the camera's log level to none stop the SDK log files?

No. That level sits in the camera's configuration tree and governs the firmware logs written on the device. The files filling your controller come from the client SDK inside your container, which never reads the camera's log level. Leave the device at "warn" with "volatile" storage and fix the sink on the controller side.

What do the numbers in GLOG_minloglevel mean and which should I pick?

The numbers are glog severities: 0 INFO, 1 WARNING, 2 ERROR, 3 FATAL. For production, 3 leaves only fatal messages, which a healthy deployment should never emit. During commissioning, 2 keeps an error trail on stderr without any files. Pick the level per stage in the image; a running process never re-reads it.

Where do the SDK's log files land when nothing is configured?

glog writes into a temp directory — /tmp on Linux — unless GLOG_log_dir points elsewhere, which is why a du aimed at /var/log finds nothing. Each process creates its own files, one per severity, with the program name and process ID embedded in the file name. That name is the fastest way to identify the writer.

Can I keep SDK messages in my own log file instead?

Yes. On 1.3.2 or newer in C++, and 1.4.1 in Python, register a custom logger that forwards into your logging module and let your rotation own the file. On older versions, use stderr: with GLOG_logtostderr=1 the container runtime captures the stream, and setting the json-file driver's max-size option caps it — the driver has no size limit by default.

Will moving the SDK to a newer version break my script?

It can: 1.1.1 to 1.3.3 spans a logging-backend replacement, and calls can change across that gap. Stage the move — build the new image, run it against one camera on the bench, and compare frame counts and decode rates before the fleet moves. Keep the camera firmware on an aligned release; mismatches there fail in different ways.